OpenPoly logo
Production Readiness

Operational runbook

Operator runbook for incidents, manual review, and support workflows.

Operational runbook

Use this runbook for day-2 operations after launch.

Routine review

AreaInspectAction
Opinion wallet cashCash Flow Triage, Balance Ops, original wallet ledgerInvestigate unresolved and failed-but-unpaid items, including old records without a creation-date cutoff.
Resolved Opinion marketsMarket PnLCompare confirmed cash, quoted fees, pending counts, and failed-payout debt; do not subtract included fees twice.
PolyflightPurchases, ticket results, obligations, native reconciliationKeep final game results separate from payout, loss acknowledgement, refund, and compensation delivery.
CallbacksWebhook Deliveries and endpoint configuration/historyInvestigate failed/dead-letter attempts and missing subscriptions; use bounded recovery after fixing the receiver.
Month-endOpinion reports and Polyflight statementsReview each product separately; resolve blocked cash evidence before exporting or treating totals as final.
Staff/supportOperator Support, staff grants, and credential ownershipFollow up shared conversations and remove access no longer needed.

Track API auth errors, wallet timeouts, entry complaints, pending principal closures, and unresolved liabilities. A green connection check, a 2xx callback, or a closed support ticket is not proof of financial reconciliation.

First-response flow

  1. Identify environment, product, and exact operator user; keep case and meaningful whitespace in external identifiers.
  2. Capture the UTC incident window and first observed failure; report timezone separately for local-date reports.
  3. Locate idempotency_key, event_id, Opinion order/redemption/market ID, or Polyflight purchase/ticket/flight/obligation ID.
  4. Separate unknown money, final product state, callback delivery, and statement impact.
  5. Preserve evidence and stop manual financial retries. Do not reproduce a money mutation just to collect a cleaner error.

Wallet incidents

If balance operation is unknown:

  1. inspect the operator wallet ledger using the same idempotency_key
  2. confirm whether the original mutation landed
  3. do not replay a primary debit or credit automatically
  4. send sanitized evidence through Operator Support for an authorized Polynion outcome decision; operators cannot resolve wallet outcomes from the dashboard

If balance operation is manual_review:

  • gather operator-side wallet logs
  • gather related trade or redemption record
  • escalate with exact IDs

For Polyflight, reconcile wallet evidence separately from the public outcome. A won or refunded ticket keeps that result while payout/refund is owed. Operators provide evidence; only an authorized Polynion administrator can record manual resolution. Then confirm the related event, liability, statement, and export changed once.

For Opinion, expand the Cash Flow Triage record to inspect wallet attempts and immutable resolution history. Access needs trades:read, redemptions:read, and balance_operations:read. A Polynion completed/failed decision records evidence; it does not send money. Failed-but-unpaid amounts remain debt and can still block settlement. Reconcile the original operation, not a replacement key.

If the wallet echoes external_user_id, idempotency_key, currency_code, or amount_minor, a non-null mismatch is contradictory evidence. Preserve both request and response; never reinterpret it as success because the HTTP response was 2xx.

Webhook incidents

  1. Inspect the exact product/user/event, endpoint, latest status, attempt timestamps, and response sample.
  2. Fix the receiver, authentication, signature/raw-body handling, or subscription/configuration issue first.
  3. Choose recovery: replay an existing delivery, or select a bounded historical backfill/bulk resend when deliveries are missing or need requeueing.
  4. Review selected target IDs, current endpoint configuration, and the confirmation before queueing. There is no resend dry-run API. Keep receiver event_id deduplication active, including for previously accepted 2xx events.
  5. Observe the queued attempt's eventual outcome. Acceptance of a replay/resend request is not proof the receiver has accepted it.

Delivery recovery does not repeat a trade, settle an unknown wallet result, recreate a deleted business event, or prove a user received money. A payload preview is reconstructed evidence, not the original signed request bytes.

Key rotation runbook

  1. Create a replacement key with the required workload scopes only.
  2. Store it server-side and update only the intended environment/workload.
  3. Confirm authentication with a permitted read-only endpoint under the team's release procedure; do not use a wallet mutation as a key probe.
  4. Revoke the old key after cutover and monitor authentication failures.
  5. Rotate wallet authentication and webhook signing credentials as separate changes; preserve historical evidence and same-key financial deduplication.

Month-end and support handoff

Opinion report dates use the operator settlement timezone. Current-month previews are not final/exportable; incomplete fee or payout evidence can block completed-month exports. Pending principal closures disclose missing recognition rather than new cash. Polyflight has separate native statements, adjustments, and payment history; neither product's report proves a bank transfer.

Open one Operator Support conversation with IDs, UTC timestamps, exact expected/observed amounts, currency, and sanitized ledger evidence. Follow up there, refresh for replies, and reopen a closed ticket if unresolved. Do not send credentials or treat support status as authorization to retry payment.

Copyright © 2026