Webhook drills

Operators should prove receiver behavior before production.

Preconditions

• webhook endpoint created in operator dashboard or operator API
• receiver secret stored on operator side
• signature verification enabled
• dedupe store enabled on event_id

Required drills

1. Success drill

• subscribe endpoint to the relevant Opinion trade and/or Polyflight purchase/payout/refund events
• trigger a real staging product action; purchase.processed confirms a completed stake debit, payout.pending or refund.pending records an obligation rather than credited cash
• confirm receiver returns 2xx
• confirm delivery status becomes sent
• confirm event stored exactly once by event_id

2. Bad signature drill

• temporarily use wrong secret on receiver side
• trigger event
• confirm receiver returns 401
• confirm delivery status becomes failed
• restore correct secret
• replay delivery
• confirm replay becomes sent

3. Retry drill

• configure the separate operator simulator receiver fail_next mode (or an equivalent operator receiver that returns 500 once)
• trigger event
• confirm first delivery attempt returns 500
• wait for retry window
• confirm later attempt becomes sent
• confirm business effect applied once only

4. Dead-letter drill

• configure receiver to keep failing with 500
• trigger event
• let retries exhaust
• confirm delivery status becomes dead_letter
• fix receiver
• replay delivery
• confirm latest attempt becomes sent

5. Duplicate protection drill

• replay an already acknowledged 2xx event
• confirm receiver returns 2xx for the same event_id without reapplying any effect (409 causes retries)
• confirm no duplicate business side effect recorded

6. Historical backfill and bulk resend drill

• Use staging-only event records and endpoints. Capture a terminal no_endpoint, endpoint_disabled, or not_subscribed event-only row, then create/enable/subscribe the receiver. Confirm configuration change alone does not send the historical event.
• Open the full delivery monitor with event date basis and All time (or the original event date), not delivery date. Select the event-only row, review all current active subscribers, and confirm Resend.
• Confirm real pending deliveries are created, signed POSTs use the original public event_id and occurrence time, and eventual receiver acknowledgements appear as retained attempts. Use the new delivery-created time only when reviewing delivery-time filters.
• Select two or more eligible rows on the same page for Bulk Resend. A rejected/ineligible target must queue none of the batch. Inspect current state before resubmitting after a timeout.
• For history_unavailable, do not invent prior HTTP evidence; explicit resend may duplicate earlier receipt and requires durable dedupe. Do not delete a real endpoint merely to manufacture this condition: deletion removes delivery history.
• Confirm already-pending rows cannot be queued again and successful/unselected sibling deliveries are not repeated by automatic dispatch.

7. Endpoint setup history drill

• Record the original URL/events/policy before a staging change; edit through the authenticated dashboard/API.
• Inspect Setup History for UTC recording time, authenticated actor, and before/after settings. No secret value/hash should appear.
• If a deletion is part of the approved staging drill, preserve needed delivery evidence first. Confirm setup history survives while delivery rows do not.
• Treat any baseline as capture-time settings with an unknown earlier actor, not a reconstructed audit of old changes.

Built-in receiver scenarios

The dashboard's staging test receiver exposes ok and always-500. The separate operator simulator callback receiver supports off, fail_next, and always_fail drill modes. The simulator's Ensure Beacon Endpoint action creates or reactivates its endpoint and synchronizes its receiver signing secret; it does not overwrite configured event selections or disable other callback URLs.

Use the relevant receiver only in staging, with its own secret and operator access. For Polyflight, confirm all eight event types are selectable and compare the immutable minor-unit stake/fee/net stake/potential payout snapshot with pending obligations versus confirmed wallet credits.

Evidence to capture

• event_id
• event_type
• first attempt timestamp UTC
• latest attempt timestamp UTC
• last status code
• delivery final status
• receiver log entry showing dedupe result
• row kind, numeric outbox event.id, public string event.event_id, endpoint ID and current URL
• saved setup-history record ID for any configuration change; keep it separate from HTTP delivery evidence

Pass criteria

• success path reaches sent
• transient failure retries without duplicate side effects
• persistent failure reaches dead_letter
• replay works after fix
• bad signature rejected before business processing
• historical resend creates actual signed callbacks only after explicit selection; no wallet operation or new business event is created
• list totals and all-filtered CSV agree with the same applied date basis, product and identifiers; reconstructed JSON is not signed request evidence