Webhook Integration taste of a reliability-focused integrations engineer

• Verify every incoming webhook with an HMAC signature over the raw request body before parsing it as JSON. Confidence: 0.95
• Compare signatures with a constant-time function, never with a plain string equality check. Confidence: 0.95
• Reject deliveries whose signed timestamp is older than about five minutes to block replay attacks. Confidence: 0.90
• Respond with a 2xx status within a few seconds and do the real work in a background job. Confidence: 0.95
• Persist the raw event first, then process it, so nothing is lost if a worker crashes mid-way. Confidence: 0.90
• Deduplicate on the provider's event ID with a unique index, because every provider retries and delivers at least once. Confidence: 0.95
• Never assume events arrive in order, and compare a version or updated_at field before overwriting local state. Confidence: 0.85
• Treat the payload as a hint and refetch the current object from the provider's API for anything money-related. Confidence: 0.80
• Return 4xx only for permanently bad requests and 5xx for temporary failures so the sender's retries behave correctly. Confidence: 0.85
• Support at least two active signing secrets at once so secrets can be rotated without downtime. Confidence: 0.85
• When sending webhooks, retry with exponential backoff and jitter for up to a day or more, then disable the endpoint and notify its owner. Confidence: 0.90
• Send a thin, versioned payload with an event type, event ID, timestamp and resource ID instead of dumping whole database rows. Confidence: 0.80
• Name event types as resource.action in the past tense, like invoice.paid or user.deleted. Confidence: 0.80
• Give customers a delivery log with request, response, status and a manual replay button. Confidence: 0.85
• Block outgoing webhook URLs that resolve to private, loopback or link-local addresses to prevent SSRF. Confidence: 0.95
• Use separate, short timeouts for outgoing deliveries and never let one slow customer endpoint block a shared queue. Confidence: 0.90
• Require HTTPS for all webhook endpoints outside local development. Confidence: 0.90
• Offer a reconciliation API or periodic sync so consumers can recover from missed events. Confidence: 0.80
• Test handlers locally with the provider's CLI or a tunnel, and keep recorded sample payloads as test fixtures. Confidence: 0.80
• Alert on rising failure rates and growing backlogs per endpoint, not only on individual errors. Confidence: 0.80
• Never log full payloads that contain personal data, secrets or card details. Confidence: 0.90