Secure webhooks: signatures, retries and processing

Read the specific sender contract

Webhooks have no universal signature format. The provider defines which bytes are signed, where the signature appears, whether a timestamp is included and how deliveries are repeated. An example from another provider can therefore be wrong at the very first check.

Your integration needs a small contract: allowed event types, test/production account used, signature mechanism, key rotation, response deadline, retry rules and status reconciliation for failed delivery. Subscribe only to events you actually process.

Even on an email platform, a webhook is initially an API event, such as a delivery-status event. Its signature is not an email DKIM signature or proof of inbox placement. The email foundations cover that other layer.

Verify the bytes actually signed

In GitHub's documented HMAC mechanism, the webhook secret and request body produce a verification value. The recipient compares it with X-Hub-Signature-256. A suitable library compares without data-dependent early exits. GitHub: Validating webhook deliveries.

Use the original data required by the mechanism. Parsing JSON and then serialising it again can change whitespace or ordering. The result may contain semantically identical fields but different bytes. Stripe explicitly requires the unchanged request body for signature verification. Stripe: Webhooks.

A deliberately small local example illustrates the principle. It demonstrates only HMAC and byte identity, not a production-ready webhook endpoint:

import hashlib
import hmac

secret = b"demo-only-not-a-production-secret"
raw_body = b'{"event_id":"demo-17","state":"ready"}'

signature = hmac.new(secret, raw_body, hashlib.sha256).hexdigest()
same_body = hmac.compare_digest(
    signature,
    hmac.new(secret, raw_body, hashlib.sha256).hexdigest(),
)
changed_body = hmac.compare_digest(
    signature,
    hmac.new(secret, raw_body + b" ", hashlib.sha256).hexdigest(),
)

assert same_body is True
assert changed_body is False

The extra space changes the verification value even though it may not affect JSON meaning. A real recipient must also check header format, permitted algorithm, missing/duplicate fields and maximum body size under its contract. Prefer the appropriate official library and establish middleware order before a general JSON parser changes the body. Never copy the fixed demo secret into a real service.

Distinguish freshness from repeat delivery

An old, validly signed message may be delivered again. Some mechanisms therefore include a signed timestamp. The recipient checks it within provider-appropriate tolerance and needs a reliable clock. An arbitrary unsigned Date header does not provide this property.

Stripe documents such timing and generates new time/signature values for another delivery attempt. Its documented tolerance is provider-specific. Successful time validation prevents neither two rapid repeats within the window nor duplicate processing of an event redelivered by the provider. Stripe: Replay attacks.

GitHub uses the body for the described HMAC check. X-GitHub-Delivery may remain the same on redelivery, but is not automatically protected by that body signature. Do not treat every freely supplied identifier alone as cryptographic replay evidence. Check the mechanism's concrete properties and additionally protect the business effect. GitHub: Best practices.

Secure acceptance durably, then respond promptly

Lengthy work directly in the reception path may exceed the response deadline. A workable design inserts an entry into a durable inbox table or queue after successful signature and basic validation. Only after reliable storage does the server confirm acceptance. If storage fails, a positive response must not pretend the job was accepted.

‘Durably capture’ means more than an ephemeral in-memory task. If the process stops immediately after the HTTP response, outstanding work must remain clear. A known repeat may be acknowledged positively again under the contract if its earlier acceptance is certain.

The provider determines the response deadline; GitHub specifies ten seconds, for example. Retry rules also differ: do not rely solely on the assumption that the sender will automatically redeliver every lost event indefinitely. GitHub: Best practices.

Identify delivery, event and business action separately

Several identifiers help deduplication, but are not interchangeable:

Protect reception through reliable storage uniqueness. Also store processing state. ‘Already seen’ must not mean a job that crashed after acceptance is never processed again. Processing therefore needs a controlled resumption path.

When applying the effect, check business association again: does the external account match the stored integration? Does the referenced object belong to this tenant? Is the state transition allowed? A valid partner message must not modify arbitrary records belonging to others based on a freely supplied internal object ID.

Plan for delays and missing events

Rely on ordering only if the specific contract promises it for your processing path. Stripe, for example, does not guarantee delivery in creation order. A later-arriving ‘created’ event must not blindly reset an already confirmed completion. Stripe: Event ordering.

Process transitions against current stored state and business rules. Provider version numbers or sequences with appropriate meaning may help. A timestamp alone is no reliable global ordering without a corresponding contract.

Some events work better as triggers for an authenticated status query. That call still needs correct account association and the rules for external APIs. Plan reconciliation for failed deliveries and persistently open jobs. ‘No message’ does not prove nothing happened.

Test key rotation and failures in a controlled way

Separate secrets by environment and integration. Keep them outside source code. Planned rotation may need a limited transition supporting old and new verification, if the provider allows it. An exposed secret needs a different assessment: continued acceptance would continue permitting forged messages. Blocking, redelivery and event reconciliation must then be planned together.

Test with synthetic signed messages: valid and altered bodies, unknown integration, permitted repeat, processing after process failure, reversed delivery order and a full inbox buffer. Where the contract includes signed timestamps, add old and impermissibly future values. Success is a correct, single business effect in the tested case, not a general ‘exactly once’ promise for distributed systems.

The HMAC example can be tested entirely locally without network calls. It proves neither correct middleware integration nor durable queue acceptance, replay defence or provider compatibility. These require dedicated integration tests. Rolling back the new receiver version must preserve accepted events and their processing state.

The API Security Check does not test these signed, stateful flows. Observe signature errors, delays and open events in your own logging and monitoring; also use the appropriate idempotency contract for duplicate jobs.

Next steps