Recent Posts
    Reliable Webhook Delivery

5 tips for reliable webhook processing

May 23, 2022

When you're building a webhook integration to receive and process webhook messages it can sometimes be hard to know where to start or if you are making the best decisions. With a few tips you can get start strong and make it easier to maintaining the integration and develop new features.

webhook data extraction

1. Develop the webhook processing locally

Webhooks are sent from a publisher to another public website. They cannot connect directly to your development server on your laptop, which can make development slow. So you don't have to wait for your deployment cycle to test change use a tool to tunnels the messages to your development server. I recommend ngrok which can be used via the command line to provide a secure tunnel.

ngrok http 80

2. Unit test your webhook parsing

Most errors processing webhooks come from decoding the webhook message. Unit test decoding the messages so when you see a message in a new variation you can make code changes with confidence that your not accidentally breaking anything.

Exactly how to unit test will depend on your programming language and framework. If the messages are very large, consider saving them to their file to keep your unit test code small, but it's perfectly reasonable to just paste the large example strings in your code. Just take care not to commit anything sensitive like a PII or keys.

3. Log decoding errors

webhook by the road

Since most errors process webhooks are about the message format make sure you are logging those errors well. Include details like exactly which field had the decoding error and what it was. Consider these two error messages...

Vague error message

Error: BadRequest

Descriptive error message

Error: BadRequest invalid timestamp, unable to decode "23 May 2022"

With the second you know that it's the timestamp field that cant be decoded, and you can see the date format it's encountered. You likely need to extend your date parsing to support the new format.

4. Monitor your webhook processing

If there is a problem with your webhook processing, you want to be able to find out about it and fix it. Your alert can be anything that works for you an email or pager work well. The key is that it's an automated way that tells you there is a problem that needs investigating. You should monitor for errors, latency and backlog.

It's not uncommon for webhook messages to have bursty traffic causing big spikes in load and causing delays processing. But you don't want to get in a situation where you cant keep up with the new messages.

You will need either your own alerting infrastructure or you could use Webhook Wizard's built-in monitoring.

5. Idempotent, repeatable webhook message processing

Sometimes the same webhook message will be sent to you as a duplicate, your system needs to be able to support that. If you're incrementing any counters, you need to deduplicate the messages. Usually, webhook messages come with an ID or a timestamp you can deduplicate by.

Webhook messages are sometimes repeated if there was a problem sending the message. It's safer to resend and support duplicate messages than to not resend and miss the message entirely.

webhook synchronize Learn more about Webhook Wizard