Webhook payloads and signature verification.

// How it works

Lifecycle of a webhook delivery.

An audit completes. Crawlux generates the audit JSON (documented in the audit JSON reference). The webhook system POSTs the JSON to your configured URL with an HMAC-SHA256 signature header. Your endpoint returns a 2xx status within 5 seconds to acknowledge receipt. If your endpoint returns non-2xx or times out, Crawlux retries up to 5 times over 24 hours with exponential backoff. Final failure after 5 retries marks the delivery as failed in your dashboard but does not affect the audit itself.

// Section 01 · Payload shape

Headers and body.

Every webhook POST carries five Crawlux headers plus the standard Content-Type. The body is the full audit JSON, identical to what you would download from the dashboard.

// Headers
Content-Type: application/json
X-Crawlux-Event: audit.completed
X-Crawlux-Signature: sha256=a4b8c92e3f1d7b6a8c5e9f2d1a4b7c8e9f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7
X-Crawlux-Audit-Id: aud_8f3k29vJ7nQ
X-Crawlux-Delivery: del_2HxJ4nP3kqM
X-Crawlux-Timestamp: 1714377407

// Body (full audit JSON, see /docs/audit-json/ for the full schema)
{
  "audit_id": "aud_8f3k29vJ7nQ",
  "version": "v3",
  "domain": "example-protocol.io",
  ...
}

// Section 02 · Signature verification

Always verify the signature before processing.

The X-Crawlux-Signature header is an HMAC-SHA256 computed using a shared secret you configured in the dashboard. The secret is the signing key; the message is the raw request body. Verify the signature before trusting the payload, otherwise anyone can POST forged audit JSON to your endpoint.

const crypto = require("crypto");

function verifyCrawluxSignature(rawBody, signatureHeader, secret) {
  const expected = crypto
    .createHmac("sha256", secret)
    .update(rawBody)
    .digest("hex");
  const received = signatureHeader.replace("sha256=", "");
  return crypto.timingSafeEqual(
    Buffer.from(expected, "hex"),
    Buffer.from(received, "hex")
  );
}

// In your Express handler
app.post("/webhooks/crawlux", (req, res) => {
  const sig = req.headers["x-crawlux-signature"];
  if (!verifyCrawluxSignature(req.rawBody, sig, process.env.CRAWLUX_WEBHOOK_SECRET)) {
    return res.status(401).send("invalid signature");
  }
  // Process the payload asynchronously and return 200 immediately
  queue.add("process-audit", req.body);
  res.status(200).send("ok");
});

// Section 03 · Events

Three event types fire on webhooks.

// Section 04 · Delivery guarantees

Reliability and retry semantics.

Crawlux retries webhook delivery up to 5 times over 24 hours if your endpoint returns non-2xx status or fails to respond within 5 seconds. Retry schedule uses exponential backoff: roughly 1 minute, 5 minutes, 30 minutes, 2 hours, then 6 hours. After 5 failed attempts the delivery is marked failed in your dashboard. You can manually retry from the dashboard for 7 days after the original event.

Always return a 2xx status quickly and process the payload asynchronously. Synchronous processing inside the webhook handler risks timeout, retry storms and duplicate processing. Recommended pattern: validate the signature, push the payload to a queue (Bull, SQS, Redis), return 200, process from the queue.

// Section 05 · Configuration

How to set up webhooks.

Webhook configuration lives in the Crawlux dashboard under Settings → Webhooks. Pro and Team tiers can configure unlimited webhooks per workspace. Each webhook gets: a URL, an array of event types to subscribe to, and a signing secret (auto-generated, displayed once at creation).

When the API ships in Q4 2026, webhooks can also be configured programmatically via POST /webhooks. Until then, configuration is dashboard-only.

// Related docs

Where to go from here.

• Audit JSON output structure — full reference for the payload body
• Error reference — for audit.failed event payloads
• REST API reference — Q4 2026 endpoint surface including POST /webhooks
• Docs overview — full developer reference index