Crawlux for developers and platforms.
Reference documentation for the Crawlux audit JSON output (live today), webhook payloads (live today) and the planned Crawlux REST API (shipping Q4 2026). Plus integration patterns for agencies, SEO platforms and CRM systems.
Current status and roadmap
Three categories of developer-facing functionality. Two are live today. One ships Q4 2026.
Every audit produces a JSON output downloadable from the dashboard. Free, Pro and Team tiers all include JSON. Structure documented in section 2.
Configure a webhook URL in the dashboard. Crawlux POSTs the full audit JSON to the URL on completion. Pro and Team tiers only. Documented in section 3.
Programmatic audit triggering, status polling, result retrieval and webhook configuration. Reference documented in sections 4 to 8 below.
Typed SDK wrappers for the REST API. Node.js (npm: crawlux-sdk), Python (pip: crawlux). Documented in section 8.
Early access waitlist
The API ships Q4 2026 but early access opens roughly 2 weeks before public launch. Email [email protected] with subject "API Waitlist" to join. Early access is free and includes API documentation review.
Audit JSON output structure
Every Crawlux audit produces a JSON file downloadable from the dashboard. The structure is stable and versioned. Below: the top-level shape and key fields.
"audit_id": "aud_8f3k29vJ7nQ",
"version": "v3",
"domain": "example-protocol.io",
"tier": "team",
"started_at": "2026-04-29T08:14:22Z",
"completed_at": "2026-04-29T08:16:47Z",
"site_category": "defi_protocol",
// Overall score and grade
"score": {
"overall": 67,
"grade": "C+",
"percentile": 58
},
// Per-check-group breakdown
"groups": [
{ "id": "technical", "score": 82, "weight": 0.15 },
{ "id": "schema", "score": 45, "weight": 0.20 },
{ "id": "ymyl", "score": 71, "weight": 0.15 },
{ "id": "ai_visibility", "score": 58, "weight": 0.20 },
{ "id": "authority", "score": 76, "weight": 0.15 },
{ "id": "on_chain", "score": 74, "weight": 0.15 }
],
// Findings array (truncated, full audit returns all 23 analyzers)
"findings": [
{
"analyzer": "B01",
"severity": "high",
"title": "Token page uses generic Product schema",
"recommendation": "Migrate to FinancialProduct schema",
"reference": "crawlux.com/guides/token-schema/"
}
]The JSON output is the system of record. Both the dashboard view and the white-label PDF report are generated from this JSON. Storing the JSON alongside your domain gives you full audit history.
Key fields explained
Unique audit identifier. Format: aud_ followed by 12 alphanumeric characters. Use this to retrieve the audit later via dashboard or (when API ships) the GET /audits endpoint.
Methodology version used for this audit. Reviewed quarterly. Pinning allows reproducible historical audits across version updates.
Auto-detected category. Values: defi_protocol, exchange, wallet, nft_platform, infrastructure, identity, generic_crypto. Determines which prompt set runs for AEO testing.
Analyzer code (A01 through F03, see the methodology page). Maps to the 23 analyzers across 6 check groups. Each finding cites the specific analyzer that fired.
Webhook payloads
Configure a webhook URL in the dashboard. When an audit completes, Crawlux POSTs the full audit JSON to the configured URL. Useful for triggering downstream automation, populating CRM records or alerting agency teams.
// Headers
Content-Type: application/json
X-Crawlux-Event: audit.completed
X-Crawlux-Signature: sha256=a4b8c92e...
X-Crawlux-Audit-Id: aud_8f3k29vJ7nQ
X-Crawlux-Delivery: del_2HxJ4nP3kqM
// Body (full audit JSON, structure as shown in section 2)
{ "audit_id": "aud_8f3k29vJ7nQ", ... }The webhook signature is HMAC-SHA256 computed using a shared secret configured in the dashboard. Always verify the signature before processing the payload.
Fired when an audit finishes successfully. Payload contains the full audit JSON. Most common event for downstream automation.
Fired when an audit cannot complete. Payload contains the audit_id, the error code and the error message. Common causes: domain unreachable, robots.txt disallow, rate limit hit.
Fired when a failed audit gets automatically retried. Up to 3 retries with exponential backoff. Final failure produces audit.failed instead.
Webhook delivery guarantees
Crawlux retries webhook delivery up to 5 times over 24 hours if your endpoint returns non-2xx status. Always return a 2xx status quickly (under 5 seconds) and process the payload asynchronously.
Planned API endpoints
The Crawlux REST API ships Q4 2026. Below: the planned endpoint surface. Subject to refinement during the early access period. Base URL: https://api.crawlux.com/v1/.
Trigger a new audit for a domain. Body accepts domain (required), tier (free/pro/team), webhook_url (optional). Returns the audit_id and audit status.
Retrieve a specific audit by ID. Returns the full audit JSON if completed, or status object (queued, running, failed) if not yet complete.
List all audits in your workspace. Supports pagination via limit and cursor parameters. Filter by status, domain, date range. Returns audit summaries.
Retrieve only the findings array for an audit. Lighter response than the full audit JSON. Useful for surfacing specific findings in dashboards.
Download the white-label PDF report (Pro and Team tiers only). Returns a signed URL valid for 60 minutes. Free tier audits do not have PDF reports.
Configure a webhook endpoint. Body accepts url (required), events (array of event types to subscribe to), secret (for HMAC signature).
Remove a webhook configuration. Subsequent audit completions will not POST to the removed webhook URL. Existing audits are not affected.
Retrieve current period usage. Returns audits used this period, audits remaining, period reset date and tier limits. Useful for usage dashboards.
Authentication
The API will use Bearer token authentication. API keys are generated per workspace from the Crawlux dashboard. Each key gets scoped to a workspace and configurable rate limits.
Authorization: Bearer crl_live_8f3k29vJ7nQa4b8c92eP3kqM
Content-Type: application/jsonAPI keys come in two environments: crl_test_ for sandbox testing without consuming audit quota and crl_live_ for production audits that consume tier quota. Webhook signatures use a separate signing secret configured per webhook.
Test mode is free
Test mode keys (crl_test_) return realistic but synthetic audit data without consuming quota. Useful for building integrations without paying for real audits during development.
Rate limits
Rate limits scale with tier. Audit quota is the binding constraint; result retrieval calls are unlimited within reason.
| Tier | Audits / month | Result retrieval | Use case |
|---|---|---|---|
| Free | 5 | Unlimited | Personal projects, evaluation, single-domain teams |
| Pro | 50 | Unlimited | Solo agencies, in-house SEO leads, multi-project consultants |
| Team | 200 | Unlimited | Marketing agencies, multi-tenant platforms, larger SEO teams |
| Enterprise | Custom | Unlimited | Dedicated infrastructure, SLA, custom integration support |
Rate limit headers are returned on every API response. X-RateLimit-Limit shows your tier limit. X-RateLimit-Remaining shows audits remaining. X-RateLimit-Reset is the Unix timestamp when the limit resets (start of next month).
Error codes
Errors return JSON with a code, a message and a request_id for support requests. HTTP status codes follow standard REST conventions.
{
"error": {
"code": "audit_quota_exceeded",
"message": "Tier quota exceeded for the current period",
"request_id": "req_2HxJ4nP3kqM8f3k",
"docs_url": "https://www.crawlux.com/docs/#errors"
}
}Domain format invalid or domain unreachable. Common causes: missing protocol, typo, DNS not propagated, domain blocked by registrar.
API key missing, malformed or revoked. Check the Authorization header format. Regenerate the key from the dashboard if compromised.
The requested feature requires a higher tier. Common case: webhook configuration on free tier (Pro or Team required), PDF download on free tier (Pro or Team required).
Monthly audit quota exhausted for the current tier. Wait until the period resets or upgrade tier. Result retrieval calls are not affected.
A required upstream data provider is unavailable (DataForSEO, CoinGecko, DefiLlama, PageSpeed). Audit will be retried automatically. Status page covers provider outages.
SDKs and client libraries
Two officially supported SDKs ship with the Q4 2026 API launch. Community SDKs in other languages are welcome and will be linked from this page.
import { Crawlux } from 'crawlux-sdk';
const crawlux = new Crawlux({ apiKey: process.env.CRAWLUX_API_KEY });
// Trigger an audit and wait for completion
const audit = await crawlux.audits.run({
domain: 'example-protocol.io',
tier: 'team',
waitForCompletion: true
});
console.log(audit.score.overall); // 67
console.log(audit.findings.length); // 23from crawlux import Crawlux
client = Crawlux(api_key=os.environ["CRAWLUX_API_KEY"])
# Trigger an audit and wait for completion
audit = client.audits.run(
domain="example-protocol.io",
tier="team",
wait_for_completion=True
)
print(audit.score.overall) # 67
print(len(audit.findings)) # 23Both SDKs provide typed wrappers around the REST API plus helpers for common patterns: polling for audit completion, signature verification on incoming webhooks, batch audit triggers across multiple domains.
Common integration patterns
Six integration patterns we see most often when teams discuss the API early access. Each pattern has a recommended endpoint sequence.
Agency dashboard integration
Trigger audits per client domain on demand. POST /audits with webhook_url pointing to your dashboard. Process the audit.completed webhook to populate client reporting views.
Quarterly cadence automation
Cron job triggers audits every 90 days per managed domain. POST /audits with webhook_url to trigger downstream alerts on score regression. Compare current to previous via stored audit JSON.
CRM enrichment
When a new lead enters CRM, trigger a free-tier audit on their domain. Populate the lead record with score and top 3 findings. Use as conversation starter for sales calls.
Slack alerting
Webhook endpoint posts to Slack when audit completes. Format the score and top 3 findings into a Block Kit message. Useful for client-facing channels and internal SEO leads.
SEO platform embed
White-label Crawlux audits inside your own SEO platform UI. Use the test API key (crl_test_) during build. Display findings in your own UI keyed by analyzer code.
Bulk domain monitoring
Trigger audits across hundreds of managed domains overnight. Batch the POST /audits calls with sleep between to respect rate limits. Aggregate results via your own reporting layer.
Common developer questions
Six questions covering the API roadmap, JSON output, authentication, SDKs and rate limits.
When does the Crawlux API ship?
The Crawlux API is planned for Q4 2026. Currently audits run via the Crawlux web dashboard with JSON output downloadable per audit. The API will allow programmatic audit triggering, status polling, webhook callbacks and result retrieval. Subscribe via the docs early-access waitlist to be notified when API access opens.
Can I get audit JSON output today?
Yes. Every Crawlux audit produces a JSON output downloadable from the audit dashboard. The JSON structure is documented on this page. Pro and Team tier audits also produce a white-label PDF report. Free tier audits produce a JSON output and a summary HTML report.
What authentication will the API use?
The API will use API key authentication via Bearer tokens. API keys will be generated from the Crawlux dashboard. Each API key gets scoped to a workspace and has configurable rate limits. Webhook callbacks will be authenticated via HMAC-SHA256 signatures using a shared secret.
Will SDKs be available?
Two SDKs are planned for the Q4 2026 launch. A Node.js SDK published to npm under the package name crawlux-sdk. A Python SDK published to PyPI under the package name crawlux. Both SDKs will provide typed wrappers around the REST API plus helpers for common patterns like polling for audit completion.
What rate limits will the API have?
Rate limits will scale with tier. Free tier: 5 audits per month. Pro tier: 50 audits per month. Team tier: 200 audits per month. Enterprise tier (custom pricing): unlimited audits with dedicated infrastructure. All tiers include unlimited result retrieval calls. Webhook callbacks count separately and are not rate-limited.
How can I get notified when the API ships?
Three ways. First, email [email protected] with subject API Waitlist to join the early access list. Second, follow the changelog page for product updates. Third, run a free audit and reply to any of the email follow-ups confirming developer interest. Early access opens approximately 2 weeks before public API launch.
Run a free audit and download the JSON
The fastest way to evaluate the audit JSON for integration: run a real audit on your own domain and inspect the output. Free first audit per domain.
JSON output live · Webhooks live · API Q4 2026 · 2-week early access window