Skip to main content

​
Webhooks

Configure webhooks to receive real-time notifications when crawl events occur. Available Events:
  • crawl.started — A crawl has begun
  • crawl.completed — A crawl finished successfully
  • crawl.failed — A crawl encountered an error
  • broken_links.found — Broken links were discovered

​
Create Webhook

​
/webhooks
Register a new webhook endpoint. Request Body
FieldTypeRequiredDescription
urlstringYesHTTPS endpoint URL
namestringNoFriendly name
eventsarrayNoEvents to subscribe to
curl -X POST https://seocrawler.app/api/v1/webhooks \
  -H "Authorization: Bearer sc_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-server.com/webhook",
    "events": ["crawl.completed", "broken_links.found"]
  }'
The secret is only returned once. Save it immediately for signature verification.

​
List Webhooks

​
/webhooks
Retrieve all configured webhooks. 
curl https://seocrawler.app/api/v1/webhooks \
  -H "Authorization: Bearer sc_your_api_key"

​
Delete Webhook

​
/webhooks/{id}
Remove a webhook endpoint. 
curl -X DELETE https://seocrawler.app/api/v1/webhooks/webhook-uuid \
  -H "Authorization: Bearer sc_your_api_key"

​
Test Webhook

​
/webhooks/{id}/test
Send a test payload to verify your endpoint is working correctly. 
curl -X POST https://seocrawler.app/api/v1/webhooks/webhook-uuid/test \
  -H "Authorization: Bearer sc_your_api_key"

​
Webhook Payloads

All webhooks include a signature header for verification: 
X-SEOCrawler-Signature: t=1699574400,v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd

​
crawl.completed

Sent when a crawl finishes successfully. 
{
  "event": "crawl.completed",
  "timestamp": "2024-01-15T10:32:45Z",
  "data": {
    "crawl_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
    "domain_name": "example.com",
    "total_links": 150,
    "broken_links": 3,
    "duration_seconds": 165
  }
}

​
crawl.failed

Sent when a crawl encounters an error. 
{
  "event": "crawl.failed",
  "timestamp": "2024-01-15T10:35:00Z",
  "data": {
    "crawl_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
    "domain_name": "example.com",
    "error": "Timeout: crawl exceeded maximum duration"
  }
}
Sent when broken links are discovered during a crawl. 
{
  "event": "broken_links.found",
  "timestamp": "2024-01-15T10:32:45Z",
  "data": {
    "crawl_id": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
    "domain_name": "example.com",
    "broken_count": 3,
    "sample_links": [
      { "url": "https://example.com/missing", "status_code": 404 }
    ]
  }
}

​
Verifying Signatures

Verify webhook authenticity by checking the signature header. 
const crypto = require('crypto');

function verifySignature(payload, signature, secret) {
  const [t, v1] = signature.split(',').map(p => p.split('=')[1]);
  
  // Check timestamp (5 min tolerance)
  if (Math.abs(Date.now() / 1000 - parseInt(t)) > 300) return false;
  
  // Verify signature
  const expected = crypto
    .createHmac('sha256', secret)
    .update(`${t}.${payload}`)
    .digest('hex');
  
  return crypto.timingSafeEqual(Buffer.from(v1), Buffer.from(expected));
}
The signature includes a timestamp to prevent replay attacks. Reject any webhook with a timestamp older than 5 minutes.