Skip to main content

​
Overview

This guide walks you through setting up SEO Crawler from account creation to your first crawl results. By the end, you’ll have a complete picture of your website’s link health.
Time to complete: ~5 minutes (plus DNS propagation time for domain verification)

​
Getting Started

1

Create Your Account

Visit app.seocrawler.app and sign up with your email or continue with Google/GitHub authentication.
Start with our Free plan to explore the platform. You can upgrade anytime to unlock more features.
2

Add Your First Domain

From your dashboard, click Add Domain and enter your website URL.
example.com
Enter just the domain name without https:// or trailing slashes. We’ll automatically detect whether your site uses HTTP or HTTPS.
3

Verify Domain Ownership

Domain verification ensures you have permission to crawl the domain. Add a DNS TXT record to prove ownership:Record Type: TXT
Host/Name: _seocrawler-verify
Value: seocrawler-verification=<your-token>
  1. Log in to your Cloudflare dashboard
  2. Select your domain
  3. Go to DNS → Records
  4. Click Add record
  5. Select Type: TXT
  6. Name: _seocrawler-verify
  7. Content: Your verification token
  8. Click Save
  1. Sign in to your GoDaddy account
  2. Go to My Products → DNS
  3. Click Add under Records
  4. Select Type: TXT
  5. Name: _seocrawler-verify
  6. Value: Your verification token
  7. TTL: 1 Hour
  8. Click Save
  1. Log in to Namecheap
  2. Go to Domain List → Manage
  3. Click Advanced DNS
  4. Click Add New Record
  5. Type: TXT Record
  6. Host: _seocrawler-verify
  7. Value: Your verification token
  8. Click the checkmark to save
DNS changes can take up to 48 hours to propagate, though most changes are visible within 15 minutes. Click Verify to check status.
4

Configure Scan Options

Before starting your crawl, choose which analyses to run:
OptionDescriptionPlan Required
SSL CheckVerify certificate validity and expirationFree
SEO AnalysisAudit meta tags, headings, and imagesSolo+
Anchor Text AnalysisEvaluate link text qualityPro+
Response TrendsTrack performance over timePro+
For your first crawl, we recommend enabling SSL Check and SEO Analysis to get a comprehensive baseline of your site’s health.
5

Start Your First Crawl

Click Start Crawl to begin. Our crawler will:
  1. Start from your homepage
  2. Discover and follow all internal links
  3. Check each URL for errors and issues
  4. Analyze pages based on your selected options
Crawl time depends on your site size and selected options. A 100-page site typically completes in 2-3 minutes.
6

Review Your Results

Once complete, you’ll see a comprehensive report including:
  • Summary Dashboard: Overall health score and key metrics
  • Broken Links: All 4xx and 5xx errors with source pages
  • Redirects: Redirect chains and potential issues
  • SEO Issues: Missing meta tags, headings, and alt text
  • Performance: Response time distribution
Click any issue to see details and recommended fixes.

​
What’s Next?

Understanding Your Results

Learn how to interpret crawl data and prioritize fixes.

Scan Options Deep Dive

Configure advanced crawl settings for your needs.

Schedule Automated Crawls

Set up recurring crawls to catch issues early.

Share Reports

Generate shareable reports for stakeholders.

​
Need Help?

Verification Issues?

Common DNS verification problems and solutions.

Contact Support

Our team typically responds within 2 hours.