Overview
Domains are the foundation of SEO Crawler. Each domain represents a website you want to monitor for broken links, SEO issues, and performance problems.Adding Your First Domain
Navigate to Domains
From your dashboard, click Domains in the sidebar, then click the Add Domain button.
Enter Domain Name
Enter your domain name in the input field:
Enter just the domain name. We’ll automatically strip any
https://, www., ports, or paths you include.Input Normalization
SEO Crawler automatically cleans up your domain input:| You Enter | We Use |
|---|---|
https://mysite.com | mysite.com |
www.mysite.com | mysite.com |
mysite.com/page | mysite.com |
mysite.com:8080 | mysite.com |
MYSITE.COM | mysite.com |
https://www.mysite.com/page?q=1 | mysite.com |
Domain Requirements
Supported Domain Types
| Type | Example | Supported |
|---|---|---|
| Root domain | mysite.com | ✅ |
| Subdomain | blog.mysite.com | ✅ |
| Deep subdomain | app.us.mysite.com | ✅ |
| With numbers | site123.com | ✅ |
| With hyphens | my-cool-site.com | ✅ |
Not Supported
The following domain types cannot be monitored:| Type | Example | Reason |
|---|---|---|
| IP addresses | 192.168.1.1 | Must be a domain name |
| Localhost | localhost:3000 | Not publicly accessible |
| Local network | mypc.local | Not publicly accessible |
| Internal domains | server.internal | Not publicly accessible |
| Tor hidden services | site.onion | Not accessible via standard HTTP |
| Reserved domains | example.com | IANA reserved for documentation |
Plan Limits
| Plan | Max Domains |
|---|---|
| Free | 1 |
| Solo | 3 |
| Pro | 10 |
| Agency | Unlimited |
Validation Error Messages
When adding a domain, you may see specific error messages:IP addresses cannot be monitored
IP addresses cannot be monitored
You entered an IP address like
192.168.1.1 or 127.0.0.1. Please enter a domain name like mysite.com instead.Localhost cannot be monitored
Localhost cannot be monitored
You entered
localhost or a localhost subdomain. Please enter a publicly accessible domain that SEO Crawler can reach over the internet.Reserved example domain
Reserved example domain
You entered
example.com, example.org, or similar IANA-reserved domains. These are reserved for documentation purposes. Please enter your actual domain.Reserved test domain
Reserved test domain
You entered
test.com or a .test domain. Please enter your actual production domain.Local/internal network domains
Local/internal network domains
You entered a domain ending in
.local, .internal, .lan, .home, or .corp. These are private network domains that aren’t publicly accessible.Domain contains invalid characters
Domain contains invalid characters
Your domain has invalid formatting like double dots (
..). Please check for typos.Domain part cannot start/end with hyphen
Domain part cannot start/end with hyphen
Domain labels (parts between dots) cannot begin or end with a hyphen. For example,
-mysite.com and mysite-.com are invalid.Domain already added
Domain already added
This exact domain is already in your account. Check your domain list to find it.
Domain limit reached
Domain limit reached
You’ve reached the maximum number of domains for your plan. Upgrade your plan or remove unused domains to add more.
After Adding a Domain
Once you add a domain, you’ll need to verify ownership before you can start crawling:- DNS Verification - Add a TXT record to prove you own the domain
- Wait for Propagation - DNS changes can take up to 48 hours (usually much faster)
- Verify - Click “Check DNS” to confirm verification
- Start Crawling - Once verified, run your first crawl
Managing Domains
Domain Dashboard
Your domain list shows:| Column | Description |
|---|---|
| Domain | Domain name and verification status |
| Health | Overall health score based on latest crawl |
| Last Crawl | When the domain was last crawled |
| Actions | Start crawl, view details, settings |
Domain Details
Click on any domain to see:- Overview - Health score, total links, broken links count
- Crawl History - List of all previous crawls with results
- Settings - Domain configuration and danger zone
Removing Domains
To remove a domain:Troubleshooting
I can't add my domain
I can't add my domain
Check that your domain:
- Is a valid domain format (e.g.,
mysite.com) - Is not an IP address or localhost
- Is not a reserved domain like
example.com - Doesn’t use a blocked TLD like
.localor.internal - You haven’t reached your plan’s domain limit
My domain shows as unverified
My domain shows as unverified
You need to complete DNS verification before crawling. Go to your domain’s verification page and follow the instructions to add a TXT record.
I accidentally added the wrong domain
I accidentally added the wrong domain
You can delete the domain from the domain details page and add the correct one. If you’ve already verified it, you’ll need to verify the new domain as well.