Firewall Whitelisting

Identifying Sleepwalker

Sleepwalker identifies itself with the following User-Agent string:

The key identifier is Sleepwalker-SEO-Spider/1.0 appended to a standard Chrome user agent.

Additionally, requests from Sleepwalker include the Referer header: https://www.sleepwalker.ai

Why Whitelist?

Sleepwalker needs to access your pages to analyze content for AI visibility optimization. If your firewall or CDN blocks our requests, you'll see "Access Denied" errors and Content Intelligence won't work.

AI Visibility tests can still run when blocked - they query AI platforms directly, not your website. Only Content Intelligence requires page access.

Cloudflare

To whitelist Sleepwalker in Cloudflare:

1. Go to Security → WAF → Custom rules
2. Click Create rule
3. Set the rule name (e.g., "Allow Sleepwalker")
4. Under "When incoming requests match...", select:
   User Agent → contains → Sleepwalker-SEO-Spider
5. Set action to Skip and check all security features to bypass
6. Click Deploy

AWS WAF

To whitelist in AWS WAF:

1. Go to AWS WAF → Web ACLs and select your ACL
2. Click Add rules → Add my own rules
3. Choose Rule builder
4. Set condition: Header → User-Agent → Contains string → Sleepwalker-SEO-Spider
5. Set action to Allow
6. Set rule priority to run before blocking rules

Other Firewalls

For other WAFs and firewalls (Akamai, Sucuri, Wordfence, etc.), create a rule that:

• Matches User-Agent containing Sleepwalker-SEO-Spider
• Allows the request or bypasses bot protection
• Has higher priority than blocking rules

robots.txt

You can also use robots.txt to explicitly allow Sleepwalker. Add the following to your robots.txt:

User-agent: Sleepwalker-SEO-Spider
Allow: /

Note: robots.txt only works if your WAF respects it. Most WAFs block before robots.txt is checked.

Need Help?

If you're still having trouble, contact us at info@sleepwalker.ai with your domain and the error you're seeing.