Skip to main content

On‑Page Crawl Option: Switch proxy pool

Routes the On‑Page crawl through a different proxy/network pool to reduce IP-based blocking. Use this when WAF/CDN rules (like Cloudflare) deny requests with 403/blocked responses or when crawls return 0 pages.

Updated over a month ago

WHAT THIS DOES

Routes the On‑Page crawl through a different proxy/network pool.

This can help when a site blocks certain IP ranges, applies aggressive bot rules, or enforces geo/IP-based restrictions.

WHEN TO ENABLE IT

  • You see 403/blocked or WAF denials.

  • The crawl consistently returns 0 pages and you suspect IP-based blocking.

NOTES

  • Switching proxy pools won’t bypass allowlist-only policies.

  • If the site uses Cloudflare/WAF rules, you may still need to allowlist Rankley’s crawler.

HOW TO USE IT

  1. Go to the Account → On‑Page Crawl Defaults.

  2. Enable Switch proxy pool.

  3. Click Save Defaults.

  4. Click Retry On‑Page (or run a new audit).

Did this answer your question?