These options affect how Rankley’s On‑Page crawler loads your website. They apply to new audits for a given account (and to Retry On‑Page when you re-run a failed crawl).
WHEN SHOULD I CHANGE THESE?
Try the default crawl first. If the On‑Page crawl returns 0 pages or shows statuses like 403/401/forbidden, adjust these settings and retry.
OPTIONS
Support cookies
What it does: Turns on cookie handling during the crawl.
Helpful when: The website requires a session cookie to see content or to move past a consent layer.
Try this when: You see a consent banner, region popup, or “please accept cookies” gate.
Possible side effects: Some sites serve different content when cookies are enabled.
Disable cookie popup
What it does: Attempts to suppress cookie/consent popups that block navigation.
Try this when: The crawler loads the page but can’t click past overlays.
Possible side effects: On some sites, the popup is part of the page logic and suppression may not help.
Switch proxy pool
What it does: Routes the crawl through a different network pool (helpful for aggressive WAF rules or geo/IP-based blocking).
Try this when: You consistently see 403/blocked or other WAF denials.
Possible side effects: IP/geolocation can change; WAFs may still block.
STILL BLOCKED?
If the site is protected by Cloudflare/WAF rules, you may need to allowlist Rankley’s crawler.
Read: Allow Rankley On‑Page Audit through Cloudflare WAF
RECOMMENDED WORKFLOW
Toggle one option at a time.
Click Save Defaults (account-level).
Click Retry On‑Page (latest report).
Wait a few minutes and re-open the report.
FAQ
Do these change PageSpeed or GBP audits?
No — these settings are for On‑Page crawling only.
Do these apply to all future audits?
Yes, for that specific account.