When a website is protected by Cloudflare, strong security settings can sometimes block important crawlers such as Meta and Google. This can create serious problems for app reviews, privacy policy validation, and search engine indexing.
A common situation is when a business submits an app to Meta and receives a warning that the privacy policy is not publicly accessible. At the same time, Google may fail to index product pages because they return a 403 Forbidden response to crawlers. In many cases, the website owner has not broken anything on purpose. The issue comes from security settings such as Under Attack Mode, strict WAF behavior, or bot blocking rules.
This article explains, in simple terms, how to keep Cloudflare protection enabled for most of a site while allowing specific public pages like /gdpr/ and /produs/ to remain accessible to Meta and Google.
Why Meta rejects a privacy policy URL
Meta requires the privacy policy URL to be public, reachable, and easy for its systems to fetch. If Cloudflare shows a challenge page, blocks bots, or returns a 403 response, Meta may treat the link as inaccessible.
This often happens when:
- the whole website is in Under Attack Mode
- Cloudflare bot protections are too aggressive
- the privacy policy page is accidentally geo-blocked
- WAF rules challenge crawler traffic
- the URL works in a browser for a person, but not for automated validation systems
In practice, the privacy policy page must behave like a normal public web page.
Why Google may stop indexing product pages
Googlebot needs to access product pages without being blocked. If a product URL under /produs/ returns a 403 error, Google may stop indexing that content.
This can happen when security rules are applied globally across the whole site, even though product pages are public content and should remain accessible.
Typical affected URLs look like this:
https://example.com/produs/sample-product/
If those pages are challenged or blocked, indexing suffers.
The main Cloudflare mistake to avoid
A very common configuration mistake is trying to use a Cloudflare expression inside the visual rule builder value box, for example:
starts_with(http.request.uri.path, "/produs/")
This is wrong if the rule builder is set to something like:
- Field: URI Full
- Operator: wildcard
In that case, Cloudflare treats the text as a literal wildcard value, not as an expression.
The correct way to match /produs/ URLs
There are two clean methods.
Option 1: Use the visual rule builder
Set the rule like this:
- Field: URI Path
- Operator: starts with
- Value:
/produs/
This matches all product pages under that path.
Option 2: Use expression mode
If you click Edit expression, use:
starts_with(http.request.uri.path, "/produs/")
This is the cleanest method for product pages.
The correct way to match the privacy policy page
For a privacy policy located at:
https://example.com/gdpr/
you can use one of these approaches.
If you want to match the entire section
Use:
starts_with(http.request.uri.path, "/gdpr/")
If you want to match only that exact page
Use:
http.request.uri.path eq "/gdpr/"
If the privacy policy is a single page, exact match is often the safest choice.
Best Cloudflare setup for public pages
A practical strategy is to keep security enabled for the whole site, but create exceptions for pages that must be publicly crawlable.
For example:
/gdpr/for privacy policy validation/produs/for product indexing
This allows you to protect login pages, admin panels, APIs, and sensitive areas more aggressively while leaving public content accessible.
Rule idea for the privacy policy path
For the privacy page, create a rule that removes challenge behavior from:
/gdpr/
This helps Meta access the URL normally.
Rule idea for product pages
For product pages, create a rule that removes challenge behavior from:
/produs/
This helps Google access and index those URLs.
What to disable for those paths
For public crawlable content, you usually want to remove or relax the protections that cause crawler failures, such as:
- Under Attack challenge pages
- overly aggressive WAF behavior
- rate limiting if it affects normal page fetching
- geographic restrictions on public content
The goal is not to remove security from the whole website. The goal is to remove only the layers that block normal access for public pages.
What not to do
Do not use:
- URI Full with a wildcard value containing a Cloudflare function
- expressions pasted into the value field of the visual builder
- overly broad “allow everything” rules across the whole domain
- crawler blocking on pages that must be public
Also avoid weakening the security of sensitive paths such as:
/wp-admin/- login pages
- checkout pages
- account pages
- backend APIs
Example logic for a safer public/private split
A simple idea is:
- keep stronger protection for admin and dynamic endpoints
- reduce challenge behavior for public informational pages
- make sure product pages and privacy pages return normal responses
This gives a better balance between security and discoverability.
How to test whether the fix works
After changing Cloudflare rules, test in this order:
- Open the URL in an incognito window.
- Confirm there is no Cloudflare challenge page.
- Confirm the page loads normally.
- Check that the URL does not return 403.
- Use Meta’s debugger for the privacy policy page.
- Use Google Search Console URL Inspection for a product page.
This helps verify whether the crawler issue is solved.
Signs that the Cloudflare rule is correct
Your rule is probably correct if:
/gdpr/opens normally without challenge/produs/...pages open normally without 403- product pages become crawlable again
- Meta no longer reports the privacy policy URL as inaccessible
Dummy-level explanation
Think of Cloudflare like a security guard at the entrance of a building.
If the guard stops everyone, then even invited guests such as Meta and Google cannot enter.
The solution is not to remove the guard from the whole building. The solution is to tell the guard:
- let visitors enter the reception area and product showroom
- keep strict checks for private offices and restricted rooms
In website terms:
/gdpr/and/produs/are public areas- admin, login, and backend sections are restricted areas
That is the cleanest way to protect the site without breaking app validation and SEO.
Final recommendation
If your privacy policy is at /gdpr/ and your product pages are under /produs/, configure Cloudflare so those paths are matched by URI Path rules, not by incorrect wildcard patterns on URI Full.
For /produs/, use:
starts_with(http.request.uri.path, "/produs/")
For a single privacy policy page at /gdpr/, use either:
http.request.uri.path eq "/gdpr/"
or
starts_with(http.request.uri.path, "/gdpr/")
Then relax the challenge and blocking behavior only for those paths, while keeping stronger protection elsewhere on the site.
That approach helps restore Meta privacy policy access and improves Google indexing without exposing the entire website.
