Do GPTBot and ClaudeBot evaluate CSP?

No. Both are raw HTML fetchers and ignore CSP. Only crawlers that render JavaScript (a subset of OAI-SearchBot, PerplexityBot, browsing agents) honor CSP.

Will a strict CSP block AI crawlers from citing my page?

Not if your site renders correctly in a real browser under the same policy. Strict CSP based on nonces or hashes is the recommended approach by web.dev and OWASP and is fully compatible with rendering crawlers.

Should I keep X-Frame-Options if I set frame-ancestors?

For modern browsers, no. Per MDN, frame-ancestors supersedes X-Frame-Options. Keep X-Frame-Options only if you must support legacy IE 11 traffic.

How do I generate a nonce per request?

Generate at least 128 bits of randomness (16 bytes) per request, base64-encode it, set it in the Content-Security-Policy header, and attach it to inline scripts and styles. Most frameworks expose a nonce via middleware.

What is 'strict-dynamic'?

A CSP keyword that lets a trusted (nonced or hashed) script propagate trust to scripts it loads. It removes the need to allowlist every host your bundle depends on and is the recommended modern approach per web.dev.

Where do CSP reports go?

To the URL specified by report-uri (legacy) or the group named in report-to plus a Report-To header. Many teams use Sentry, Report URI, or a custom endpoint that buckets violations by directive and user agent.

CSP Policy AI Crawler Allowlist Specification

A Content-Security-Policy that uses nonce- or hash-based script-src (strict CSP) is fully compatible with AI crawlers because crawlers that render JavaScript honor the same nonces a real browser does; the failure modes are missing nonces on hydration scripts, blocked third-party fonts, and frame-ancestors 'none' accidentally applied to embeddable card surfaces.

TL;DR

Ship a strict CSP based on nonces or hashes, run Content-Security-Policy-Report-Only for one full crawl cycle, and verify that bot-class user agents trigger zero violations. Replace X-Frame-Options with frame-ancestors. AI crawlers that fetch raw HTML ignore CSP; AI crawlers that render JavaScript respect it identically to a browser.

Scope

This specification covers the Content-Security-Policy header for sites that want to remain citation-eligible by AI search engines. It applies to canonical content pages, server-rendered apps, single-page apps with hydration, and CDN-edge configurations.

It does not cover Trusted Types, COEP/COOP, or Permissions-Policy; those are addressed in the Security Headers Configuration for AI Crawlers reference.

How AI crawlers interact with CSP

CSP is a browser-side enforcement mechanism. It only takes effect when an AI crawler renders the page in a headless browser. Three classes of AI traffic apply:

Raw fetchers (GPTBot, ClaudeBot, Google-Extended, most training crawlers) ignore CSP entirely. They parse HTML and never execute scripts.
Rendering crawlers (subset of OAI-SearchBot, PerplexityBot, ChatGPT user-triggered, browsing agents) execute JavaScript. They honor CSP exactly as a Chromium browser does.
Agent / tool bots behave like full browsers and respect every directive including report-only.

The practical implication: a CSP that breaks hydration in a real browser will also break it for rendering crawlers, and your content will be invisible in answer surfaces.

Directive matrix for AI crawler compatibility

Directive	Recommended value	AI crawler note
default-src	'self'	Safe baseline; rendering crawlers fetch same-origin assets first.
script-src	'self' 'nonce-{random}' 'strict-dynamic'	Strict CSP per web.dev. Crawlers honor nonces.
style-src	'self' 'nonce-{random}'	Avoid 'unsafe-inline'. Use nonces for critical CSS.
img-src	'self' data: https:	Permissive https: allows third-party CDN images, including embedded video thumbnails AI surfaces depend on.
font-src	'self' https://fonts.gstatic.com	Blocked fonts cause render layout shifts; rendering crawlers may flag the page low quality.
connect-src	'self' https://api.example.com	Allow any fetch endpoints needed for hydration. Bots time out on blocked XHR.
frame-ancestors	'none' on auth pages; 'self' or specific allowlist elsewhere	Per MDN, supersedes X-Frame-Options. Safe for AI crawlers; affects only browser embeds.
base-uri	'self'	No effect on crawlers.
form-action	'self'	No effect on crawlers (they don't submit forms).
upgrade-insecure-requests	Enabled	Helps crawlers that follow protocol upgrades.
report-uri / report-to	Endpoint URL	Required for report-only mode debugging.

Strict CSP recipe

Follow the OWASP strict-CSP recommendation (OWASP, 2025) and Google's web.dev guide. The minimum policy:

Content-Security-Policy:
default-src 'self';
script-src 'self' 'nonce-r4nd0m' 'strict-dynamic' https:;
style-src 'self' 'nonce-r4nd0m';
img-src 'self' data: https:;
font-src 'self' https://fonts.gstatic.com;
connect-src 'self' https://api.example.com;
frame-ancestors 'self';
base-uri 'self';
form-action 'self';
upgrade-insecure-requests;
report-to csp-endpoint

Generate a fresh nonce per request, attach it to every inline