125 articles · Page 6 of 11
Brotli vs Gzip Compression for AI Crawlers
Brotli vs Gzip for AI crawler responses: ratio, CPU cost, Accept-Encoding negotiation, AI bot support, and CDN configuration patterns.
Cache-Control Headers Reference for AI Crawlers
Reference for Cache-Control directives (max-age, s-maxage, immutable, stale-while-revalidate) that influence AI crawler refresh frequency and citation freshness.
Canonicalization for AI Answers: Avoiding Duplicate and Conflicting Sources
Canonicalize duplicate and conflicting sources so AI answers cite the right URL. Practical playbook with rel=canonical, redirects, sitemap, and update policies.
schema.org Claim Type for AI Fact-Checking
schema.org Claim type spec for AI fact-checking and grounding—appearance, firstAppearance properties and ClaimReview pairing patterns.
Conditional GET and ETag Handling for AI Crawlers
Conditional GET and ETag handling for AI crawlers: ETag generation, If-None-Match, If-Modified-Since, 304 Not Modified, and bandwidth-saving patterns.
Content Fingerprinting for AI Citations: Detection, Attribution, and Anti-Plagiarism
Practical guide to content fingerprinting for AI citation detection — SimHash, MinHash, embedding hashes, C2PA, and DMCA workflows for publishers.
Core Web Vitals and AI Citation Correlation: Does Page Speed Affect Citations?
What independent studies say about Core Web Vitals (LCP, INP, CLS, FCP) and AI citation rates across ChatGPT, Perplexity, and Google AI Overviews.
CORS Headers for AI Search Snippets
Configure CORS headers (Access-Control-Allow-Origin and friends) so AI search engines and embedded snippet widgets can fetch your content cleanly.
CSP Policy AI Crawler Allowlist Specification
Specification for CSP directives that keep AI crawlers (GPTBot, ClaudeBot, PerplexityBot) able to render and cite content without weakening XSS defense.
Dataset Schema for AI Citations
Schema.org Dataset JSON-LD spec for AI citations: required name/description/license, distribution with DataDownload, variableMeasured, temporalCoverage, and FAIR alignment.
DNS Prefetch and Preconnect for AI Crawlers
Reference for using dns-prefetch and preconnect resource hints with AI crawlers and browser agents: semantics, ordering, and impact on render-stage crawls.
Edge caching rules for AI crawlers (spec)
Edge caching spec for AI crawlers: per-bot TTL matrix, vary-on-user-agent rules, surrogate-key purges, and Cloudflare/Fastly/Akamai code samples.