Technical Implementation · Page 2
llms.txt, ai.txt, structured data, and other technical specs for AI search readiness.
148 articles · Page 2 of 13
AI Crawler IP Allowlist Reference
Reference list of official AI crawler IP range endpoints, user agents, and reverse-DNS verification methods for GPTBot, ClaudeBot, PerplexityBot, Googlebot, and more.
AI Crawler Prefetch Hints Specification
How to use Resource Hints, Link headers, and 103 Early Hints to accelerate AI crawler discovery while keeping origin load and crawl budget under control.
AI Crawler Rate Limiting Reference: Throttling GPTBot, ClaudeBot, and PerplexityBot Without Losing Citations
Reference table of safe rate limits for GPTBot, ClaudeBot, PerplexityBot, and other AI crawlers, with citation-impact tradeoffs and edge recipes.
AI Search Author Schema Specification
Author entity markup for AI citation: Person schema, sameAs identifiers, credentials, and Wikidata linkage that lift author authority signals.
AI Search Canonical URL Handling Specification
How AI search engines (ChatGPT, Perplexity, Gemini, Google AI Overviews) resolve rel=canonical, hreflang, and parameterized URLs when selecting and citing sources.
AI Search Content Freshness Signals
Reference of freshness signals AI crawlers track — lastmod, dateModified, version banners, changelogs, and substantive republishes — and how they influence citation recency.
AI Search Optimization for Glossary Pages: A Specification
AI search glossary page spec: term, definition, anchor link, and DefinedTerm schema patterns that maximize citations from ChatGPT and Perplexity.
AI Search Redirect Chain Behavior Reference
Reference for how 301, 302, 307, 308, meta refresh, and JavaScript redirects affect AI search citation persistence on ChatGPT, Perplexity, Gemini, and AI Overviews.
AI Search Table Data Optimization
Make HTML tables AI-citable: header semantics, captions, key-value pairs, and DataTable schema patterns for high-extraction tabular content.
ai.txt Starter Template: Copy-Ready AI Access Policy File
A copy-ready ai.txt starter template for declaring AI crawler access policies, attribution requirements, and content licensing terms.
ai.txt: AI Agent Access Policy Reference
ai.txt is an emerging root-level file that declares site-wide permissions and attribution rules for AI training, citation, and inference.
API Content Design for AI Consumption: Patterns for Reference Docs
Design patterns for API reference documentation that AI agents can parse, cite, and call: canonical examples, error tables, parameter schemas, and dual human/LLM delivery.