Vector Embeddings for AI Search: How Content Becomes Retrievable

TL;DR

AI engines turn your prose into vectors and your reader's question into a vector, then retrieve the chunks whose vectors are closest. Win retrieval by writing in self-contained passages, repeating canonical entities near the top of each section, keeping headings stable, and giving the chunker clean break points. Authority and freshness still matter, but they cannot rescue content that does not survive chunking.

What is a vector embedding?

A vector embedding is a fixed-length list of floating-point numbers (commonly 256, 768, 1024, or 3,072 dimensions) produced by an embedding model. The model is trained so that semantically similar text yields similar vectors when compared with cosine similarity or dot product.

When a user asks a generative engine a question, the engine:

1. Embeds the question.
2. Searches a pre-built index of content embeddings for the closest matches (typically via Approximate Nearest Neighbor algorithms).
3. Hands the top matches to the LLM as context.
4. Generates an answer that may cite the matched sources.

If your content's vectors are not close to the query vector, the LLM never sees you, and you cannot be cited — no matter how well-written the page is.

Why this matters for publishers

Traditional SEO targeted keyword overlap with the query. AI search targets semantic overlap, which is shaped by:

• The embedding model, which the engine controls.
• The chunking strategy, which the engine controls.
• The structure and content of your page, which you control.

That last bucket is the entire publisher game. Pages that survive chunking with intact, canonical, entity-dense passages dominate retrieval; pages that don't, lose — even at equal authority. See LLM citation benchmarks for how to measure outcomes.

How AI engines actually use embeddings

1. Crawl and ingest

A retrieval pipeline fetches your URL (subject to robots.txt and AI bot policies), strips boilerplate, and converts HTML to plain text. See AI Crawl Budget for the upstream economics.

2. Chunk

The pipeline splits the text into chunks (often 200-1,000 tokens). Boundaries usually respect heading structure, paragraph breaks, list items, and code blocks. Aggressive engines chunk by token windows and rely on overlap to prevent context loss.

3. Embed

Each chunk is passed through an embedding model. Modern models (text-embedding-3, Cohere Embed v3, Voyage, Gemini) embed sub-document units — sentences, sections, passages — not whole pages. A 4,000-word article might generate 8-12 vectors, each indexed separately.

4. Index

Vectors are stored in an Approximate Nearest Neighbor index (HNSW, IVF-PQ, ScaNN). The index is rebuilt periodically and consulted at query time.

5. Retrieve

At query time, the engine embeds the user's prompt, retrieves the top k nearest chunks (often k = 5-20), and may rerank them with a heavier cross-encoder. The retrieved chunks — not the whole page — are what the LLM "reads".

6. Generate and cite

The LLM produces an answer grounded in those chunks. Engines that cite (Perplexity, ChatGPT search, Gemini grounding) attribute spans of the answer back to the chunks they came from, and surface the source URL. If your chunk did not make it into retrieval, you cannot be cited.

The publisher's control surface

You cannot pick the model. You cannot pick the chunker. You can pick everything that shapes what gets chunked.

Chunking-friendly structure

• Use real headings. H2/H3 give the chunker clean boundaries. Long unbroken prose forces window-based chunking that splits answers mid-thought.
• Cap section length. Aim for 250-600 words per H2 section so each section becomes a self-contained chunk.
• One concept per section. Mixing concepts in a section produces a vector that resembles neither cleanly.
• Avoid stitched anchors. A heading like "Setup, configuration, and FAQs" merges three queries into one chunk; split it.

Passage isolation

A chunk that gets retrieved must be able to answer the query on its own, because the LLM may only see that chunk. To make passages self-sufficient:

• Open every section with a one-sentence answer.
• Restate the entity name (don't rely on a pronoun referring back to the H1).
• Resolve acronyms on first use within the section.
• Avoid "as discussed above" — "above" doesn't survive chunking.

Entity density

Embeddings cluster around entities. Pages that mention the canonical entity (and its key co-occurring entities) within each section produce vectors that cluster near queries about that entity. See entity salience for AI citations for the broader pattern.

Stable anchors

When retrieval pipelines store URL fragments alongside chunks (Perplexity, AI Overviews), stable anchors mean repeated retrievals reinforce a single canonical chunk. Drifting anchors fragment the signal.

Answer-first formatting

Answer-first paragraphs are optimal for both retrieval and generation. They put the highest-information sentence at a position the embedding model heavily weights, and they give the LLM something extractable when it stitches the answer.

FAQ-shaped passages

FAQ blocks (### Q: ...) are particularly retrieval-friendly because they encode the query and the answer in one compact chunk. Use them deliberately for high-volume informational queries.

Similarity math (just enough)

Most engines compare embeddings with cosine similarity:

cos(θ) = (A · B) / (|A| * |B|)

A score near 1.0 means "semantically very similar"; near 0 means unrelated. Some pipelines use raw dot product (faster, equivalent when vectors are normalized).

Practical implications:

• Vectors live on a unit sphere; "close" is measured by angle, not Euclidean distance.
• Two passages can have the same words and different vectors if the surrounding context is different. Context bleed is real.
• Tiny grammatical changes (e.g., "cannot" vs "can not") barely affect the vector; structural changes (a new opening sentence) can move it noticeably.

Reranking

After the initial nearest-neighbor pass, many pipelines run a reranker — a heavier cross-encoder that scores each candidate chunk against the query directly. Rerankers reward:

• Direct lexical overlap with the query (welcome news for old-school keyword discipline).
• Coherent answer-shaped passages.
• Authoritative source signals (citations, freshness, schema).

A passage that survives nearest-neighbor and reranking is the one that gets passed to the LLM and likely cited.

How to apply this

1. Audit chunkability. For each Tier 1 page, paste it into a chunker (LangChain, LlamaIndex, or a quick token-window script) and verify the boundaries land at headings, not mid-paragraph.
2. Tighten section openers. Rewrite the first sentence of each section to answer the section's implicit question.
3. Re-anchor entities. Ensure the canonical entity appears within the first 50 words of every section that should rank for it.
4. Add FAQ blocks for the top 3-5 queries each page should win.
5. Stabilize anchors and ship redirects when headings change.
6. Track the outcome with LLM citation benchmarks.

Common mistakes

• Massive intros. A 600-word lead-in pushes the answer below the chunk window.
• Pronoun overuse. "It supports…" without naming the entity yields a vector that doesn't cluster around the entity.
• Word-soup sections. Mixing five sub-topics in one H2 produces a vector that retrieves for none of them.
• Plain-text imitations of headings. Bolded paragraph leads do not break the chunker the way real H2/H3 elements do.
• No FAQ. You leave easy retrieval gold on the table.
• Static lastmod. Pipelines bias toward fresher embeddings.

FAQ

Q: Do I need to host my own vector database to win AI citations?

No. Major engines maintain their own indexes. Your job is to make your published content embed well in their index. A self-hosted vector database is only relevant if you are building an internal RAG product.

Q: Does keyword density still matter when retrieval uses embeddings?

Less than it used to, more than zero. Rerankers reward lexical overlap with the query, and embeddings cluster around entities you mention by name. Strict density tuning is dead; deliberate canonical naming is alive.

Q: How long should each chunk be?

Most retrieval pipelines target 200-1,000 tokens per chunk with some overlap. You don't pick the chunk size, but you can shape it indirectly with section length — a 250-600-word H2 section maps cleanly to a single chunk in most pipelines.

Q: Why do my pages rank in Google but not get cited by ChatGPT?

Usually because the page chunks poorly: long unbroken prose, buried answers, missing entity restatements per section, or no FAQ. Google rewards page-level signals; AI search rewards chunk-level retrievability.

Q: Will the embedding model change matter for my SEO?

Mostly no. Newer models (text-embedding-3, Embed v3) are more semantically robust, which generally favors well-structured publishers. The few times it hurts are when an old page leaned on quirky token-level matches; those edge cases get washed out by a stronger model.