Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 29 lessons
  1. Foundations: what RAG is and a working pipeline

    1. 1. Getting started: what RAG is and when to use it
    2. 2. Types of RAG: a map from naive to agentic
    3. 3. Lab: build the smallest RAG pipeline end to end
    4. 4. How embeddings and vector search find meaning
    5. 5. Document types: what each one breaks, and how to prepare it
  2. Better retrieval: chunking and search

    1. 6. Chunking documents so answers survive
    2. 7. Chunking strategies compared: which to use when
    3. 8. Keyword search vs vector search, and fusing both
    4. 9. Retrieval algorithms compared: which to use when
    5. 10. Reranking the top results before the model reads them
    6. 11. Rewriting the question before you search
  3. Answers you can trust, measure and debug

    1. 12. Lab: grounded answers with citations and refusals
    2. 13. Measuring a RAG system with an evaluation set
    3. 14. Lab: evaluation methods and frameworks: Ragas and DeepEval
    4. 15. Troubleshooting a RAG system, symptom by symptom
  4. Production in a company

    1. 16. Access control: only retrieve what the user may see
    2. 17. Defending against prompt injection in retrieved text
    3. 18. Running RAG in production: freshness, cost and monitoring
    4. 19. Real-world RAG: what worked, what failed, and why
  5. Agentic RAG

    1. 20. Lab: from a fixed pipeline to an agent with a search tool
    2. 21. Self-correcting retrieval: grade, retry, fall back
    3. 22. GraphRAG and routing across many sources
    4. 23. Lab: numbers, tables and financial documents
  6. Frameworks: LangChain, LangGraph and LangSmith

    1. 24. The RAG toolbox: what each library does and why you import it
    2. 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
    3. 26. Lab: Ask Brightline in LangChain
    4. 27. Lab: agentic RAG as a LangGraph graph
    5. 28. Lab: tracing, evaluating and monitoring with LangSmith
    6. 29. Wrap-up: your production agentic RAG checklist
See the leaderboard

18 min

Chunking documents so answers survive

Split long policy documents into chunks that keep their meaning and context, and measure the difference.

By the end of this lesson you can

  • Spot a chunk boundary that separates a rule from its exception
  • Chunk a document by its headings, with fixed-size splits and overlap as a fallback
  • Add a context line to each chunk before embedding it

From earlier lessons · 1 of 2

Why must questions and documents use the same embedding model?

Pick one answer.

From earlier lessons · 2 of 2

Where is vector search weakest?

Pick one answer.

Ask Brightline now reads the full travel policy. An employee asks whether a $300 Boston hotel is allowed with a director's approval. It answers "No, the limit is $250". The approval rule is in the index. It just never reached the prompt.

Where we are and what breaks

Lab 1 used six one-paragraph snippets. Real policies are long documents with sections. Embedding a whole 40-page policy as one vector blurs every topic into one position, so nothing matches well. We have to cut documents into chunks: pieces small enough to match one question, but complete enough to answer it.

Here is a short version of the travel policy. We'll chunk it three ways and see which survives.

python
TRAVEL = """# Travel policy
## Hotels
Book through the travel portal. Limits per night: $250 in Tier 1 cities (New York,
San Francisco, Chicago, Boston) and $175 elsewhere. Exceptions: stays above the
limit need a director's written approval before booking.
## Meals
Up to $75 per day. Alcohol is not reimbursable.
## Flights
Economy for flights under 6 hours. Premium economy is allowed above 6 hours."""

Naive version: fixed-size chunks

The simplest chunker counts words and cuts. Production chunkers count tokens, but words are close enough to see the problem.

python
def chunk_fixed(text, size=20, overlap=0):
words = text.split()
step = size - overlap
return [" ".join(words[i:i + size]) for i in range(0, len(words), step)]
for c in chunk_fixed(TRAVEL):
print("-", c)
Expected output: four chunks. The first stops mid-list at "(New York,". The second starts "San Francisco, Chicago, Boston) and $175 elsewhere. Exceptions: stays above the limit..." and never says hotel.

The second chunk holds the approval rule, but it says "the limit" and never mentions hotels. Its vector lands nowhere near a question about Boston hotels. This is context loss: a chunk that only makes sense next to its neighbours.

Layer 1: overlap

Overlap repeats the last few words of each chunk at the start of the next, so a sentence cut at a boundary appears whole in at least one chunk.

python
for c in chunk_fixed(TRAVEL, size=20, overlap=6):
print("-", c)
Expected output: five chunks. Boundary words now appear twice, but the approval rule is still split across two chunks, and neither says hotel.

Overlap rescues sentences cut at a boundary, but it can't tie this rule to the word hotel. Microsoft's Azure AI Search guidance suggests starting at about 512 tokens (roughly 2,000 characters) with 25% overlap, then testing on your own documents. Our 20-word chunks are tiny only so the cuts are visible.

[13]

Layer 2: chunk by structure

Policies already tell you where ideas begin and end: their headings. Structure-aware chunking splits on headings, and only falls back to fixed-size splits when a section is too long.

python
def chunk_by_heading(text, doc_title):
chunks = []
for block in text.split("\n## ")[1:]: # skip the title line
section, _, body = block.partition("\n")
chunks.append({"doc": doc_title, "section": section, "text": body.strip()})
return chunks
CHUNKS = chunk_by_heading(TRAVEL, "Travel policy")
print([c["section"] for c in CHUNKS])
Expected output: ['Hotels', 'Meals', 'Flights']. The hotel limit and its exception now share one chunk.

Layer 3: add context before embedding

A chunk from page 30 may still say "these limits" with no subject. The fix is to embed each chunk with a short context line: the document title and section. Anthropic's contextual retrieval goes further and has a model write 50-100 tokens of context per chunk.

In their tests, that alone cut failed retrievals by 35%. With prompt caching, they put the one-time cost at about $1.02 per million document tokens. Those figures come from Anthropic's September 2024 post; measure on your own data.

[2][13]

python
import numpy as np
from rag import embed
def embed_text(c):
return f"{c['doc']} > {c['section']}: {c['text']}"
vecs = embed([embed_text(c) for c in CHUNKS])
def search(question, k=2):
scores = vecs @ embed([question])[0]
return [CHUNKS[i]["section"] for i in np.argsort(-scores)[:k]]
print(search("Can I book a $300 hotel in Boston if my director approves?"))
Expected output: ['Hotels', 'Meals']. The hotel chunk, with its exception, ranks first.

We embed the text with its context line, but we still show the model the plain chunk with its source. Keep the two apart: one is for matching, the other is for reading.

Re-run the earlier cases

python
for q in ["How much can I spend on a hotel in Chicago?",
"Can I expense wine with dinner?",
"Do I fly economy to London?"]:
print(q, "->", search(q, k=1))
Expected output: Hotels, Meals and Flights in turn. Nothing that worked before broke.
Rendering diagram…
The chunking pipeline: structure first, fixed-size splits only as a fallback, context on every chunk.

Store metadata with every chunk: source document, section, last-updated date and the groups allowed to read it. You'll filter on dates in lesson 18 and on groups in lesson 16.

Treat chunking as a semi-permanent decision. Changing it means re-chunking and re-embedding everything, and re-running your evaluation set.

Match

Match each document to a good first chunking strategy

Pick an item on the left, then its match on the right.

Worth keeping

  1. Split by the document's own structure first, and by size only as a fallback.
  2. Overlap protects sentences cut at a boundary; start near 512 tokens with 25% overlap.
  3. Embed each chunk with its title and section, and show the model the plain chunk.
  4. Changing your chunking means re-embedding everything, so test before you commit.

Heading chunks fixed the travel policy. But Brightline also indexes chat transcripts with no headings, and a table of expense limits. Next lesson compares seven chunking strategies and when each one wins.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (2)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [2]Anthropic (2024), Introducing Contextual Retrieval
  2. [13]Microsoft Learn: Chunk large documents for RAG and vector search