All 29 lessons
Foundations: what RAG is and a working pipeline
Better retrieval: chunking and search
Answers you can trust, measure and debug
Production in a company
Agentic RAG
Frameworks: LangChain, LangGraph and LangSmith
- 24. The RAG toolbox: what each library does and why you import it
- 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
- 26. Lab: Ask Brightline in LangChain
- 27. Lab: agentic RAG as a LangGraph graph
- 28. Lab: tracing, evaluating and monitoring with LangSmith
- 29. Wrap-up: your production agentic RAG checklist
18 min
Chunking strategies compared: which to use when
Compare seven chunking strategies on the same documents and pick the right one for each document type.
By the end of this lesson you can
- Name seven chunking strategies and what each one optimises
- Pick a strategy for a document type, including tables and code
- Explain when parent-child, semantic or late chunking is worth the cost
From earlier lessons · 1 of 2
Why did the heading-based chunker beat fixed-size chunks on the travel policy?
Pick one answer.
From earlier lessons · 2 of 2
What's a good starting point for fixed-size chunks?
Pick one answer.
Brightline indexes three kinds of content: 40-page policy documents with headings, IT chat transcripts with no structure at all, and an expense-limits table. Lesson 6's heading chunker works beautifully on the first. On the transcripts it produces one 9,000-word chunk. On the table it splits rows away from their column names.
One chunker doesn't fit every document
Lesson 6 built one chunker for one document. A real index holds many document types, and each breaks a chunker differently. There are seven strategies worth knowing. Each makes a different trade between three things: keeping ideas whole, keeping chunks small enough to match one question, and the cost to build.
Microsoft's RAG guide puts it bluntly: chunking is a semi-permanent choice, so compare approaches on your own content before you commit.
The seven strategies
- Fixed-size with overlap. Cut every N tokens and repeat a few at each boundary. It's cheap and works on anything, but cuts ideas in half. Use it as the fallback.
- Sentence-based. Group whole sentences up to a size limit. No sentence is ever cut, but one idea can still span two chunks. It suits prose with little structure, such as emails and reviews.
- Recursive splitting. Try the biggest separator first (blank lines between paragraphs), and fall back to line breaks, then sentences, then words, only where a piece is still too big. It's the common default in frameworks, and lesson 26 uses one.
- Structure-aware. Split on headings, sections or HTML tags, as lesson 6 did. It's the best choice when documents have real structure, such as policies, manuals and docs sites.
- Semantic chunking. Embed each sentence and cut where the meaning shifts between neighbours. It's the only option that finds topic boundaries without headings, but it costs an embedding per sentence and needs tuning.
- Parent-child (small-to-big). Index small chunks for precise matching, but hand the model the larger parent section they came from. It gives both precise matching and full context.
- Contextual and late chunking. Keep each chunk aware of its surroundings. Lesson 6 added a context line per chunk. Late chunking (Günther et al., 2024) embeds the whole document in one pass with a long-context model, then pools the vectors per chunk.
Worked example: semantic chunking
Six sentences, two topics: hotels, then a lost laptop. We embed each sentence and measure the similarity between neighbours. A low value suggests a topic change.
import numpy as npfrom rag import embedsentences = ["Book hotels through the travel portal.","The nightly limit is $250 in Tier 1 cities and $175 elsewhere.","Stays above the limit need a director's written approval.","Report a lost or stolen laptop within 1 hour.","Do not try to recover data from it yourself.","The security desk will lock the device remotely.",]v = embed(sentences)sims = [float(v[i] @ v[i + 1]) for i in range(len(v) - 1)]cut = np.percentile(sims, 25)print([round(s, 2) for s in sims], [i + 1 for i, s in enumerate(sims) if s <= cut])
The lowest similarity, 0.04, sits exactly where hotels end and laptops begin. That's a correct break. But the 25th-percentile rule also cut inside the laptop topic, at 0.14. Short sentences with different words look unrelated to a small model.
That's semantic chunking's real cost: an embedding per sentence, plus a threshold to tune, and it still makes mistakes. Use it for long documents without headings where topics really do shift. Add a minimum chunk size so it can't produce one-sentence chunks.
Worked example: parent-child chunks
import rePARENTS = {"travel-hotels": "Book hotels through the travel portal. The limit is $250 per night in Tier 1 cities and $175 elsewhere. Stays above the limit need a director's written approval before booking.","sec-devices": "Report a lost or stolen laptop within 1 hour. Do not try to recover data yourself. The security desk will lock the device remotely.",}CHILDREN = [(pid, s) for pid, text in PARENTS.items() for s in re.split(r"(?<=\.)\s+", text)]child_vecs = embed([s for _, s in CHILDREN])def small_to_big(question):pid, child = CHILDREN[int(np.argmax(child_vecs @ embed([question])[0]))]return child, PARENTS[pid] # match on the sentence, answer from the sectionprint(small_to_big("Who approves a hotel above the limit?")[0])
The small chunk matched the question precisely. The model still reads the booking rule and both limits, because it receives the parent. Use parent-child when questions are specific but answers need surrounding context, as with policies and manuals.
Tables and code need their own rules
Tables break every text chunker, because a row means nothing without its column names. Keep small tables whole. For large ones, write one chunk per row with the headers repeated: "Expense type: Meals. Daily limit: $75. Receipt needed: yes."
Code should be split by function or class, never mid-function. Keep the file path and the enclosing class name as the chunk's context line.
Slides and scans need layout-aware parsing before any chunking, which is outside this course.
Match
Match each Brightline source to its first strategy
Pick an item on the left, then its match on the right.
Worth keeping
- Pick the chunker by document structure, not one for the whole index.
- Recursive splitting is the safe default; structure-aware wins when headings exist.
- Parent-child gives precise matching and full context at once.
- Semantic chunking finds topic shifts without headings, but costs more and needs tuning.
Error code VPN-0x80 now sits in a clean chunk of its own. Vector search still scores the Wi-Fi code nearly as high. Next lesson adds the old-fashioned search that never confuses two codes.
Leaderboard
Loading…
50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.
Sources for this lesson (2)
Each fact in the lesson is marked with the number of the page it comes from.