All 29 lessons
Foundations: what RAG is and a working pipeline
Better retrieval: chunking and search
Answers you can trust, measure and debug
Production in a company
Agentic RAG
Frameworks: LangChain, LangGraph and LangSmith
- 24. The RAG toolbox: what each library does and why you import it
- 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
- 26. Lab: Ask Brightline in LangChain
- 27. Lab: agentic RAG as a LangGraph graph
- 28. Lab: tracing, evaluating and monitoring with LangSmith
- 29. Wrap-up: your production agentic RAG checklist
18 min
Retrieval algorithms compared: which to use when
Map the retrieval algorithms, from exact search to ANN indexes, sparse, dense, multi-vector and MMR, and pick one per need.
By the end of this lesson you can
- Choose an index type (flat, HNSW, IVF, quantized) for a corpus size and memory budget
- Pick sparse, dense, hybrid or multi-vector retrieval for a type of question
- Use MMR and metadata filters, and know what each can break
From earlier lessons · 1 of 2
Why does reciprocal rank fusion combine ranks instead of scores?
Pick one answer.
From earlier lessons · 2 of 2
Which questions does keyword search handle better than vector search?
Pick one answer.
Ask Brightline goes company-wide: 10 million chunks across every department. The exact search from lab 1 now needs about 15 GB of memory, and a scan of all 4 billion multiplications per question. Something has to give, and you get to choose what.
Two separate choices
"Retrieval algorithm" covers two different decisions. Mixing them up is the most common source of confusion:
- How to represent text: sparse lexical (BM25), learned sparse, dense vectors, or many vectors per passage. This decides what counts as a match.
- How to search those representations fast: exact scan, graph index, clustered index or compressed vectors. This decides speed, memory and how often a true match is missed.
You pick one of each, then optionally combine methods (hybrid search) and post-process the results (rerank, diversify, filter).
Choice 1: how to represent text
- Sparse lexical (BM25). Exact words, with an inverted index. It's best for codes, names and jargon. Lesson 8 covered it.
- Learned sparse (SPLADE). A model learns weights for words, including related words the text doesn't contain. It's still served from an inverted index (Formal et al., 2021). Use it when you want keyword-style infrastructure with some sense of meaning.
- Dense bi-encoder. One vector per chunk, as in lesson 4. It's best for paraphrase and synonyms, and it's the default.
- Multi-vector late interaction (ColBERT). A vector for every token, matched token by token at query time. ColBERT's authors report it ran about two orders of magnitude faster than BERT cross-encoders while staying competitive in quality (Khattab and Zaharia, 2020). The cost is much more storage per passage. Use it when dense retrieval misses fine-grained matches and you can afford the index size.
Choice 2: how to search fast
- Flat (exact). Compare against every vector. FAISS's guide says it's the only index that guarantees exact results. It's fine up to tens or hundreds of thousands of vectors.
- HNSW (graph). Hop through a graph of neighbours, as in lesson 4. It has excellent speed and recall but needs the most memory, and FAISS's HNSW can't delete vectors. pgvector reports HNSW queries beat IVFFlat on the speed-recall trade-off, but it builds slower and uses more memory.
- IVF (clustered). Group vectors into clusters, then search only the closest few clusters, set by
nprobeorprobes. It builds faster and uses less memory than HNSW, but needs data before you build it. pgvector suggestsrows / 1000clusters up to 1 million rows, and starting withsqrt(lists)probes. - Quantized (PQ and similar). Compress each vector to a few bytes and search the compressed codes. It gives a huge memory saving for some recall, often with exact re-scoring of the top results.
Worked example: what 10 million chunks cost
d, n = 384, 10_000_000 # all-MiniLM-L6-v2 vectors, 10M chunksflat = d * 4 # 4 bytes per floathnsw = d * 4 + 32 * 2 * 4 # FAISS's formula with M=32 neighbourspq = 48 / 2 # PQ48x4fs: about M/2 bytes per vectorfor name, per_vec in [("flat", flat), ("hnsw", hnsw), ("pq", pq)]:print(f"{name:5} {per_vec:6.0f} B/vector {per_vec * n / 1e9:5.2f} GB")
HNSW is fastest but needs the most memory, slightly more than storing the raw vectors. Quantization fits the same corpus in about 240 MB, a 64-fold saving, at some cost to recall. A common production setup is IVF or HNSW over quantized vectors, then exact re-scoring of the top few hundred candidates.
After search: diversity and filters
Maximal marginal relevance (MMR) (Carbonell and Goldstein, 1998) picks results one at a time. Each pick balances relevance to the question against similarity to what's already picked. The weight λ sets the balance: 1 is pure relevance, 0 is pure diversity. Use it when your index has near-duplicates, such as overlapping chunks, FAQs that restate a policy, or several versions of one document.
def mmr(question, k=4, lam=0.7, fetch=10):q = embed([question])[0]rel = vectors @ qcandidates, chosen = list(np.argsort(-rel)[:fetch]), []while candidates and len(chosen) < k:def score(i):dup = max((float(vectors[i] @ vectors[j]) for j in chosen), default=0.0)return lam * rel[i] - (1 - lam) * dupbest = max(candidates, key=score)chosen.append(best); candidates.remove(best)return [DOCS[i]["id"] for i in chosen]
plain top 4: faq-hotel, travel-hotel, travel-booking, travel-loyaltymmr (0.7): faq-hotel, travel-booking, office-boston, travel-loyalty
MMR did its job: it dropped travel-hotel because it repeats the FAQ's limits, and added the Boston office page. But travel-hotel also held the director-approval rule, which the FAQ doesn't. Diversity removed a chunk with a unique detail. So use MMR with a high λ (0.7-0.9), and remove true duplicates when you index rather than relying on MMR at query time.
Metadata filters restrict search by fields such as department, date or allowed groups. Lesson 16 depends on them. Where the filter runs matters. pgvector's documentation shows the trap: with an approximate index, filtering happens after the index scan. If a condition matches 10% of rows, HNSW with the default ef_search of 40 returns only about 4 rows. Fixes include iterative index scans, pre-filtering engines, or exact search over a small filtered set.
A self-query retriever has a model turn "hotel rules updated this year" into a search plus a filter such as updated >= 2026-01-01.
Match
Match each situation to the retrieval choice
Pick an item on the left, then its match on the right.
Worth keeping
- Choose how text is represented and how it's searched as two separate decisions.
- Flat is exact and fine up to about 100,000 vectors; beyond that, HNSW or IVF.
- Quantization cuts memory by tens of times for some recall; re-score the top results exactly.
- Use MMR gently, and check that filters don't starve an approximate index.
Whatever index you pick, first-stage search puts the right chunk in the top 10 almost every time, but not always in the top 3. The model only reads three. Next lesson adds a second, slower judge that reorders the shortlist.
Leaderboard
Loading…
50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.
Sources for this lesson (5)
Each fact in the lesson is marked with the number of the page it comes from.