Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 29 lessons
  1. Foundations: what RAG is and a working pipeline

    1. 1. Getting started: what RAG is and when to use it
    2. 2. Types of RAG: a map from naive to agentic
    3. 3. Lab: build the smallest RAG pipeline end to end
    4. 4. How embeddings and vector search find meaning
    5. 5. Document types: what each one breaks, and how to prepare it
  2. Better retrieval: chunking and search

    1. 6. Chunking documents so answers survive
    2. 7. Chunking strategies compared: which to use when
    3. 8. Keyword search vs vector search, and fusing both
    4. 9. Retrieval algorithms compared: which to use when
    5. 10. Reranking the top results before the model reads them
    6. 11. Rewriting the question before you search
  3. Answers you can trust, measure and debug

    1. 12. Lab: grounded answers with citations and refusals
    2. 13. Measuring a RAG system with an evaluation set
    3. 14. Lab: evaluation methods and frameworks: Ragas and DeepEval
    4. 15. Troubleshooting a RAG system, symptom by symptom
  4. Production in a company

    1. 16. Access control: only retrieve what the user may see
    2. 17. Defending against prompt injection in retrieved text
    3. 18. Running RAG in production: freshness, cost and monitoring
    4. 19. Real-world RAG: what worked, what failed, and why
  5. Agentic RAG

    1. 20. Lab: from a fixed pipeline to an agent with a search tool
    2. 21. Self-correcting retrieval: grade, retry, fall back
    3. 22. GraphRAG and routing across many sources
    4. 23. Lab: numbers, tables and financial documents
  6. Frameworks: LangChain, LangGraph and LangSmith

    1. 24. The RAG toolbox: what each library does and why you import it
    2. 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
    3. 26. Lab: Ask Brightline in LangChain
    4. 27. Lab: agentic RAG as a LangGraph graph
    5. 28. Lab: tracing, evaluating and monitoring with LangSmith
    6. 29. Wrap-up: your production agentic RAG checklist
See the leaderboard

18 min

Retrieval algorithms compared: which to use when

Map the retrieval algorithms, from exact search to ANN indexes, sparse, dense, multi-vector and MMR, and pick one per need.

By the end of this lesson you can

  • Choose an index type (flat, HNSW, IVF, quantized) for a corpus size and memory budget
  • Pick sparse, dense, hybrid or multi-vector retrieval for a type of question
  • Use MMR and metadata filters, and know what each can break

From earlier lessons · 1 of 2

Why does reciprocal rank fusion combine ranks instead of scores?

Pick one answer.

From earlier lessons · 2 of 2

Which questions does keyword search handle better than vector search?

Pick one answer.

Ask Brightline goes company-wide: 10 million chunks across every department. The exact search from lab 1 now needs about 15 GB of memory, and a scan of all 4 billion multiplications per question. Something has to give, and you get to choose what.

Two separate choices

"Retrieval algorithm" covers two different decisions. Mixing them up is the most common source of confusion:

  1. How to represent text: sparse lexical (BM25), learned sparse, dense vectors, or many vectors per passage. This decides what counts as a match.
  2. How to search those representations fast: exact scan, graph index, clustered index or compressed vectors. This decides speed, memory and how often a true match is missed.

You pick one of each, then optionally combine methods (hybrid search) and post-process the results (rerank, diversify, filter).

Choice 1: how to represent text

  • Sparse lexical (BM25). Exact words, with an inverted index. It's best for codes, names and jargon. Lesson 8 covered it.
  • Learned sparse (SPLADE). A model learns weights for words, including related words the text doesn't contain. It's still served from an inverted index (Formal et al., 2021). Use it when you want keyword-style infrastructure with some sense of meaning.
  • Dense bi-encoder. One vector per chunk, as in lesson 4. It's best for paraphrase and synonyms, and it's the default.
  • Multi-vector late interaction (ColBERT). A vector for every token, matched token by token at query time. ColBERT's authors report it ran about two orders of magnitude faster than BERT cross-encoders while staying competitive in quality (Khattab and Zaharia, 2020). The cost is much more storage per passage. Use it when dense retrieval misses fine-grained matches and you can afford the index size.

[22][23]

Choice 2: how to search fast

  • Flat (exact). Compare against every vector. FAISS's guide says it's the only index that guarantees exact results. It's fine up to tens or hundreds of thousands of vectors.
  • HNSW (graph). Hop through a graph of neighbours, as in lesson 4. It has excellent speed and recall but needs the most memory, and FAISS's HNSW can't delete vectors. pgvector reports HNSW queries beat IVFFlat on the speed-recall trade-off, but it builds slower and uses more memory.
  • IVF (clustered). Group vectors into clusters, then search only the closest few clusters, set by nprobe or probes. It builds faster and uses less memory than HNSW, but needs data before you build it. pgvector suggests rows / 1000 clusters up to 1 million rows, and starting with sqrt(lists) probes.
  • Quantized (PQ and similar). Compress each vector to a few bytes and search the compressed codes. It gives a huge memory saving for some recall, often with exact re-scoring of the top results.

[20][21]

Worked example: what 10 million chunks cost

python
d, n = 384, 10_000_000 # all-MiniLM-L6-v2 vectors, 10M chunks
flat = d * 4 # 4 bytes per float
hnsw = d * 4 + 32 * 2 * 4 # FAISS's formula with M=32 neighbours
pq = 48 / 2 # PQ48x4fs: about M/2 bytes per vector
for name, per_vec in [("flat", flat), ("hnsw", hnsw), ("pq", pq)]:
print(f"{name:5} {per_vec:6.0f} B/vector {per_vec * n / 1e9:5.2f} GB")
Real output: flat 1536 B/vector 15.36 GB; hnsw 1792 B/vector 17.92 GB; pq 24 B/vector 0.24 GB. Formulas from the FAISS index guide.

HNSW is fastest but needs the most memory, slightly more than storing the raw vectors. Quantization fits the same corpus in about 240 MB, a 64-fold saving, at some cost to recall. A common production setup is IVF or HNSW over quantized vectors, then exact re-scoring of the top few hundred candidates.

[20]

After search: diversity and filters

Maximal marginal relevance (MMR) (Carbonell and Goldstein, 1998) picks results one at a time. Each pick balances relevance to the question against similarity to what's already picked. The weight λ sets the balance: 1 is pure relevance, 0 is pure diversity. Use it when your index has near-duplicates, such as overlapping chunks, FAQs that restate a policy, or several versions of one document.

[24]

python
def mmr(question, k=4, lam=0.7, fetch=10):
q = embed([question])[0]
rel = vectors @ q
candidates, chosen = list(np.argsort(-rel)[:fetch]), []
while candidates and len(chosen) < k:
def score(i):
dup = max((float(vectors[i] @ vectors[j]) for j in chosen), default=0.0)
return lam * rel[i] - (1 - lam) * dup
best = max(candidates, key=score)
chosen.append(best); candidates.remove(best)
return [DOCS[i]["id"] for i in chosen]
DOCS and vectors are lesson 10's 14 snippets plus two near-duplicates: an FAQ restating the hotel limit, and an overlapping chunk.
text
plain top 4: faq-hotel, travel-hotel, travel-booking, travel-loyalty
mmr (0.7): faq-hotel, travel-booking, office-boston, travel-loyalty
Real output for "What do I need to know about hotels for my Boston trip?"

MMR did its job: it dropped travel-hotel because it repeats the FAQ's limits, and added the Boston office page. But travel-hotel also held the director-approval rule, which the FAQ doesn't. Diversity removed a chunk with a unique detail. So use MMR with a high λ (0.7-0.9), and remove true duplicates when you index rather than relying on MMR at query time.

Metadata filters restrict search by fields such as department, date or allowed groups. Lesson 16 depends on them. Where the filter runs matters. pgvector's documentation shows the trap: with an approximate index, filtering happens after the index scan. If a condition matches 10% of rows, HNSW with the default ef_search of 40 returns only about 4 rows. Fixes include iterative index scans, pre-filtering engines, or exact search over a small filtered set.

A self-query retriever has a model turn "hotel rules updated this year" into a search plus a filter such as updated >= 2026-01-01.

[21]

Match

Match each situation to the retrieval choice

Pick an item on the left, then its match on the right.

Worth keeping

  1. Choose how text is represented and how it's searched as two separate decisions.
  2. Flat is exact and fine up to about 100,000 vectors; beyond that, HNSW or IVF.
  3. Quantization cuts memory by tens of times for some recall; re-score the top results exactly.
  4. Use MMR gently, and check that filters don't starve an approximate index.

Whatever index you pick, first-stage search puts the right chunk in the top 10 almost every time, but not always in the top 3. The model only reads three. Next lesson adds a second, slower judge that reorders the shortlist.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (5)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [20]FAISS wiki: Guidelines to choose an index
  2. [21]pgvector README: HNSW, IVFFlat and filtering
  3. [22]Khattab and Zaharia (2020), ColBERT
  4. [23]Formal et al. (2021), SPLADE
  5. [24]Carbonell and Goldstein (1998), MMR diversity-based reranking