Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 29 lessons
  1. Foundations: what RAG is and a working pipeline

    1. 1. Getting started: what RAG is and when to use it
    2. 2. Types of RAG: a map from naive to agentic
    3. 3. Lab: build the smallest RAG pipeline end to end
    4. 4. How embeddings and vector search find meaning
    5. 5. Document types: what each one breaks, and how to prepare it
  2. Better retrieval: chunking and search

    1. 6. Chunking documents so answers survive
    2. 7. Chunking strategies compared: which to use when
    3. 8. Keyword search vs vector search, and fusing both
    4. 9. Retrieval algorithms compared: which to use when
    5. 10. Reranking the top results before the model reads them
    6. 11. Rewriting the question before you search
  3. Answers you can trust, measure and debug

    1. 12. Lab: grounded answers with citations and refusals
    2. 13. Measuring a RAG system with an evaluation set
    3. 14. Lab: evaluation methods and frameworks: Ragas and DeepEval
    4. 15. Troubleshooting a RAG system, symptom by symptom
  4. Production in a company

    1. 16. Access control: only retrieve what the user may see
    2. 17. Defending against prompt injection in retrieved text
    3. 18. Running RAG in production: freshness, cost and monitoring
    4. 19. Real-world RAG: what worked, what failed, and why
  5. Agentic RAG

    1. 20. Lab: from a fixed pipeline to an agent with a search tool
    2. 21. Self-correcting retrieval: grade, retry, fall back
    3. 22. GraphRAG and routing across many sources
    4. 23. Lab: numbers, tables and financial documents
  6. Frameworks: LangChain, LangGraph and LangSmith

    1. 24. The RAG toolbox: what each library does and why you import it
    2. 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
    3. 26. Lab: Ask Brightline in LangChain
    4. 27. Lab: agentic RAG as a LangGraph graph
    5. 28. Lab: tracing, evaluating and monitoring with LangSmith
    6. 29. Wrap-up: your production agentic RAG checklist
See the leaderboard

15 min

Keyword search vs vector search, and fusing both

Compare BM25 and vector search on the same questions and combine them with reciprocal rank fusion.

By the end of this lesson you can

  • Explain how BM25 scores a chunk in plain words
  • Pick keyword, vector or hybrid search for a type of question
  • Fuse two rankings with reciprocal rank fusion

From earlier lessons · 1 of 2

Why embed each chunk with its title and section?

Pick one answer.

From earlier lessons · 2 of 2

What does overlap protect against?

Pick one answer.

Brightline's IT index has three error codes: VPN-0x80, NET-0x80 and PRN-0x81. Ask vector search about VPN-0x80 and the Wi-Fi chunk scores 0.38, not far below the right one at 0.49. Add a few more codes and the wrong one can win.

The decision you'll face

Every retrieval system picks how to match a question to chunks. Vector search matches meaning. Lexical search matches the actual words. Most production systems run both. To see why, we'll send the same questions through each.

We add two IT snippets to lab 1's six, so the index has the three error codes. DOCS is lab 1's list.

python
import re
import numpy as np
from rank_bm25 import BM25Okapi # pip install rank_bm25
from rag import embed
DOCS += [
{"id": "it-wifi", "text": "Wi-Fi error NET-0x80 means your network certificate expired. Reconnect to BL-Secure to renew it."},
{"id": "it-printer", "text": "Printer error PRN-0x81 means the toner is low. Replace the cartridge from the supply room."},
]
tokenize = lambda t: re.findall(r"[a-z0-9-]+", t.lower())
bm25 = BM25Okapi([tokenize(d["text"]) for d in DOCS])
vecs = embed([d["text"] for d in DOCS])
DOCS is lab 1's six-snippet list; now eight snippets.

BM25 in plain words

BM25 is the standard lexical ranking. It gives a chunk points for each question word it contains, with three adjustments:

  • Rare words count more. A match on "vpn-0x80" is worth far more than a match on "the".
  • Repeats have diminishing returns. Saying "hotel" ten times doesn't make a chunk ten times better.
  • Long chunks are discounted. Otherwise they'd win just by containing more words.

Same questions, both searches

python
def keyword_top(q, k=2):
s = bm25.get_scores(tokenize(q))
return [(DOCS[i]["id"], round(float(s[i]), 2)) for i in np.argsort(-s)[:k]]
def vector_top(q, k=2):
s = vecs @ embed([q])[0]
return [(DOCS[i]["id"], round(float(s[i]), 2)) for i in np.argsort(-s)[:k]]
for q in ["What does VPN-0x80 mean?", "Can I stay home on Fridays?"]:
print(q, "\n keyword:", keyword_top(q), "\n vector: ", vector_top(q))
text
What does VPN-0x80 mean?
keyword: [('it-vpn', 1.49), ('travel-hotel', 0.0)]
vector: [('it-vpn', 0.49), ('it-wifi', 0.38)]
Can I stay home on Fridays?
keyword: [('travel-meals', 1.65), ('travel-hotel', 0.0)]
vector: [('hr-remote', 0.24), ('travel-hotel', 0.21)]
Real output with all-MiniLM-L6-v2. Keyword scores are exact; vector scores can shift slightly between library versions.

Each search fails where the other succeeds. Keyword search found the one chunk containing "vpn-0x80" and gave everything else zero. For the home question, its only match was the word "on", so it returned the meals policy. Vector search found remote work from meaning alone, but only just: 0.24 against 0.21 for the hotel policy. For the VPN code it ranked the right chunk first, with the Wi-Fi code close behind. That margin shrinks as the index grows.

When to use each:

  1. Exact strings such as error codes, part numbers, names and dates. Keyword search wins.
  2. Paraphrase and synonyms, such as "stay home" for "work remotely". Vector search wins.
  3. Typos and abbreviations. Vector search usually copes; keyword search misses unless you add fuzzy matching.
  4. Mixed real traffic. Use both, which is called hybrid search.

Microsoft's search documentation makes the same split. Keyword search is better at product codes, jargon, dates and names, while vectors find conceptually similar text.

[12]

Take a guess

You fuse both rankings. Chunk A is 1st in keyword search and 5th in vector search. Chunk B is 2nd in both. Which comes first after fusion?

No pressure. Guess now, and the lesson will show you if you were right.

[12]

Reciprocal rank fusion

Why not just add the scores? Because they're on different scales. BM25 scores have no upper limit, while cosine similarity sits between about 0 and 1. A sum would let one search drown out the other.

Reciprocal rank fusion (RRF) ignores raw scores and uses only positions. Each chunk earns 1 ÷ (k + rank) from every list it appears in, and the totals are sorted. The constant k is commonly 60. It stops the top position in one list from dominating. Azure AI Search uses RRF automatically for hybrid queries.

[12]

python
def rrf(rankings, k=60):
scores = {}
for ranking in rankings:
for rank, doc_id in enumerate(ranking, 1):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)
print(rrf([["A", "B", "C", "D", "E"], ["X", "B", "Y", "Z", "A"]])[:3])
Expected output: ['B', 'A', 'X']. The bet's answer, computed. Pass keyword and vector id lists to fuse the real searches.

Hybrid search is measurably better in published tests. In Anthropic's 2024 experiments, contextual embeddings alone cut failed retrievals by 35%. Adding BM25 on contextualised chunks raised the cut to 49%.

[2]

Match

Match each question to the search that should carry it

Pick an item on the left, then its match on the right.

Worth keeping

  1. Keyword search wins on exact strings; vector search wins on meaning.
  2. Never add scores from different searches; fuse their ranks.
  3. RRF gives each chunk 1/(k + rank) per list, with k commonly 60.
  4. Default to hybrid search for company documents, then measure.

Hybrid search works on 14 snippets. Brightline's full index will hold 10 million chunks, and exact search over all of them takes 15 GB of memory. Next lesson maps the retrieval algorithms and indexes, and when to use each.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (2)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [2]Anthropic (2024), Introducing Contextual Retrieval
  2. [12]Microsoft Learn: Relevance scoring in hybrid search using RRF