Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 29 lessons
  1. Foundations: what RAG is and a working pipeline

    1. 1. Getting started: what RAG is and when to use it
    2. 2. Types of RAG: a map from naive to agentic
    3. 3. Lab: build the smallest RAG pipeline end to end
    4. 4. How embeddings and vector search find meaning
    5. 5. Document types: what each one breaks, and how to prepare it
  2. Better retrieval: chunking and search

    1. 6. Chunking documents so answers survive
    2. 7. Chunking strategies compared: which to use when
    3. 8. Keyword search vs vector search, and fusing both
    4. 9. Retrieval algorithms compared: which to use when
    5. 10. Reranking the top results before the model reads them
    6. 11. Rewriting the question before you search
  3. Answers you can trust, measure and debug

    1. 12. Lab: grounded answers with citations and refusals
    2. 13. Measuring a RAG system with an evaluation set
    3. 14. Lab: evaluation methods and frameworks: Ragas and DeepEval
    4. 15. Troubleshooting a RAG system, symptom by symptom
  4. Production in a company

    1. 16. Access control: only retrieve what the user may see
    2. 17. Defending against prompt injection in retrieved text
    3. 18. Running RAG in production: freshness, cost and monitoring
    4. 19. Real-world RAG: what worked, what failed, and why
  5. Agentic RAG

    1. 20. Lab: from a fixed pipeline to an agent with a search tool
    2. 21. Self-correcting retrieval: grade, retry, fall back
    3. 22. GraphRAG and routing across many sources
    4. 23. Lab: numbers, tables and financial documents
  6. Frameworks: LangChain, LangGraph and LangSmith

    1. 24. The RAG toolbox: what each library does and why you import it
    2. 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
    3. 26. Lab: Ask Brightline in LangChain
    4. 27. Lab: agentic RAG as a LangGraph graph
    5. 28. Lab: tracing, evaluating and monitoring with LangSmith
    6. 29. Wrap-up: your production agentic RAG checklist
See the leaderboard

15 min

Reranking the top results before the model reads them

Add a cross-encoder reranker that rescores the top candidates, and order context so the model uses it.

By the end of this lesson you can

  • Explain why a cross-encoder ranks better than a bi-encoder but can't search a whole corpus
  • Add a retrieve-wide, rerank, keep-narrow stage to the pipeline
  • Order passages in the prompt so the model uses the best one

From earlier lessons · 1 of 2

Why does reciprocal rank fusion use ranks instead of raw scores?

Pick one answer.

From earlier lessons · 2 of 2

Which search finds error code PRN-0x81 most reliably?

Pick one answer.

Ask Brightline's index now has 14 snippets, including a few near-misses about hotels. Someone asks who has to approve a pricier hotel. Vector search ranks the loyalty-points rule first. The approval rule comes second, and with k=1 it never reaches the model.

Where we are, shown failing

Real indexes are full of near-misses: chunks that share a topic but not the answer. We add six to lab 1's list, then rebuild the vectors so retrieve() from lab 1 sees them.

python
DOCS += [
{"id": "travel-loyalty", "text": "Hotel loyalty points earned on business stays belong to the employee. Do not choose a hotel for its points."},
{"id": "travel-booking", "text": "Book hotels and flights through the travel portal at least 7 days before the trip."},
{"id": "travel-conf", "text": "Conference hotels booked through the event organiser are exempt from nightly limits."},
{"id": "fin-approval", "text": "Expense reports above $1,000 need a director's approval in the finance system before payment."},
{"id": "office-denver", "text": "The Denver office is open 7am to 7pm. Visitors sign in at reception on the ground floor."},
{"id": "office-boston", "text": "The Boston office has 40 hot desks. Book a desk in the Brightline app before you arrive."},
]
vectors = embed([d["text"] for d in DOCS]) # lab 1's retrieve() reads this
print([(d["id"], round(s, 2)) for d, s in retrieve("Who has to approve a pricier hotel?", k=3)])
Expected output: [('travel-loyalty', 0.49), ('travel-hotel', 0.46), ('fin-approval', 0.4)]. DOCS now has the 8 snippets from lesson 8 plus these 6.

The right chunk is there, a close second. The first stage did its job, which is recall: getting the answer into the shortlist. It's weak at precision: putting the best item at the very top. A reranker fixes precision.

Two kinds of model

The embedding model you've used so far is a bi-encoder. It encodes the question and each chunk separately, so chunk vectors can be computed once and searched in milliseconds. But the question and the chunk never meet inside the model.

A cross-encoder reads the question and one chunk together, as a pair, and outputs a relevance score. It can notice that "approve" in the question matches "written approval" for hotels, not loyalty points. The cost is one model pass per pair, with nothing precomputed. Scoring a million chunks per question is out of reach, but scoring 50 is cheap.

The Sentence Transformers documentation describes exactly this split. Cross-encoders are usually more accurate and slower, so they're used to rerank a bi-encoder's top results.

[17]

Rendering diagram…
Retrieve wide for recall, rerank for precision, keep narrow for the prompt.

Add the reranking layer

python
from sentence_transformers import CrossEncoder
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2") # ~90 MB, runs on CPU
def rerank(question, hits, keep=3):
ranked = reranker.rank(question, [doc["text"] for doc, _ in hits])
return [hits[r["corpus_id"]] for r in ranked[:keep]]
q = "Who has to approve a pricier hotel?"
print([doc["id"] for doc, _ in rerank(q, retrieve(q, k=10))])
Expected output: ['travel-hotel', 'fin-approval', 'travel-loyalty']. The approval rule moves from second to first.

Our run gave the hotel rule a score of −7.03, against −9.87 and −10.24. These models output raw scores, not 0-to-1 probabilities, so only the order matters. Scoring 10 pairs took about 130 ms on a laptop CPU, including a first-call warm-up. Hosted rerankers and GPUs are faster; measure your own.

Re-run a harder case

python
q = "Can I go over the hotel limit if someone senior says yes?"
print([doc["id"] for doc, _ in rerank(q, retrieve(q, k=10))])
Expected output: ['travel-conf', 'travel-hotel', 'travel-loyalty'].

The reranker isn't perfect. It ranks the conference exemption first, which is a fair reading of "go over the limit". But the approval rule is still second. That's why you keep 3-5 chunks rather than 1. The reranker improves the order, and the model reads the rest.

Put the best passage where the model looks

Order inside the prompt matters too. In the Lost in the Middle study (Liu et al., 2023), models used information best at the start or end of a long context. Accuracy dropped when the key passage sat in the middle, even for long-context models.

So put the top-ranked chunk first. Keep the context short enough that nothing important sits deep in the middle. Reranking gives you that order for free.

[3]

How much does all this buy? In Anthropic's 2024 tests, contextual embeddings plus contextual BM25 cut failed retrievals by 49%. Adding a reranker raised that to 67%, measured as the share of answers missing from the top 20 chunks. They reranked the top 150 candidates down to 20. Treat these as one vendor's results on its datasets, and measure on yours.

[2]

Predict, then reveal

Your hybrid search returns 100 candidates and the reranker keeps 5. Answers are often incomplete, but the right chunk is usually ranked 6th to 8th. What do you change first?

Worth keeping

  1. First-stage search is for recall; the reranker is for precision.
  2. Cross-encoders read question and chunk together, so they rank better but can't scan a corpus.
  3. Retrieve 50-150, rerank, keep 5-20; tune the counts with your evaluation set.
  4. Put the best chunk first; models use the middle of long context least.

The reranker can only reorder what the first search found. When an employee types "and in Denver?", the first search has nothing to work with. Next lesson fixes the question itself.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (3)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [2]Anthropic (2024), Introducing Contextual Retrieval
  2. [3]Liu et al. (2023), Lost in the Middle
  3. [17]Sentence Transformers docs: Cross-Encoder usage