All 29 lessons
Foundations: what RAG is and a working pipeline
Better retrieval: chunking and search
Answers you can trust, measure and debug
Production in a company
Agentic RAG
Frameworks: LangChain, LangGraph and LangSmith
- 24. The RAG toolbox: what each library does and why you import it
- 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
- 26. Lab: Ask Brightline in LangChain
- 27. Lab: agentic RAG as a LangGraph graph
- 28. Lab: tracing, evaluating and monitoring with LangSmith
- 29. Wrap-up: your production agentic RAG checklist
15 min
Rewriting the question before you search
Improve retrieval for vague, multi-part and follow-up questions by rewriting, splitting and expanding them.
By the end of this lesson you can
- Rewrite a follow-up question into a standalone search query
- Split a multi-part question into sub-queries and merge the results
- Use HyDE or multi-query expansion for questions in the wrong vocabulary
From earlier lessons · 1 of 2
Why retrieve 50 or more candidates when the model reads only five?
Pick one answer.
From earlier lessons · 2 of 2
A reranker is a cross-encoder. What makes it slow?
Pick one answer.
An employee asks about the hotel limit in Chicago. Ask Brightline says $250 a night, with a citation. Then they type "and in Denver?" Retrieval returns the Denver office's opening hours.
Every fix so far improved how chunks are prepared and ranked. This lesson fixes the other input: the question. We'll run three rounds on Ask Brightline. Each round has the same shape: try, inspect, name the issue, change one thing, rerun.
The code uses llm() from rag.py and retrieve() from lab 1, now reading all 14 snippets from lessons 8 and 10.
Round 1: a follow-up question
print([doc["id"] for doc, _ in retrieve("and in Denver?")])
Issue: the follow-up has no subject
"And in Denver?" means nothing without the previous turn. The retriever only sees the words it's given. The fix is query rewriting: ask a model to turn the follow-up into a standalone search query, using the chat history.
def standalone(history, question):prompt = (f"Conversation so far:\n{history}\n\nFollow-up: {question}\n""Rewrite the follow-up as one standalone search query. ""Reply with the query only.")return llm(prompt).strip()q = standalone("User: How much can I spend on a hotel in Chicago?\nAssistant: Up to $250 a night [1].","and in Denver?")print(q, [doc["id"] for doc, _ in retrieve(q)])
The rewritten query names the topic, so travel-hotel ranks first on merit, at 0.64. A terse rewrite such as "Denver hotel nightly limit" let the conference-hotel chunk edge ahead, so ask for a full phrase. Search with the rewrite, but keep the employee's original words for the final answer. The rewrite is for the retriever, not the reader.
Round 2: a two-part question
Lab 1 asked about the Denver hotel limit and wine with dinner in one breath. One query embeds both topics into one vector. The two chunks scored 0.51 and 0.47, so with k=1 only one of them reached the prompt.
Issue: one query, two topics
Query decomposition splits the question into sub-queries, searches each, and merges the results. Microsoft's RAG guidance uses the same move. A question like "PTO for remote workers hired after 2023" becomes focused subqueries, because the documents say "time off" and "telecommute".
import jsondef sub_queries(question):prompt = ("Split this question into the separate search queries needed to answer it. "f"Reply with a JSON list of strings only.\n\nQuestion: {question}")return json.loads(llm(prompt))parts = sub_queries("What's the hotel limit in Denver, and can I expense wine with dinner?")hits = {doc["id"] for p in parts for doc, _ in retrieve(p, k=1)}print(parts, hits)
Round 3: the wrong vocabulary
An employee writes: "the dongle thing for remote access is dead". The policy says "VPN token" and "Reset token". Keyword search has nothing to match, and vector search ties it-vpn with the laptop security rule at 0.29.
Issue: the question doesn't sound like the answer
Questions and answers are written differently. HyDE (hypothetical document embeddings, Gao et al., 2022) bridges that gap. A model writes a short, possibly wrong, passage that would answer the question. You embed that passage and search with it.
The made-up details don't matter, because you only use it to find real chunks that sound like an answer. The paper's authors note the encoder's dense vector filters out much of the invented detail.
def hyde(question):return llm("Write two sentences from a company IT policy that answer: " + question)fake = hyde("the dongle thing for remote access is dead")print(fake[:80], [doc["id"] for doc, _ in retrieve(fake, k=1)])
A close cousin is multi-query expansion. Ask the model for three paraphrases of the question, search with each, and fuse the rankings with rrf() from lesson 8. It costs one model call and a few searches, and it helps most with short, vague questions.
The rule behind all three rounds
Every rewrite adds a model call before the search, often a second or more. So don't rewrite every question. Rewrite follow-ups when there's chat history. Decompose when the question asks two things. Expand when the first retrieval scores are weak. A small, fast model is usually enough for rewriting.
Each round followed the same process. Look at what was retrieved, name the issue, change one thing, rerun. That process matters more than any single technique.
Match
Match each failing question to the fix
Pick an item on the left, then its match on the right.
Worth keeping
- Search with a rewritten query, but answer the original question.
- Rewrite follow-ups, decompose multi-part questions, expand vague ones.
- HyDE searches with a fake answer, because answers resemble answers.
- Each rewrite costs a model call, so trigger it only when it's needed.
Retrieval is now in good shape. But give the model perfect chunks and it can still add a rule that isn't there, or cite passage 3 for a fact from passage 1. Next lab makes the answer prove itself.
Leaderboard
Loading…
50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.
Sources for this lesson (2)
Each fact in the lesson is marked with the number of the page it comes from.