Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 29 lessons
  1. Foundations: what RAG is and a working pipeline

    1. 1. Getting started: what RAG is and when to use it
    2. 2. Types of RAG: a map from naive to agentic
    3. 3. Lab: build the smallest RAG pipeline end to end
    4. 4. How embeddings and vector search find meaning
    5. 5. Document types: what each one breaks, and how to prepare it
  2. Better retrieval: chunking and search

    1. 6. Chunking documents so answers survive
    2. 7. Chunking strategies compared: which to use when
    3. 8. Keyword search vs vector search, and fusing both
    4. 9. Retrieval algorithms compared: which to use when
    5. 10. Reranking the top results before the model reads them
    6. 11. Rewriting the question before you search
  3. Answers you can trust, measure and debug

    1. 12. Lab: grounded answers with citations and refusals
    2. 13. Measuring a RAG system with an evaluation set
    3. 14. Lab: evaluation methods and frameworks: Ragas and DeepEval
    4. 15. Troubleshooting a RAG system, symptom by symptom
  4. Production in a company

    1. 16. Access control: only retrieve what the user may see
    2. 17. Defending against prompt injection in retrieved text
    3. 18. Running RAG in production: freshness, cost and monitoring
    4. 19. Real-world RAG: what worked, what failed, and why
  5. Agentic RAG

    1. 20. Lab: from a fixed pipeline to an agent with a search tool
    2. 21. Self-correcting retrieval: grade, retry, fall back
    3. 22. GraphRAG and routing across many sources
    4. 23. Lab: numbers, tables and financial documents
  6. Frameworks: LangChain, LangGraph and LangSmith

    1. 24. The RAG toolbox: what each library does and why you import it
    2. 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
    3. 26. Lab: Ask Brightline in LangChain
    4. 27. Lab: agentic RAG as a LangGraph graph
    5. 28. Lab: tracing, evaluating and monitoring with LangSmith
    6. 29. Wrap-up: your production agentic RAG checklist
See the leaderboard

20 min

Lab: from a fixed pipeline to an agent with a search tool

Turn retrieval into a tool the model can call, run a capped agent loop, and see when that beats a pipeline.

By the end of this lesson you can

  • Tell a workflow from an agent, and pick the simpler one when it's enough
  • Expose retrieval as a tool with a clear description and input schema
  • Run an agent loop with a step cap and a stop rule

From earlier lessons · 1 of 2

A retired policy still answers questions. What's the likely cause?

Pick one answer.

From earlier lessons · 2 of 2

Which cost usually dominates a RAG answer?

Pick one answer.

An employee asks two things at once. Can they expense a Boston hotel for a conference their director approved? And do they still need meal receipts? The answer spans three snippets. Our pipeline runs one search, keeps three chunks, and misses the meal-receipt rule.

Lab: give the model the search tool

So far our code decided every step. Anthropic's guide to building agents draws the line clearly. A workflow runs the model and tools through predefined code paths. An agent lets the model direct its own steps and tool use.

The pipeline from lessons 3-12 is a workflow. In this lab the model decides what to search for, and how many times, until it can answer. The pattern of interleaving reasoning with actions comes from the ReAct paper (Yao et al., 2022).

[9][10]

python
from rag import _client, MODEL # the client and model name from rag.py
AGENT_SYSTEM = ("You answer Brightline employees' policy questions. Search before answering, "
"once per distinct topic. Cite passage ids like (travel-hotel). "
"If two searches find nothing relevant, say you don't know.")
Setup. Each loop step is one model call, so outputs vary and the agent may search in a different order.

Step 1: describe the tool

Tool use means the model can ask your code to run a function. The model only sees the name, description and input schema, so write the description like documentation for a new colleague.

python
TOOLS = [{
"name": "search_policies",
"description": ("Search Brightline's HR, travel, IT and security policies. "
"Returns up to 3 passages with ids. Use a standalone query for one topic."),
"input_schema": {"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"]},
}]
def search_policies(query):
hits = rerank(query, retrieve(query, k=10), keep=3) # lessons 3 and 10, unchanged
return "\n".join(f"({d['id']}) {d['text']}" for d, _ in hits)
print(search_policies("conference hotel exemption")[:70])
Expected output: (travel-conf) Conference hotels booked through the event organiser are...

Step 2: run the tools the model asks for

python
def run_tools(reply):
results = []
for block in reply.content:
if block.type == "tool_use":
print("search:", block.input["query"])
results.append({"type": "tool_result", "tool_use_id": block.id,
"content": search_policies(block.input["query"])})
return results

Step 3: the agent loop

The agent loop repeats: call the model, run any tools it asked for, send the results back. It stops when the model answers without asking for a tool.

python
def agent(question, max_steps=5):
messages = [{"role": "user", "content": question}]
for _ in range(max_steps):
reply = _client.messages.create(model=MODEL, max_tokens=1000, system=AGENT_SYSTEM,
tools=TOOLS, messages=messages)
messages.append({"role": "assistant", "content": reply.content})
if reply.stop_reason != "tool_use":
return "".join(b.text for b in reply.content if b.type == "text")
messages.append({"role": "user", "content": run_tools(reply)})
return "I couldn't finish this one. Please ask HR or IT directly."

Step 4: the question the pipeline missed

python
print(agent("Can I expense a Boston hotel for a conference my director approved, "
"and do I still need a receipt for meals?"))
text
search: Boston hotel limit approval
search: conference hotel exemption
search: meal receipts
Conference hotels booked through the organiser are exempt from the nightly limit
(travel-conf). Otherwise Boston is $250 a night, and above that you need your
director's written approval before booking (travel-hotel). Meals: keep receipts
and submit them within 30 days (travel-meals).
Illustrative output. In our real run the agent searched twice (Boston hotel and conference, then meal receipts) and answered correctly. Expect three or four model calls; your searches will differ.

The model split the question into three topics on its own and searched each one. In lesson 11 we wrote code to decompose questions. Here the model does it, and it can follow up when a result raises a new question.

Issue: the agent never stops

python
print(agent("What's the parking policy at the Denver office?"))
Expected output: two or three searches such as "Denver office parking" and "employee parking reimbursement", then "I couldn't find a parking policy...".

There's no parking policy, so every search comes back weak. Without a limit, some models keep rephrasing and searching. Each search is another model call, so cost and latency grow with every step. Two guards fix it. The step cap (max_steps=5) ends the loop in code, whatever the model wants. The stop rule in the system prompt tells the model to give up after two empty searches.

Microsoft's agentic RAG guidance recommends the same. Cap tool calls per request, typically at 5 to 10. Track tokens per request, and tell the agent in its instructions when to stop.

[16]

Agents cost more. This lab's conference question took four model calls where the pipeline took one. Anthropic's guide advises finding the simplest solution that works and adding complexity only when it clearly improves results. Use the agent for multi-part or exploratory questions, and the fixed pipeline for the simple ones. Lesson 22 routes between them.

[10]

Put in order

Put one agent turn in order

Drag, or focus an item and press Space, move with the arrow keys, then Space to drop.

  1. The model asks for search_policies with a query
  2. Send the question and tool list to the model
  3. The model reads them and decides: search again or answer
  4. Your code runs the search and returns passages
  5. The loop ends on an answer or the step cap

Worth keeping

  1. A workflow follows your code paths; an agent chooses its own steps.
  2. The tool description is the model's only documentation, so write it carefully.
  3. Cap the steps in code and give the model a stop rule.
  4. Agents cost several calls per question, so use them where one search isn't enough.

The agent trusts every search result it gets. When a search returns three chunks that are all slightly off-topic, it writes a confident answer from them. Next lesson teaches it to grade what it found and try again.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (3)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [9]Yao et al. (2022), ReAct: Synergizing Reasoning and Acting
  2. [10]Anthropic (2024), Building effective agents
  3. [16]Microsoft Learn: Develop an agentic RAG solution