All 29 lessons
Foundations: what RAG is and a working pipeline
Better retrieval: chunking and search
Answers you can trust, measure and debug
Production in a company
Agentic RAG
Frameworks: LangChain, LangGraph and LangSmith
- 24. The RAG toolbox: what each library does and why you import it
- 25. The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
- 26. Lab: Ask Brightline in LangChain
- 27. Lab: agentic RAG as a LangGraph graph
- 28. Lab: tracing, evaluating and monitoring with LangSmith
- 29. Wrap-up: your production agentic RAG checklist
20 min
Lab: from a fixed pipeline to an agent with a search tool
Turn retrieval into a tool the model can call, run a capped agent loop, and see when that beats a pipeline.
By the end of this lesson you can
- Tell a workflow from an agent, and pick the simpler one when it's enough
- Expose retrieval as a tool with a clear description and input schema
- Run an agent loop with a step cap and a stop rule
From earlier lessons · 1 of 2
A retired policy still answers questions. What's the likely cause?
Pick one answer.
From earlier lessons · 2 of 2
Which cost usually dominates a RAG answer?
Pick one answer.
An employee asks two things at once. Can they expense a Boston hotel for a conference their director approved? And do they still need meal receipts? The answer spans three snippets. Our pipeline runs one search, keeps three chunks, and misses the meal-receipt rule.
Lab: give the model the search tool
So far our code decided every step. Anthropic's guide to building agents draws the line clearly. A workflow runs the model and tools through predefined code paths. An agent lets the model direct its own steps and tool use.
The pipeline from lessons 3-12 is a workflow. In this lab the model decides what to search for, and how many times, until it can answer. The pattern of interleaving reasoning with actions comes from the ReAct paper (Yao et al., 2022).
from rag import _client, MODEL # the client and model name from rag.pyAGENT_SYSTEM = ("You answer Brightline employees' policy questions. Search before answering, ""once per distinct topic. Cite passage ids like (travel-hotel). ""If two searches find nothing relevant, say you don't know.")
Step 1: describe the tool
Tool use means the model can ask your code to run a function. The model only sees the name, description and input schema, so write the description like documentation for a new colleague.
TOOLS = [{"name": "search_policies","description": ("Search Brightline's HR, travel, IT and security policies. ""Returns up to 3 passages with ids. Use a standalone query for one topic."),"input_schema": {"type": "object","properties": {"query": {"type": "string"}},"required": ["query"]},}]def search_policies(query):hits = rerank(query, retrieve(query, k=10), keep=3) # lessons 3 and 10, unchangedreturn "\n".join(f"({d['id']}) {d['text']}" for d, _ in hits)print(search_policies("conference hotel exemption")[:70])
Step 2: run the tools the model asks for
def run_tools(reply):results = []for block in reply.content:if block.type == "tool_use":print("search:", block.input["query"])results.append({"type": "tool_result", "tool_use_id": block.id,"content": search_policies(block.input["query"])})return results
Step 3: the agent loop
The agent loop repeats: call the model, run any tools it asked for, send the results back. It stops when the model answers without asking for a tool.
def agent(question, max_steps=5):messages = [{"role": "user", "content": question}]for _ in range(max_steps):reply = _client.messages.create(model=MODEL, max_tokens=1000, system=AGENT_SYSTEM,tools=TOOLS, messages=messages)messages.append({"role": "assistant", "content": reply.content})if reply.stop_reason != "tool_use":return "".join(b.text for b in reply.content if b.type == "text")messages.append({"role": "user", "content": run_tools(reply)})return "I couldn't finish this one. Please ask HR or IT directly."
Step 4: the question the pipeline missed
print(agent("Can I expense a Boston hotel for a conference my director approved, ""and do I still need a receipt for meals?"))
search: Boston hotel limit approvalsearch: conference hotel exemptionsearch: meal receiptsConference hotels booked through the organiser are exempt from the nightly limit(travel-conf). Otherwise Boston is $250 a night, and above that you need yourdirector's written approval before booking (travel-hotel). Meals: keep receiptsand submit them within 30 days (travel-meals).
The model split the question into three topics on its own and searched each one. In lesson 11 we wrote code to decompose questions. Here the model does it, and it can follow up when a result raises a new question.
Issue: the agent never stops
print(agent("What's the parking policy at the Denver office?"))
There's no parking policy, so every search comes back weak. Without a limit, some models keep rephrasing and searching. Each search is another model call, so cost and latency grow with every step. Two guards fix it. The step cap (max_steps=5) ends the loop in code, whatever the model wants. The stop rule in the system prompt tells the model to give up after two empty searches.
Microsoft's agentic RAG guidance recommends the same. Cap tool calls per request, typically at 5 to 10. Track tokens per request, and tell the agent in its instructions when to stop.
Agents cost more. This lab's conference question took four model calls where the pipeline took one. Anthropic's guide advises finding the simplest solution that works and adding complexity only when it clearly improves results. Use the agent for multi-part or exploratory questions, and the fixed pipeline for the simple ones. Lesson 22 routes between them.
Put in order
Put one agent turn in order
Drag, or focus an item and press Space, move with the arrow keys, then Space to drop.
- The model asks for search_policies with a query
- Send the question and tool list to the model
- The model reads them and decides: search again or answer
- Your code runs the search and returns passages
- The loop ends on an answer or the step cap
Worth keeping
- A workflow follows your code paths; an agent chooses its own steps.
- The tool description is the model's only documentation, so write it carefully.
- Cap the steps in code and give the model a stop rule.
- Agents cost several calls per question, so use them where one search isn't enough.
The agent trusts every search result it gets. When a search returns three chunks that are all slightly off-topic, it writes a confident answer from them. Next lesson teaches it to grade what it found and try again.
Leaderboard
Loading…
50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.
Sources for this lesson (3)
Each fact in the lesson is marked with the number of the page it comes from.