RAG systems end to end: from a basic pipeline to production agentic RAG
Build a company document assistant in Python and LangChain, then make it accurate, secure, measurable and agentic.
29 lessons · about 476 min · Free
Start the courseWhat it is
RAG is a pattern where your code finds the most relevant passages in your own documents and gives them to a language model to answer from.
Language models don't know your company's private, changing documents, and they invent answers when asked. RAG lets an assistant answer from those documents with citations you can check, without retraining the model.
What you'll be able to do
- Explain what RAG is, name its types, and choose the right one for a problem
- Prepare each document type and choose chunking strategies and retrieval algorithms to match
- Evaluate with retrieval and answer metrics using Ragas, DeepEval and LangSmith, and troubleshoot failures stage by stage
- Run RAG in production with permissions, injection defences, correct numbers, tracing and monitoring
- Build agentic RAG in plain Python and as a LangGraph graph, and learn from real deployments
Who it's for
Engineers who can write Python and want to ship a document assistant that a company can trust in production.
Not for people new to programming, or for researchers looking for model training methods; this course builds systems from existing models.
Before you start: Comfortable reading and writing Python; Have called an LLM API once (send a prompt, read the reply).
How it starts
Retrieval-augmented generation (RAG) is a pattern for answering from your own documents. Your code finds the passages that match a question, puts them in the prompt, and asks a language model to answer from them.
The model writes the answer. Your documents supply the facts. That split is the whole idea, and every lesson in this course improves one side of it.
Meet Brightline Logistics, a fictional 4,000-person freight company. Its employees ask the same questions every day. How much can I spend on a hotel in Chicago? How do I reset my VPN token?
Ask a general model that question…
What you'll learn
Foundations: what RAG is and a working pipeline
Getting started: what RAG is and when to use it
Explain what retrieval-augmented generation does, where it fits next to a plain model call, and when to skip it.
Types of RAG: a map from naive to agentic
Name the main kinds of RAG, from naive pipelines to graph, multimodal and agentic systems, and what problem each one solves.
Lab: build the smallest RAG pipeline end to end
Build a working question-answering pipeline over six policy documents in about 40 lines of Python.
How embeddings and vector search find meaning
Rebuild vector search from its parts and explain why similar meaning lands near in vector space.
Document types: what each one breaks, and how to prepare it
See how PDFs, scans, tables, spreadsheets, slides, emails, code, contracts and web pages each break a naive pipeline, and fix each one.
Better retrieval: chunking and search
Chunking documents so answers survive
Split long policy documents into chunks that keep their meaning and context, and measure the difference.
Chunking strategies compared: which to use when
Compare seven chunking strategies on the same documents and pick the right one for each document type.
Keyword search vs vector search, and fusing both
Compare BM25 and vector search on the same questions and combine them with reciprocal rank fusion.
Retrieval algorithms compared: which to use when
Map the retrieval algorithms, from exact search to ANN indexes, sparse, dense, multi-vector and MMR, and pick one per need.
Reranking the top results before the model reads them
Add a cross-encoder reranker that rescores the top candidates, and order context so the model uses it.
Rewriting the question before you search
Improve retrieval for vague, multi-part and follow-up questions by rewriting, splitting and expanding them.
Answers you can trust, measure and debug
Lab: grounded answers with citations and refusals
Write the generation step so answers cite their sources and say 'I don't know' when the context is silent.
Measuring a RAG system with an evaluation set
Build a small test set and score retrieval and answers so every change is measured, not guessed.
Lab: evaluation methods and frameworks: Ragas and DeepEval
Score Ask Brightline with the popular retrieval and answer metrics, using Ragas and DeepEval, and read what each score tells you.
Troubleshooting a RAG system, symptom by symptom
Trace a bad answer to the stage that caused it and apply the fix, using a repeatable debugging playbook.
Production in a company
Access control: only retrieve what the user may see
Enforce document permissions at retrieval time so the assistant never leaks restricted content.
Defending against prompt injection in retrieved text
Stop instructions hidden inside documents from taking over the assistant, using layered defences.
Running RAG in production: freshness, cost and monitoring
Keep the index fresh, control cost and latency, and watch the system so failures surface before users report them.
Real-world RAG: what worked, what failed, and why
Study verified deployments and failures in finance, customer support and law, and draw the design lessons from each.
Agentic RAG
Lab: from a fixed pipeline to an agent with a search tool
Turn retrieval into a tool the model can call, run a capped agent loop, and see when that beats a pipeline.
Self-correcting retrieval: grade, retry, fall back
Add a grader that checks retrieved chunks and the draft answer, then rewrites, retries or escalates.
GraphRAG and routing across many sources
Decide when a knowledge graph, a database query or plain retrieval answers a question best, and route to it.
Lab: numbers, tables and financial documents
See why basic RAG gets financial maths wrong, then fix it with structured extraction, a code tool and checks on every number.
Frameworks: LangChain, LangGraph and LangSmith
The RAG toolbox: what each library does and why you import it
Open up what pip installs for RAG: what each library contains, which depend on which, and which you can skip.
The framework landscape: LangChain, LangGraph, LlamaIndex, Haystack
Map what each RAG framework gives you, what it hides, and when plain Python is the better choice.
Lab: Ask Brightline in LangChain
Rebuild the RAG pipeline with LangChain's loaders, splitters, vector store, retriever and chat model.
Lab: agentic RAG as a LangGraph graph
Build the grade, rewrite and retry agent from earlier as an explicit LangGraph graph with a loop cap.
Lab: tracing, evaluating and monitoring with LangSmith
Trace every request, run evaluations on a dataset, and set up production monitoring and feedback in LangSmith.
Wrap-up: your production agentic RAG checklist
Review the full system you built and test your judgement on real company scenarios.
Sources
Lessons cite these where they use them.
- LangGraph docs: Overview
- Anthropic (2024), Building effective agents
- Microsoft Learn: Changed and deleted blobs in indexing
- LangChain docs: Migrating to LangChain v1
- Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- DoorDash Engineering: LLM-based Dasher support automation
- Microsoft Learn: Chunk large documents for RAG and vector search
- LangSmith docs: Online evaluations
- LangGraph docs: Build a custom RAG agent
- Chen et al. (2021), FinQA
- Pipitone and Alami (2024), LegalBench-RAG
- LlamaIndex Framework docs
- Edge et al. (2024), From Local to Global: A Graph RAG Approach
- Uber Engineering: Genie on-call copilot
- Liu et al. (2023), Lost in the Middle
- PyPI: sentence-transformers
- Microsoft Learn: Relevance scoring in hybrid search using RRF
- LangSmith docs: Alerts
- Formal et al. (2021), SPLADE
- Singh et al. (2025), Agentic RAG: A Survey
- Microsoft Learn: Document Intelligence layout model
- LangChain docs: Vector store integrations
- Haystack docs: Introduction
- Yan et al. (2024), Corrective Retrieval Augmented Generation
- Magesh et al. (2024), Hallucination-Free? Assessing legal AI research tools
- Asai et al. (2023), Self-RAG
- Uber Engineering: Enhanced Agentic-RAG
- Islam et al. (2023), FinanceBench
- PyPI: numpy
- Sawarkar et al. (2024), Blended RAG
- Ragas docs: Testset generation for RAG
- Gao et al. (2023), RAG for Large Language Models: A Survey
- AWS ML Blog (2023): How Q4 Inc. handled numerical data in its RAG chatbot
- OWASP Top 10 for LLM Applications 2025
- Es et al. (2023), Ragas: Automated Evaluation of RAG
- LangChain docs: Retrieval
- Jeong et al. (2024), Adaptive-RAG
- Databricks blog: Financial insights with NYSE/ICE
- LangSmith docs: Attach user feedback
- LangChain pricing (LangSmith plans)
- Carbonell and Goldstein (1998), MMR diversity-based reranking
- cAST (2025): chunking code by abstract syntax tree
- LangChain docs: Build a RAG agent
- DeepEval docs: Unit testing in CI/CD
- Khattab and Zaharia (2020), ColBERT
- Sentence Transformers docs: Cross-Encoder usage
- Gao et al. (2022), PAL: Program-aided Language Models
- Anthropic (2024), Introducing Contextual Retrieval
- LangSmith docs: Tracing quickstart
- Chen et al. (2022), Program of Thoughts Prompting
- pgvector README: HNSW, IVFFlat and filtering
- Günther et al. (2024), Late Chunking
- Faysse et al. (2024), ColPali
- FAISS wiki: Guidelines to choose an index
- Ragas docs: Context Recall
- DeepEval docs: Faithfulness metric
- Google Cloud blog (2025): Deutsche Bank DB Lumina
- SEC: EDGAR APIs (XBRL financial data)
- Morgan Stanley press release (March 2023)
- Auer et al. (2024), Docling Technical Report
- Ragas docs: Context Precision
- PyPI: langchain-core
- Ragas docs: Migrating from v0.3 to v0.4
- Ragas docs: Faithfulness
- LangSmith docs: Compare experiment results
- pgvector-python README
- Microsoft Learn: Security filters for trimming results
- LangSmith docs: Evaluation quickstart
- OpenAI customer story: Morgan Stanley
- LangSmith docs: Dashboards
- Docling documentation
- Microsoft Learn: Develop an agentic RAG solution
- Zhu et al. (2021), TAT-QA
- PyPI: langchain-huggingface
- PyPI: langchain (dependencies and Python version)
- Microsoft Learn: RAG in Azure AI Search
- Yao et al. (2022), ReAct: Synergizing Reasoning and Acting
- Microsoft Learn: RAG preparation phase
- Claude docs: Citations
- Unstructured docs: Partitioning
- Xu et al. (2024), RAG with Knowledge Graphs for Customer Service QA (LinkedIn)
- Gao et al. (2022), Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)
- Claude docs: Code execution tool
- Microsoft Learn: RAG chunking phase
- PyPI: langgraph
- Moffatt v. Air Canada, 2024 BCCRT 149
Leaderboard
Loading…
Questions
- What is this course about?
- RAG is a pattern where your code finds the most relevant passages in your own documents and gives them to a language model to answer from.
- What will I be able to do after it?
- Explain what RAG is, name its types, and choose the right one for a problem. Prepare each document type and choose chunking strategies and retrieval algorithms to match. Evaluate with retrieval and answer metrics using Ragas, DeepEval and LangSmith, and troubleshoot failures stage by stage. Run RAG in production with permissions, injection defences, correct numbers, tracing and monitoring. Build agentic RAG in plain Python and as a LangGraph graph, and learn from real deployments.
- Who is it for?
- Engineers who can write Python and want to ship a document assistant that a company can trust in production. Not for people new to programming, or for researchers looking for model training methods; this course builds systems from existing models.
- How long does it take?
- 29 lessons in 6 modules, about 8 hours in total. Your progress is saved, so you can take it in short sessions.
- What do I need to know first?
- Comfortable reading and writing Python. Have called an LLM API once (send a prompt, read the reply).
- Is it free?
- Yes. It's free; you sign in so your progress and quiz results are saved.
- Where does the content come from?
- Lessons cite 86 sources where they use them, including LangGraph docs: Overview; Anthropic (2024), Building effective agents; Microsoft Learn: Changed and deleted blobs in indexing.