Introduction to Retrieval-Augmented Generation (RAG)

No coding required — go from "what even is RAG?" to shipping a real, working assistant.

⏰ 8 weeks🎯 Standard🧩 Capstone🖥 Any device · our GPUs

🎯 About this course

Most RAG tutorials assume you already know how to program. This course doesn't. Every concept — from embeddings to orchestration — is taught first with a plain-language metaphor and a real business example, then practised hands-on with free, no-code tools. By the end, you won't just understand RAG in theory: you'll have built, tuned, evaluated and pitched a real, working assistant over a knowledge base you chose yourself.

✅ Prerequisites

No programming or coding experience required — every hands-on step in this course uses point-and-click, no-code tools.
You should be comfortable using a web browser, creating a free online account, and copying and pasting text.
Bring one real (or realistic) knowledge base you care about — a product FAQ, a course syllabus, a policy document — you will use it throughout the course and in your capstone.

Recommended courses to take first

Python & Maths for Machine Learning →

Curriculum

Topics covered

Understanding RAG — The Big Idea 6 topics

Why plain LLMs aren't enough
Ask ChatGPT (or any large language model) "What is our refund policy?" and it will confidently make something up — because it has never seen your company's refund policy. It was trained on a huge slice of the public internet up to some cutoff date, then frozen. It doesn't know about documents it has never read, and it doesn't automatically know when it's wrong. This shows up as two separate problems: - **Hallucination** — the model states something false with total confidence, because generating a plausible-sounding sentence is literally what it was built to do. It isn't lying; it simply has no built-in concept of "I don't actually know this." - **Staleness** — even what the model *does* know is frozen at its training cutoff. A model trained in early 2024 has never heard of a product you launched last month, a policy you changed last week, or an internal wiki page that has always been private. ```mermaid %% title: What a frozen model can and cannot see flowchart LR T["Public internet, up to the training cutoff"] --> M["Language model (frozen after training)"] M --> ANS["Answers from memory — and guesses when unsure"] P["Your private documents"] -. never seen .-> M R["Last week's policy change"] -. never seen .-> M W["Your internal wiki"] -. never seen .-> M ``` > 🏢 **Business case:** A telecom company's support chatbot, built on a plain LLM, once told a customer their unused data would "roll over automatically" — that had never been true. The team spent more on the resulting complaints and refunds than the chatbot project had cost to build. The fix wasn't a smarter model; it was giving the *existing* model the actual, current policy document to read before it answered. Retrieval-Augmented Generation (RAG) exists to solve exactly this pair of problems: it hands the model your real, current, private documents at the moment it's asked a question, instead of asking it to answer purely from memory. > 🌍 **In the wild:** This problem is well known enough that major cloud providers sell products specifically to fix it. Google Cloud's Vertex AI platform ships a feature literally called "grounding" that retrieves real, current sources before the model answers — the exact RAG idea you're about to learn, offered as a paid, managed product because so many businesses hit this exact wall. > 💡 **Tip:** Whenever you catch a language model confidently stating something you know to be wrong or outdated, ask yourself: "did this model actually have access to the source document, or was it guessing from training data?" That question is the seed of every RAG use case you'll design in this course.
What RAG actually is — the open-book exam metaphor
Imagine two students taking the same history exam. Student A has to answer purely from memory — anything they don't remember, they have to guess. Student B is allowed to bring the entire textbook into the exam room, look up the relevant chapter for each question, and then write their answer using both what they remember *and* what's on the page in front of them. Student B will always be more accurate, especially on obscure or recently-added facts. Retrieval-Augmented Generation turns every language model into "Student B." Concretely, RAG is a two-step process for answering any question: 1. **Retrieve** — search your own collection of documents for the handful of passages most relevant to the question being asked. 2. **Generate** — hand those passages to the language model *along with* the question, and ask it to answer using only what's in front of it. The name is literally a description of the process: **R**etrieval, **A**ugmented (the prompt is "augmented" with real content), **G**eneration (the model then generates the final answer). ```mermaid %% title: The three steps of every RAG answer flowchart LR Q["User question"] --> R["Retrieve: search your documents"] KB[("Your documents")] --> R R --> A["Augment: question + retrieved passages go into one prompt"] A --> G["Generate: language model writes the answer from that prompt"] G --> ANS["Grounded answer (+ citation)"] ``` > 💡 **Tip:** When you're explaining RAG to a non-technical stakeholder, skip the acronym entirely and use the open-book exam line — "we give the model the right page of the textbook before we ask it the question." It lands every time. > 🌍 **In the wild:** You've likely already used RAG without knowing its name. Reddit's own "Reddit Answers" feature retrieves real posts and comments from across the site before writing a summary answer — the "textbook" is the site's own content. Gmail's built-in Gemini assistant works the same way when you ask "when is my flight" — it retrieves the actual confirmation email from your inbox rather than guessing. And on X (formerly Twitter), the Grok assistant answers questions about "what's happening right now" by retrieving real, live posts from X first, then writing an answer grounded in them. Crucially, RAG doesn't change the language model itself at all. The same underlying model — say, a general-purpose assistant — becomes an expert on *your* company's HR policy, *your* product catalogue, or *your* medical guidelines purely because of what it's shown at answer time, not because it was retrained.
RAG vs. fine-tuning vs. plain prompting — choosing the right tool
There are three broad ways to make a language model behave the way you need. Beginners often reach for the most expensive one first, so it's worth knowing all three before you commit to anything. | Approach | What it does | Good for | Not good for | |---|---|---|---| | **Plain prompting** | You just ask well — clear instructions, maybe an example or two, in the message itself | Formatting, tone, simple reasoning tasks | Anything requiring facts the model was never trained on | | **RAG** | Retrieves relevant documents and includes them in the prompt at answer time | Grounding answers in current, private, or frequently-changing information | Teaching the model a brand-new *skill* or *style* it fundamentally lacks | | **Fine-tuning** | Retrains (part of) the model's internal weights on examples of the behaviour you want | Teaching a durable new skill, tone, or output format at low cost per call | Keeping the model up to date with fast-changing facts (you'd have to retrain constantly) | > 🏢 **Business case:** A legal-tech startup initially planned to fine-tune a model on their entire contracts database so it would "know" every clause. It was slow, expensive, and — worse — every time a contract was amended, they'd need to retrain again. They switched to RAG: the model stayed exactly as it was, and every answer simply retrieved the current version of the relevant clause. Cost dropped by an order of magnitude and answers became *more* accurate, because they were always based on the latest document, not a snapshot from training day. A useful rule of thumb: if the problem is **"the model doesn't know this fact"**, reach for RAG. If the problem is **"the model doesn't know how to do this kind of task at all"**, fine-tuning (or better prompting) is the right lever. Many production systems eventually use both — but nearly everyone should start with RAG, because it's cheaper, faster to build, and easier to keep current. ```mermaid %% title: Which lever to reach for flowchart TD START["The model isn't behaving how I need"] --> Q1{"Is it missing a FACT it was never taught?"} Q1 -- "Yes" --> Q2{"Does that fact change often, or is it private?"} Q2 -- "Yes" --> RAG["Use RAG (retrieve the current document at answer time)"] Q2 -- "No, it's fixed & small" --> PROMPT["Just put the fact in the prompt"] Q1 -- "No, it lacks a SKILL, tone or output format" --> Q3{"Can better instructions or examples fix it?"} Q3 -- "Yes" --> PROMPT2["Improve the prompt (few-shot examples)"] Q3 -- "No, it needs it baked in" --> FT["Fine-tune"] ``` > ⚠️ **Watch out:** RAG cannot fix a model that is bad at reasoning, or teach it a tone of voice it can't already produce. It only ever adds *facts* to a prompt — it doesn't change how the model thinks.
Real-world RAG in action — five industries, five examples
RAG is not a lab curiosity — it's already the backbone of many assistants you may have used without realising it. Five concrete examples, one per industry: ```mermaid %% title: Where RAG shows up in the real world mindmap root((RAG in the wild)) Customer support Return & refund policy Product manuals Internal knowledge HR handbook IT runbooks Legal & compliance Contract clause lookup Regulatory filings Healthcare Clinical guideline lookup Sales enablement Battlecards & pricing Public products Reddit Answers Gmail Gemini X Grok ``` 1. **Customer support (e-commerce).** A shopper asks "can I return this after the holiday season?" The assistant retrieves the exact, current return-window policy for that product category and answers from it — instead of guessing from generic e-commerce knowledge. 2. **Internal knowledge search (any large company).** An employee asks "how do I file an expense claim over ₹50,000?" The assistant searches the company's internal wiki/HR portal and answers with the actual current process, including who needs to approve it. 3. **Legal and compliance research.** A paralegal asks "what does clause 14.2 of the vendor agreement say about liability caps?" The assistant retrieves that exact clause from the actual signed contract, rather than describing what such a clause "typically" says. 4. **Healthcare guideline lookup.** A nurse asks "what's the current dosage guideline for this medication in paediatric patients?" The assistant retrieves from the hospital's current, approved clinical guidelines document — never from general internet knowledge, which could be outdated or region-specific. 5. **Sales enablement.** A sales rep asks "what's our competitive positioning against Competitor X for the enterprise tier?" The assistant retrieves the latest battlecard the marketing team uploaded last week. > 💡 **Tip:** Notice the pattern across all five: a *specific* question, about *specific*, *changeable* information, that lives in a document somewhere. That pattern — "a real document holds the true answer" — is the single best signal that a use case is a good fit for RAG. > 🌍 **In the wild:** The same pattern shows up at companies you already know. LinkedIn uses retrieval-style techniques in its "Collaborative Articles" AI features, pulling from real member-contributed knowledge rather than generating from scratch. Reddit's Answers feature retrieves real community discussions before answering. AWS sells a managed product for exactly this — Amazon Bedrock Knowledge Bases and Amazon Kendra — so that any company can plug their own document library into a retrieval pipeline without building one from scratch, and Google Cloud offers the direct equivalent with Vertex AI Search. You'll pick one of these (or a similar use case of your own) as your working example for the rest of this course — the same knowledge base will follow you from the concept modules all the way to your capstone.
The business case: cost, ROI and risk reduction
Before you build anything, it helps to be able to make the case for *why* a business should invest in a RAG assistant at all. Three angles matter most: ```mermaid %% title: Three ways a RAG assistant pays for itself flowchart TD RAG["RAG assistant"] --> D["Cost deflection: fewer repeat questions reach a human"] RAG --> S["Speed & availability: answers in seconds, 24/7, no manual retraining"] RAG --> G["Grounding & audit: every answer cites a fixable source document"] D --> ROI["Lower cost per answered question"] S --> ROI G --> TRUST["Enough trust to expand the assistant's scope"] ``` **1. Cost deflection.** Every question a RAG assistant answers correctly is a question a human support agent, HR generalist, or paralegal didn't have to spend time on. Teams typically track this as a *deflection rate* — the percentage of incoming questions fully resolved by the assistant without a human. Even a modest 20–30% deflection rate on a high-volume support queue can be a large recurring saving. **2. Speed and availability.** A RAG assistant answers in seconds, at 3 a.m., in any timezone, without needing to be trained on every policy update by hand. For a global business, this alone can be the deciding factor. **3. Risk reduction through grounding.** Because a well-built RAG system answers *from* a specific document and can *cite* it, mistakes are traceable and auditable — you can always find out exactly which document produced a wrong answer and fix that document, rather than trying to "un-teach" a fine-tuned model. > 🏢 **Business case:** An HR team estimated that 40% of employee queries to the People team were repeat questions already answered in the employee handbook. They built a RAG assistant over the handbook. Within a quarter, first-line HR queries to actual humans dropped by a third, and — because every assistant answer cited the exact handbook section — HR could trust it enough to expand it to benefits questions the following quarter. > 💡 **Tip:** When pitching a RAG project internally, don't lead with the technology. Lead with a number: "X% of our support tickets are questions our own documentation already answers." That's your business case in one sentence — and it's exactly the memo you'll write for this course's first assignment.
Where RAG shines and where it struggles
RAG is powerful, but it is not magic, and setting the right expectations early will save you a lot of frustration later. ```mermaid %% title: Good fit vs. poor fit for basic RAG flowchart TB Q["A user question"] --> C{"Does the answer live in one findable, current document?"} C -- "Yes" --> SHINE["RAG shines: policy / price / spec lookup, answered with a citation"] C -- "No" --> S2{"What kind of question is it?"} S2 -- "Needs many docs synthesised" --> HARD1["Needs extra design (multi-document reasoning)"] S2 -- "Live system state" --> HARD2["Not a retrieval problem at all"] S2 -- "Docs contradict each other" --> HARD3["Fix the knowledge base first"] ``` **Where it shines:** - Answering questions where the true answer lives in a specific, findable document. - Domains that change often (policies, prices, product specs) where retraining a model constantly would be impractical. - Situations where you need to *show your work* — a citation back to the source document builds trust. **Where it struggles:** - **Questions that require reasoning *across* many documents** — e.g. "summarise how our returns policy has changed over the last three years" requires synthesising several versions, which a simple retrieve-then-answer pipeline handles poorly without extra design. - **Documents that are poorly organised or contradictory.** If your knowledge base itself has two conflicting versions of a policy, RAG will faithfully retrieve and repeat the confusion. - **Very short, ambiguous questions** with no clear match in the knowledge base — the retriever may return irrelevant passages, and the model may still try to answer from them. - **Real-time facts that no document captures** — "how many people are in the support queue right now" is a live-systems question, not a document-retrieval question. > ⚠️ **Watch out:** The single most common beginner mistake is assuming RAG will be accurate simply because it "has access to documents." It's only as good as (a) how well those documents are chunked and organised, and (b) how good the underlying retrieval and prompt design are — both of which you'll practise hands-on later in this course. > 💡 **Tip:** Keep a running list of the *bad* answers your system gives while you're building. In this field, a well-kept "why did it get this wrong" log is worth more than a dozen textbook examples of it going right.

Architecture & Components 7 topics

The two-pipeline mental model: ingestion vs. query
Every RAG system, no matter how it's built, is really two separate pipelines that happen to share the same storage: **1. The ingestion pipeline** (runs ahead of time, whenever your documents change): take raw documents → break them into chunks → convert each chunk into a numeric representation (an embedding) → store the chunk and its embedding in a searchable index. This happens *once per document*, not once per question. **2. The query pipeline** (runs every time a user asks a question): take the question → convert it into the same kind of numeric representation → search the index for the most similar stored chunks → hand those chunks plus the question to the language model → return its answer. > 💡 **Tip:** If you remember nothing else about RAG architecture, remember this split. Almost every confusing question a beginner has ("why didn't it find my new document," "why is it slow," "why is it expensive") resolves once you ask "is this an ingestion problem or a query problem?" Think of ingestion as *stocking a library* and query as *a librarian answering a question using that library*. You stock the shelves once (and restock whenever new books arrive); every visitor who asks a question just uses whatever is already on the shelves. The rest of this module walks through every component in both pipelines, one at a time. ```mermaid %% title: The two pipelines share one vector database flowchart TB subgraph ING["INGESTION pipeline — runs when documents change"] direction LR D["Raw documents"] --> C["Split into chunks"] --> E1["Embed each chunk"] end subgraph QRY["QUERY pipeline — runs on every question"] direction LR U["User question"] --> E2["Embed the question"] --> S["Search for nearest chunks"] S --> P["Build prompt: question + top chunks"] --> L["Language model"] --> ANS["Answer"] end E1 --> VDB[("Vector database")] VDB --> S ```
Component: the knowledge base (your source documents)
The knowledge base is simply the collection of real documents you want your assistant to be able to answer from: PDFs, Word documents, web pages, spreadsheets, help-centre articles, wiki pages, even transcripts of past support conversations. Good knowledge bases share three traits: ```mermaid %% title: A document is only useful once it passes three checks flowchart LR DOC["A source document"] --> C1{"Current? (not an old draft)"} C1 -- "No" --> FIX1["Archive it"] C1 -- "Yes" --> C2{"Authoritative? (the one true version)"} C2 -- "No" --> FIX2["Delete the duplicates"] C2 -- "Yes" --> C3{"Selectable text? (not a scanned image)"} C3 -- "No" --> FIX3["Run OCR / get a text version"] C3 -- "Yes" --> IN["Ready to ingest"] ``` - **Current** — outdated documents produce outdated (and sometimes actively wrong) answers. A RAG system is only ever as fresh as its source documents. - **Authoritative** — one clearly "correct" version of each policy, not five drafts scattered across different folders with different last-edited dates. - **Findable text** — a scanned image of a document with no extractable text is invisible to a RAG system until it's been through text extraction (often called OCR). > 🏢 **Business case:** A company once built a RAG assistant over "the shared drive" without first cleaning it up — and the drive contained three versions of the same pricing sheet, two of them years out of date. The assistant would confidently quote whichever version it happened to retrieve, sometimes the wrong one. The actual fix took an afternoon of deleting outdated files — no code, no model change. > 💡 **Tip:** Before you touch any tool, spend 30 minutes just *auditing* your knowledge base: are there duplicate or conflicting versions of the same document? Archiving or deleting the outdated ones is the single highest-leverage, lowest-effort step you can take to improve RAG accuracy — and you'll do exactly this for your own knowledge base in this module's first hands-on task.
Component: chunking — breaking documents into retrievable pieces
You can't hand an entire 40-page policy manual to the model every time someone asks one small question — it would be slow, expensive, and the model would have to hunt through irrelevant pages for the answer. Instead, every document is split into smaller pieces called **chunks** *before* it's stored — this is called **chunking**. A chunk is typically a paragraph or a small group of paragraphs — small enough to be specific, but large enough to still make sense on its own without the rest of the document around it. ```mermaid %% title: One document becomes many independently-retrievable chunks flowchart LR DOC["40-page policy manual"] --> SPLIT{"Split on natural boundaries"} SPLIT --> K1["Chunk 1 (Returns policy)"] SPLIT --> K2["Chunk 2 (Sale items clause)"] SPLIT --> K3["Chunk 3 (Exchange policy)"] SPLIT --> K4["Chunk ...n"] K1 --> EMB["Embed each chunk"] K2 --> EMB K3 --> EMB K4 --> EMB EMB --> VDB[("Vector database")] ``` **Why chunk size matters:** - **Chunks too large** — you retrieve a lot of irrelevant text alongside the one useful sentence, diluting the model's attention and wasting budget. - **Chunks too small** — you lose surrounding context; a chunk that says "This does not apply to premium members" is useless if it's been separated from the sentence that says what "this" refers to. > 💡 **Tip:** A good starting rule of thumb for beginners: chunk by natural document structure first (one chunk per section or per FAQ question-and-answer pair), and only fall back to a fixed size (like "roughly 200 words") when a document has no clear structure at all. Structure-aware chunking is almost always better than blindly cutting every N words. > 🏢 **Business case:** An HR FAQ document was originally chunked by a fixed word count, which cut several question-and-answer pairs in half — the retriever would find "How many casual leave days do I get?" but the chunk boundary had sliced off the actual number from the answer. Switching to "one chunk per Q&A pair" fixed the entire category of complaints in one afternoon, with no other changes. You'll practise this exact skill — and this exact mistake — hands-on later in this course.
Component: embeddings — turning meaning into numbers
Computers can't compare the *meaning* of two pieces of text directly — they need numbers. An **embedding model** solves this by converting any piece of text into a long list of numbers (a "vector") such that pieces of text with *similar meaning* end up as *similar numbers*, even if they don't share a single word in common. ```mermaid %% title: Similar meaning lands nearby on the "meaning map" flowchart LR Q1["How do I reset my password?"] --> EM["Embedding model"] Q2["I forgot my login credentials, help!"] --> EM Q3["What are your office hours?"] --> EM EM --> V1(["• near..."]) EM --> V2(["• ...near"]) EM --> V3(["•    far away"]) V1 -. "close = same meaning" .- V2 ``` > 💡 **Tip:** Think of an embedding as GPS coordinates for meaning. "How do I reset my password?" and "I forgot my login credentials, help!" use almost no words in common, but an embedding model places them right next to each other on the meaning-map — because they're asking the same thing. Two sentences that use the *same words* but mean *opposite things* ("refunds are allowed" vs. "refunds are not allowed") can actually sit closer together than you'd expect, because embeddings capture topic and phrasing more than logical negation — which is exactly why retrieval alone is never the whole story. Every chunk in your knowledge base gets converted into one of these number-lists when it's ingested, and every incoming question gets converted into one too, using the *same* embedding model. Whichever stored chunks have coordinates closest to the question's coordinates are the ones retrieved. > ⚠️ **Watch out:** The embedding model used to store your documents and the one used to encode a new question must be the *same* model. Switching embedding providers means every document has to be re-processed from scratch — there's no way to "convert" embeddings from one model's coordinate system to another's. > 🌍 **In the wild:** Embeddings power far more than RAG. LinkedIn uses embedding-based matching at the core of its job and "People You May Know" recommendations — turning your profile and a job posting into number-lists and finding the closest match. Facebook and Instagram's News Feed and Explore ranking work the same way, finding posts whose embedding is close to what you tend to engage with. It's the same core idea you just learned, reused for a completely different purpose. You don't need to understand the mathematics behind this to use it well — you need to understand this one idea: embeddings measure *closeness of meaning*, and that's the entire trick behind "finding the right passage" in RAG.
Component: the vector database — your searchable memory
Once every chunk has been converted into an embedding (a list of numbers), you need somewhere to store all of those number-lists so you can quickly find the closest ones to any new question. That specialised storage system is called a **vector database** (sometimes just called a "vector store" or "vector index"). A vector database is purpose-built to answer one question extremely fast, even across millions of stored chunks: *"given this new list of numbers, which stored lists of numbers are the closest matches?"* This is fundamentally different from a normal database, which is built to find *exact* matches (like "find the customer whose ID is exactly 4471"), not *approximate closeness* of meaning. ```mermaid %% title: A normal database vs. a vector database flowchart TB subgraph NRM["Normal database"] NQ["WHERE id = 4471"] --> NR["The one exact row"] end subgraph VEC["Vector database"] VQ["Question vector [0.12, -0.44, ...]"] --> VR["The k nearest vectors (closest meaning)"] end ``` Popular hosted vector databases you'll encounter in the industry include Pinecone, Weaviate, Qdrant, and Chroma — most offer a free tier that's more than enough for learning and small projects, with a web dashboard so you never have to write a line of code to use them. > 💡 **Tip:** For your first project, a hosted, free-tier vector database is the right choice — it removes an entire category of setup problems (servers, storage, backups) so you can focus on getting your RAG pipeline working end to end. You'll set one up yourself in this course's infrastructure module. > 🏢 **Business case:** A small team once tried to avoid "another vendor" by storing embeddings in a plain spreadsheet and comparing them by hand with a formula. It worked for a 50-document demo and completely fell over at 5,000 documents — search became too slow to be usable. This is exactly the problem a vector database is purpose-built to solve, and why nearly every real RAG system uses one. > 🌍 **In the wild:** The major cloud providers all ship their own production-grade version of what you're about to set up. AWS offers vector search built directly into Amazon OpenSearch Service (and a fully managed retrieval layer on top, in Amazon Bedrock Knowledge Bases); Google Cloud offers Vertex AI Vector Search. Companies running at LinkedIn or Facebook scale — billions of items — use purpose-built internal versions of exactly this idea. You're learning the same architecture those systems run on; only the scale differs.
Component: the retriever — finding the right pieces
Given a question, it asks the vector database for the top few most-similar chunks (often called "top-k", e.g. "the top 5 closest chunks"). Those chunks are what gets handed to the model. There are two broad flavours worth knowing about, even before you build anything: - **Semantic (vector) search** — finds chunks with *similar meaning*, using embeddings as covered in the previous topic. Great at catching paraphrases and synonyms. - **Keyword search** — finds chunks containing the *exact words* used in the question, like a traditional search engine. Great at catching exact codes, names, and jargon that an embedding model might blur together with similar-sounding terms. - **Hybrid search** — runs both and combines the results. In practice, this catches more real user questions than either alone, because real questions are a mix of "vague paraphrase" and "exact term I need matched precisely" (like a product SKU or a policy number). ```mermaid %% title: Hybrid retrieval — two searches, merged, then trimmed flowchart TD Q["User question"] --> SEM["Semantic search (embeddings — catches paraphrases)"] Q --> KW["Keyword search (exact terms — catches codes & names)"] SEM --> M["Merge & rank (reciprocal rank fusion)"] KW --> M M --> TOPK["Keep top-k (e.g. top 5)"] TOPK --> RR["Optional: re-rank for precision"] RR --> CTX["Context passed to the model"] ``` > 💡 **Tip:** If your retriever keeps missing questions that use an exact product code, a person's name, or a specific number, that's a classic sign you need keyword or hybrid search, not a "smarter" embedding model — the embedding model was never designed to match exact codes reliably. > 🌍 **In the wild:** You use hybrid search every day without noticing. Gmail's own search bar blends exact keyword matching (an exact sender name, an exact word in a subject line) with smarter semantic matching (finding an email about "the flight confirmation" even if you search "travel booking"). Reddit's site search and LinkedIn's people search work the same way — neither pure keyword nor pure semantic search alone would satisfy how differently real users phrase the same request. The number of chunks retrieved (top-k) is itself a tuning knob: too few and you risk missing the right passage; too many and you dilute the prompt with irrelevant text and drive up cost. You'll get hands-on practice tuning exactly this setting later in the course.
Component: the generator and the orchestrator — writing the answer
Once the retriever has found the most relevant chunks, two more pieces finish the job: **The generator** is simply the language model itself (the same kind of model behind ChatGPT-style assistants). It receives the retrieved chunks *and* the original question in a single prompt, and is instructed to answer using only what it's been given. **The orchestrator** is the "conductor" that ties the whole pipeline together: it takes the incoming question, calls the retriever, builds the combined prompt from the retrieved chunks and the question, sends that prompt to the generator, and returns the final answer — often adding extra steps like conversation memory, citation formatting, or a safety check along the way. > 💡 **Tip:** In a no-code tool, the orchestrator is usually the visual canvas itself — the boxes-and-arrows diagram you build by dragging blocks (a "document loader" block, a "retriever" block, a "prompt" block, a "language model" block) and connecting them in order. You are the one designing the orchestration, even without writing code — you're just doing it visually instead of in a programming language. > 🌍 **In the wild:** "Orchestration" — chaining several AI components into one smooth pipeline — is not unique to RAG. Skype Translator, a long-running Microsoft feature, chains speech recognition, machine translation, and speech synthesis into a single pipeline so two people speaking different languages can have a live conversation — three separate AI components, orchestrated in sequence, exactly like the retriever-then-generator chain you're learning here. Putting the whole architecture together, end to end — here it is as a UML component view, showing what each part depends on: ```mermaid %% title: RAG components and how they connect (UML class view) classDiagram class KnowledgeBase { +documents +keepCurrent() +removeDuplicates() } class Chunker { +chunkSize +overlap +split(document) } class EmbeddingModel { +dimension +embed(text) vector } class VectorDatabase { +addChunk(vector, text) +search(queryVector, k) chunks } class Retriever { +topK +searchType +retrieve(question) chunks } class Generator { +model +answer(prompt) text } class Orchestrator { +handle(question) answer +buildPrompt(question, chunks) +addCitations() } KnowledgeBase --> Chunker : feeds Chunker --> EmbeddingModel : each chunk EmbeddingModel --> VectorDatabase : stores vectors Retriever --> EmbeddingModel : embeds question Retriever --> VectorDatabase : nearest-neighbour search Orchestrator --> Retriever : gets context Orchestrator --> Generator : sends prompt ``` That's the entire architecture of a RAG system. Every tool you'll touch in this course — no matter how fancy its dashboard looks — is built from exactly these pieces.

How Orchestration Works 5 topics

A question's journey — the request lifecycle end to end
Let's trace one real question through a working RAG system, step by step, so the architecture from the last module stops being abstract. **Question:** *"Can I get a refund if I bought the item during a sale?"* 1. The question arrives at the orchestrator. 2. The orchestrator sends the question to the embedding model, getting back its numeric representation. 3. The orchestrator asks the vector database: "which stored chunks are closest to this?" It gets back, say, the top 4 chunks — perhaps: the general refund policy, the sale-items clause, the exchange policy, and an unrelated shipping-delay clause (retrieval isn't always perfect). 4. The orchestrator builds a prompt roughly like: *"Using only the following context, answer the question. If the context doesn't contain the answer, say you don't know. Context: [the 4 chunks]. Question: Can I get a refund if I bought the item during a sale?"* 5. That full prompt is sent to the generator (the language model). 6. The model reads the context, notices the sale-items clause directly addresses the question, and answers from it — ideally citing which chunk it used. 7. The final answer (and, in a well-built system, its citation) is returned to the user. ```mermaid %% title: One question's journey through the system sequenceDiagram actor User participant Orch as Orchestrator participant Emb as Embedding model participant VDB as Vector database participant LLM as Language model User->>Orch: "Can I get a refund if I bought it during a sale?" Orch->>Emb: embed the question Emb-->>Orch: question vector Orch->>VDB: find nearest chunks VDB-->>Orch: top 4 chunks (some relevant, some not) Orch->>Orch: build prompt (context + question + "say I don't know if unsure") Orch->>LLM: send prompt LLM-->>Orch: answer grounded in the sale-items clause (+ citation) Orch-->>User: final answer with its source ``` > 💡 **Tip:** Every single failure you'll ever debug in a RAG system happens at one of these seven steps. When something goes wrong, don't guess — trace it: was the wrong chunk retrieved (step 3)? was the prompt built badly (step 4)? or did the model ignore good context it was given (step 6)? Each has a completely different fix. > 🌍 **In the wild:** When you ask X's Grok assistant "what are people saying about [some event] right now," it runs this exact seven-step lifecycle live — except its "knowledge base" is a constantly-updating stream of real posts on X rather than a fixed set of PDFs. The architecture is identical to what you're learning here; only the ingestion pipeline runs continuously instead of ahead of time. Notice that steps 1–7 all happen in under a couple of seconds, and they happen *fresh, every single time* a question is asked — nothing about this process was "trained into" the model in advance.
Building the augmented prompt — combining context and question
The single most underrated skill in RAG is writing the instructions that wrap around the retrieved context — often called the **prompt template**. A weak template is the most common reason a technically-correct pipeline still gives poor answers. A solid, beginner-friendly template has four ingredients: 1. **A role and boundary** — "You are a helpful assistant that answers only from the provided context." 2. **The retrieved context, clearly marked** — so the model can tell what's "the source material" versus what's "the question." 3. **An explicit instruction for the unknown case** — "If the answer is not contained in the context, say you don't know — do not guess." 4. **The actual question.** ```mermaid %% title: The four parts that get assembled into one prompt flowchart LR R["1. Role & boundary 'answer only from context'"] --> P(("Final prompt")) C["2. Retrieved context (clearly delimited)"] --> P U["3. Unknown-case rule 'say you don't know'"] --> P Q["4. The user's question"] --> P P --> LLM["Language model"] ``` A simple example, in plain English rather than code: ``` You are a helpful assistant. Answer ONLY using the context below. If the context does not contain the answer, say "I don't know based on the information I have." Context: """ [retrieved chunk 1] [retrieved chunk 2] """ Question: [the user's question] ``` > 💡 **Tip:** That one sentence — "if the answer is not contained in the context, say you don't know" — is worth more to your system's trustworthiness than almost any other single change you can make. Beginners often skip it because it feels obvious; experienced teams treat it as non-negotiable. > ⚠️ **Watch out:** Even with a good template, a model will sometimes still "fill in the gaps" from its own general knowledge rather than admitting it doesn't know. This is why later in the course you'll learn to *test* this behaviour deliberately, not just assume the instruction worked.
Orchestration tools — no-code, low-code and code
You never have to write a program to build a real RAG pipeline today. The whole spectrum, from friendliest to most flexible: - **No-code visual builders** (e.g. Flowise, Langflow) — you drag boxes onto a canvas (a document loader, a chunker, an embedder, a vector store, a retriever, a prompt, a language model) and connect them with arrows. This is exactly what you'll use for hands-on work in this course. - **Low-code / notebook tools** — pre-built templates where you fill in a few settings (an API key, a folder path) without writing logic yourself, but the underlying steps are shown in a linear notebook rather than a visual canvas. - **Full code frameworks** (e.g. LangChain, LlamaIndex) — libraries that professional developers use to write custom RAG pipelines with fine-grained control. Powerful, but requires programming experience. ```mermaid %% title: The orchestration spectrum — all four run the same architecture flowchart LR A["No-code visual builders Flowise, Langflow you'll use this"] --> B["Low-code / notebook fill-in-the-blank templates"] B --> C["Code frameworks LangChain, LlamaIndex"] C --> D["Fully managed platforms AWS Bedrock Knowledge Bases Google Cloud Vertex AI Search"] A -.->|"more control, more setup →"| D ``` > 💡 **Tip:** Every one of these tools is orchestrating the *exact same* architecture you learned in the last module — document loader, chunker, embedder, vector store, retriever, prompt, generator. Learning the concepts first (as you're doing in this course) means you can pick up *any* of these tools later, including the code-based ones, without starting from zero. > 🏢 **Business case:** Many real companies validate a RAG idea with a no-code tool first — proving the business case and getting real user feedback in days — and only invest in a custom, code-based build once the no-code prototype has shown clear value and needs to scale beyond what the visual tool comfortably handles. You'll follow exactly this path: prototype now, in this course, with no-code tools. > 🌍 **In the wild:** There's a fourth rung above all three: fully managed enterprise platforms, where a cloud provider runs the entire orchestration layer for you. AWS's Amazon Bedrock Knowledge Bases and Google Cloud's Vertex AI Search both let a large company point at their documents and get a production-grade RAG pipeline with almost no setup — the same building blocks you're learning, packaged and run at massive scale. Once a no-code prototype like yours proves the idea, this is often where it goes next inside a real business.
Memory and multi-turn conversations
So far we've treated every question as if it arrived in isolation. Real conversations don't work that way — a user might ask "what's the return window for electronics?" and then follow up with "and what about the same thing for clothing?" — that second question only makes sense if the assistant remembers what "the same thing" refers to. **Conversation memory** is the mechanism that carries recent turns of the conversation forward so follow-up questions can be understood. In an orchestrated pipeline, this usually means: before retrieving, the orchestrator first rewrites a vague follow-up question into a *complete, standalone* question using the recent conversation history — e.g. turning "what about clothing?" into "what is the return window for clothing?" — and only *then* runs retrieval on the rewritten question. ```mermaid %% title: A vague follow-up is rewritten before retrieval runs sequenceDiagram actor User participant Orch as Orchestrator participant Hist as Conversation history participant VDB as Vector database participant LLM as Language model User->>Orch: "and what about the same thing for clothing?" Orch->>Hist: fetch recent turns Hist-->>Orch: earlier Q was "return window for electronics?" Orch->>Orch: rewrite to standalone: "what is the return window for clothing?" Orch->>VDB: retrieve on the rewritten question VDB-->>Orch: clothing returns clause Orch->>LLM: answer from that context LLM-->>User: "Clothing can be returned within 30 days…" ``` > ⚠️ **Watch out:** If you skip this step, retrieval on a bare follow-up like "what about clothing?" will often fail entirely, because that sentence alone doesn't contain enough meaning for the embedding model to find the right chunk. This is one of the most common "why does my chatbot get confused after the first message" bugs beginners hit. > 💡 **Tip:** Most no-code orchestration tools include a ready-made "conversational memory" block you can simply drop into your flow — you rarely need to design this rewriting step from scratch. Your job is to know *why* it's there and to test that it's actually working, which you'll do in this course's testing task.
Guardrails, citations and "I don't know" in the orchestration layer
Two orchestration-layer features separate a trustworthy RAG assistant from a risky one: **Citations.** A well-designed pipeline doesn't just return an answer — it returns *which document and passage* the answer came from. This can be as simple as instructing the model to end its answer with "(Source: Refund Policy, section 3)" or as sophisticated as returning clickable links back to the exact source. Citations let a human quickly verify a surprising answer, and let you trace a wrong answer back to a fixable document. **Explicit "I don't know" handling.** As covered in the prompt template topic, the model should be told to say it doesn't know when the retrieved context doesn't contain the answer. But orchestration can go one step further: some pipelines check *how relevant* the retrieved chunks actually were (a similarity score) and refuse to even attempt an answer if nothing scored above a set threshold — catching cases where the model might otherwise ignore the instruction and guess anyway. ```mermaid %% title: A confidence gate before the model is even called flowchart TD Q["User question"] --> RET["Retrieve top chunks + their similarity scores"] RET --> GATE{"Top score above the threshold?"} GATE -- "No" --> IDK["Return a fixed honest fallback: 'I couldn't find a confident answer'"] GATE -- "Yes" --> BUILD["Build prompt from those chunks"] BUILD --> LLM["Language model answers"] LLM --> CITE["Attach citation to each claim"] CITE --> OUT["Answer + sources to the user"] ``` > 🏢 **Business case:** A healthcare-adjacent assistant added a rule at the orchestration level: if no retrieved chunk scored above a minimum relevance threshold, the system would automatically respond "I don't have a confirmed answer for this — please contact a pharmacist" instead of letting the model attempt one. This single guardrail, added after a near-miss in testing, became the team's most-cited reason the system was approved for real use. > 💡 **Tip:** Measure your system's "I don't know" rate on purpose, using a mix of answerable and deliberately unanswerable test questions. A rate near zero on the *unanswerable* group is a red flag — it usually means your system is quietly fabricating grounded-sounding answers instead of admitting a gap. You'll build exactly this kind of test set later in the course.

Essentials, Trade-offs & Business Fit 5 topics

The essentials checklist — what you must get right
Before your first RAG project, it helps to have a short checklist of the things that matter most — in rough order of impact: ```mermaid %% title: The essentials, in order of impact (top matters most) flowchart TD KB["1. Clean, current, authoritative knowledge base"] --> CH["2. Sensible, structure-aware chunking"] CH --> EM["3. One embedding model, used consistently"] EM --> RT["4. A retriever tuned for your content"] RT --> PT["5. A prompt template: 'answer only from context'"] PT --> CI["6. Citations on every answer"] CI --> EV["7. A test set, re-checked regularly"] ``` 1. **A clean, current, authoritative knowledge base.** Garbage in, garbage out — this beats every other item on this list combined. 2. **Sensible chunking** that respects the natural structure of your documents. 3. **The same embedding model** used consistently for both documents and questions. 4. **A retriever tuned for your content** — semantic, keyword, or hybrid, with a sensible top-k. 5. **A prompt template that instructs the model to answer only from context**, and to say "I don't know" otherwise. 6. **Citations**, so answers can be verified and wrong answers can be traced to a fixable source. 7. **A way to measure quality** — a test set of real questions with known-good answers, checked regularly, not just once at launch. > 💡 **Tip:** Notice that only two of these seven (embedding consistency, retriever tuning) are really "technical" in the traditional sense. The other five are things anyone — including someone with zero programming background — can own and improve directly. This is exactly why RAG is such an accessible entry point into building real AI systems. > ⚠️ **Watch out:** Teams that skip straight to "let's pick the fanciest vector database" before fixing a messy knowledge base almost always end up rebuilding later. Always start at the top of this list, not the bottom.
The pros of RAG
A clear-eyed summary of what RAG genuinely gets you, useful when you need to defend the approach to a sceptical stakeholder: ```mermaid %% title: What RAG buys you mindmap root((Pros of RAG)) Always current edit the doc, next answer reflects it Cheap to keep fresh no retraining runs Traceable & auditable every answer cites a source Works with private data read at answer time, not trained in Less hallucination with 'answer only from context' Accessible to build no-code tools are enough ``` - **Always current** — update the source document, and the very next answer reflects the change. No retraining, no waiting. - **Cheap to keep fresh** compared to fine-tuning, which requires a full retraining run (and real cost) every time the underlying facts change. - **Traceable and auditable** — a good implementation can always show *which* document produced an answer, which matters enormously in regulated or compliance-sensitive settings. - **Works with private data** without exposing it to a model's training process — your documents are only ever read at answer time, not baked into the model's weights. - **Reduces (but does not eliminate) hallucination**, especially when paired with a strong "answer only from context" instruction and a confidence threshold. - **Accessible to build**, especially with no-code tools — as you're learning firsthand in this course. > 💡 **Tip:** When someone asks "why not just use ChatGPT directly?", the honest answer is usually some combination of these six points — pick whichever two matter most for your specific business case, and lead with those rather than reciting the whole list. > 🌍 **In the wild:** The "always current" benefit is exactly why Gmail's Gemini assistant can answer "when is my flight" correctly the day after you receive the confirmation email — it's retrieving from your live inbox, not a frozen training snapshot. The same logic is why Reddit's Answers feature can discuss last week's community discussions accurately: it retrieves the actual current posts rather than relying on knowledge baked in months earlier.
The cons and risks of RAG
An equally clear-eyed look at the honest downsides — knowing these up front will save you from over-promising what a RAG project can deliver: ```mermaid %% title: The honest downsides mindmap root((Cons & risks)) Only as good as your documents bad docs, bad answers Retrieval can fail silently fluent but wrong answer Added latency & cost extra step + bigger prompt Ongoing maintenance not build-once-and-forget Weak at multi-doc reasoning needs extra design ``` - **It's only as good as your documents.** Outdated, contradictory, or poorly-written source material produces outdated, contradictory, or poorly-written answers — RAG cannot fix bad documentation, it can only faithfully retrieve it. - **Retrieval can fail silently.** If the right chunk simply wasn't retrieved (wrong chunking, wrong embedding, question phrased unusually), the model may still generate a fluent-sounding but wrong or incomplete answer. - **Added latency and cost.** Every question now requires an extra retrieval step, plus a (usually larger) prompt to the model containing all the retrieved context — both add time and cost compared to a plain prompt. - **Setup and maintenance overhead.** Someone has to keep the knowledge base current, re-run ingestion when documents change, and periodically check quality — RAG is not "build once and forget." - **It cannot reason across many documents well** by default — synthesising a trend across dozens of documents needs extra design beyond basic retrieve-then-answer. > 🏢 **Business case:** A firm rolled out a RAG assistant expecting it to fully replace a support team within a month. It reduced ticket volume meaningfully, but never approached "full replacement" because a meaningful share of questions needed judgement calls no document could resolve. The lesson: pitch RAG as a force-multiplier that handles the well-documented majority of questions well, not a complete substitute for human judgement. > 💡 **Tip:** Keep this list next to the pros list from the previous topic. A credible business case names both — stakeholders trust a realistic pitch far more than an overly optimistic one.
A decision framework — should your business use RAG?
A simple framework, in the form of five yes/no questions, to sanity-check whether a use case is a good first RAG project: 1. **Does the true answer live in a specific, findable document?** (If the answer requires judgement, opinion, or live data no document captures, RAG alone won't help.) 2. **Does that information change often enough that "baking it into a model" would go stale quickly?** (If it never changes, plain prompting with the fact included might be simpler than building a whole pipeline.) 3. **Is there real volume** — enough repeat questions to justify the setup and maintenance effort? 4. **Can you tolerate the assistant being wrong sometimes, with a human fallback available?** (Every RAG system, however well built, will sometimes retrieve the wrong chunk or misread it.) 5. **Do you have someone willing to own the knowledge base** — keeping it current, watching quality, fixing bad documents when they cause bad answers? ```mermaid %% title: Should this use case be your first RAG project? flowchart TD Q1{"Does the true answer live in a specific, findable document?"} Q1 -- "No" --> STOP["RAG alone won't help — reshape the idea"] Q1 -- "Yes" --> Q2{"Does that information change often / is it private?"} Q2 -- "No, fixed & public" --> PROMPT["Simpler: put the fact in the prompt"] Q2 -- "Yes" --> Q3{"Enough repeat volume to justify the effort?"} Q3 -- "No" --> WAIT["Not worth it yet"] Q3 -- "Yes" --> Q4{"Can you tolerate occasional wrong answers + a human fallback?"} Q4 -- "No" --> RISK["Too risky without heavy guardrails"] Q4 -- "Yes" --> Q5{"Someone owns the knowledge base long-term?"} Q5 -- "No" --> OWNER["Find an owner first"] Q5 -- "Yes" --> GO["Strong first project — scores 4–5 / 5"] ``` > 💡 **Tip:** Score a candidate use case out of 5 using this list before you build anything. A use case that scores 4 or 5 is an excellent starting project; a use case that scores 2 or below is a sign to either reshape the idea or pick a different first project. You'll apply this exact framework — in writing — as this module's assignment, using the real (or realistic) use case you chose back in the concept module.
Business use-case matrix — support, sales, HR, compliance, research
A working reference table connecting business function to the kind of knowledge base and value each typically brings: ```mermaid %% title: Each business function has its own knowledge base flowchart LR SUP["Customer support"] --> SUPKB[("Help articles, manuals, policies")] SALES["Sales enablement"] --> SALESKB[("Battlecards, pricing, case studies")] HR["HR / People"] --> HRKB[("Handbook, benefits, leave policy")] LEGAL["Legal / compliance"] --> LEGALKB[("Contracts, filings, policy")] IT["Engineering / IT"] --> ITKB[("Runbooks, architecture, postmortems")] RES["Research / analysis"] --> RESKB[("Reports, papers, prior projects")] ``` | Function | Typical knowledge base | Typical win | |---|---|---| | **Customer support** | Help-centre articles, product manuals, policy documents | Faster first response, lower ticket volume for repeat questions | | **Sales enablement** | Battlecards, pricing sheets, case studies | Reps get accurate competitive answers in seconds during a live call | | **HR / People** | Employee handbook, benefits guides, leave policy | Fewer repeat questions to HR generalists, consistent answers | | **Legal / compliance** | Contracts, regulatory filings, internal policy | Faster clause lookup, auditable citations for every answer | | **Internal engineering / IT** | Runbooks, architecture docs, incident postmortems | Faster onboarding, less tribal knowledge lost when people leave | | **Research / analysis** | Reports, papers, past project documentation | Faster literature review, consistent summarisation of prior work | > 💡 **Tip:** Every one of these can be evaluated with the same five-question framework from the previous topic. When you're brainstorming your own capstone idea, scan this table for the function closest to a problem you personally understand — domain familiarity is one of the biggest predictors of a good capstone, because you'll be able to judge for yourself whether an answer is actually right.

Setting Up Your Infrastructure 6 topics

Preparing your knowledge base documents
Every hands-on module in this course uses one real knowledge base that you choose and keep using throughout — a product FAQ, a course syllabus, an employee handbook, a hobby project's documentation, anything you have genuine, current documents for and can judge answers about yourself. ```mermaid %% title: Getting your knowledge base ready flowchart LR G["Gather 3–10 real, current documents"] --> T["Check each has selectable text"] T --> D["Remove outdated & duplicate versions"] D --> Q["Write 8–10 test questions + your expected answers"] Q --> SET["Set the questions aside until the evaluation task"] ``` **What to gather:** - 3–10 documents (PDF, Word, or plain text) totalling anywhere from a few pages to a couple of hundred — enough to be interesting, not so much that reviewing it by hand becomes impossible while you're learning. - Documents that are the *current, correct* version — not drafts, not documents you know are outdated. **What to check before moving on:** - Open each document and confirm you can select and copy its text (a scanned image with no selectable text will need extra processing most beginner tools don't yet handle well — swap it for a text-based version if you can). - Remove or set aside any duplicate or clearly outdated versions, per the "authoritative source" principle from the architecture module. > 💡 **Tip:** Keep a simple text file next to your documents listing 8–10 real questions you'd expect someone to ask, along with what you personally believe the correct answer is. This becomes your evaluation test set later in the course — write it now, while the documents are fresh in your mind, and resist the temptation to look at it again until you're actually testing. > ⚠️ **Watch out:** Choosing documents you don't actually understand well makes every later exercise harder, because you won't be able to tell a right answer from a plausible-sounding wrong one. Familiarity beats sophistication for a learning project.
Getting and securing an LLM API key
To use a language model inside a no-code tool, you'll need an **API key** — a long, private password-like string that lets a tool call a language model provider's service on your behalf and (usually) bills a small amount per use. ```mermaid %% title: Get an API key and keep it safe flowchart TD A["Create a provider account"] --> B["Open the 'API keys' section"] B --> C["Create a key, name it 'intro-to-rag-course'"] C --> D["Copy it once → store in a password manager"] D --> E["Set a spending cap or alert"] C -. "never paste into" .-> X["public chat · shared doc · a code repository"] ``` **General steps (the exact screens vary by provider, but the shape is always the same):** 1. Create an account with a language model provider (well-known options include OpenAI, Anthropic, and Google — any of these work fine for this course). 2. Find the "API keys" section of your account dashboard (usually under developer or account settings). 3. Create a new key, give it a clear name (e.g. "intro-to-rag-course"), and copy it somewhere safe *immediately* — most providers only show the full key once. 4. Set a small spending limit if the option is available, so you can experiment freely without risking a surprise bill. > ⚠️ **Watch out:** Treat an API key exactly like a password. Never paste it into a public chat, a shared document, or a public code repository — anyone who has it can use your account and spend your money. If you ever suspect a key has leaked, revoke it immediately from your provider dashboard and create a new one. > 💡 **Tip:** Most providers offer a small amount of free trial credit for new accounts — more than enough to complete every hands-on task in this entire course. Set a spending cap anyway; it costs nothing and removes any anxiety about experimenting freely. Keep this key on hand — you'll paste it into your no-code orchestration tool later in this module, and nowhere else.
Setting up a hosted vector database
Next, set up a place to store your documents' embeddings once they're created. A hosted, free-tier vector database is the right choice for this course — no servers, no installation. ```mermaid %% title: Create an empty vector index flowchart TD A["Create a free hosted vector-database account"] --> B["Create a new index / collection"] B --> C["Set its 'dimension' to match your embedding model exactly"] C --> D["Copy the connection details (endpoint + API key)"] C -. "if mismatched later" .-> E["'Dimension mismatch' error → delete & recreate (index is still empty)"] ``` **General steps (again, screens vary by provider — Pinecone, Weaviate Cloud, Qdrant Cloud, and Chroma Cloud all offer a similar free-tier flow):** 1. Create a free account with a hosted vector database provider. 2. Create a new "index" or "collection" — this is the empty container your document chunks will be stored in. You'll usually be asked for a name (e.g. "intro-to-rag-kb") and a "dimension" number. 3. **About that dimension number:** it must exactly match the output size of the embedding model you plan to use (a common embedding model outputs 1536 numbers per chunk, for example — check your chosen embedding model's documentation for its exact number). Getting this wrong is the single most common first-time setup error. 4. Copy the index's connection details (an endpoint URL and/or an API key) — you'll paste these into your orchestration tool in the next topic. > ⚠️ **Watch out:** If your orchestration tool later complains about a "dimension mismatch," it almost always means the vector database index was created with a different dimension number than the embedding model actually produces. The fix is simply to delete and recreate the index with the correct number — no data is lost, because at this stage the index is still empty. > 💡 **Tip:** Leave a browser tab open on your vector database's dashboard while you work through the rest of this module — being able to watch chunks actually appear in your index as you test your pipeline is one of the best "aha, I understand this now" moments in the whole course. > 🌍 **In the wild:** If you already have an AWS or Google Cloud account through work or study, know that the same idea exists there under a different name — Amazon OpenSearch Service (with its k-NN vector search) on AWS, and Vertex AI Vector Search on GCP. They work on the identical principle you're setting up now; a free-tier dedicated vector database is simply the fastest way to learn the concept before you'd ever need that heavier infrastructure.
Choosing and setting up a no-code orchestration tool
A visual, drag-and-drop orchestration builder. Two widely used, free, open options are **Flowise** and **Langflow** — both give you a canvas of connectable blocks representing exactly the components you learned in the architecture module. ```mermaid %% title: Set up the orchestration canvas flowchart LR A["Sign up for Flowise or Langflow (hosted)"] --> B["Start a new empty flow"] B --> C["Add credentials once: LLM API key + vector-DB details"] C --> D["Locate the blocks: Loader · Splitter · Embeddings · Vector Store · Retriever · Prompt · LLM"] ``` **General setup steps:** 1. Sign up for a hosted version of your chosen tool (both offer a hosted option so you don't need to install anything locally), or follow the tool's one-click "deploy" option if you'd prefer to host it yourself later. 2. Once inside, start a **new flow** (an empty canvas). 3. In the tool's settings or "credentials" area, add your LLM provider's API key (from the previous topic) and your vector database's connection details (from the topic before that). Good tools store these securely and let every block in your flow reuse them without re-typing. 4. Familiarise yourself with the block palette — look specifically for blocks named something like "Document Loader," "Text Splitter" (this is the chunker), "Embeddings," "Vector Store," "Retriever," "Prompt Template," and "LLM" (or "Chat Model"). You now recognise every one of these from the architecture module — you're simply about to connect them visually instead of describing them in words. > 💡 **Tip:** Don't try to build the whole pipeline in one sitting. Add one block, run the flow, confirm it worked (or read the error message carefully), then add the next block. This "one block at a time" habit will save you hours of confused debugging compared to wiring ten blocks together and then discovering something is wrong somewhere in the middle. > ⚠️ **Watch out:** A block turning red or showing an error icon almost always means a required setting is missing or a credential wasn't saved correctly — read the specific error message rather than assuming the whole approach is wrong. Most first-time errors are one missing field, not a fundamental problem.
Wiring it together — your first end-to-end RAG flow
With your tool set up and credentials saved, it's time to connect the blocks into one working pipeline, mirroring the two-pipeline architecture from earlier in this course. **The ingestion side of your flow:** 1. Add a **Document Loader** block and point it at one of your prepared documents. 2. Connect it to a **Text Splitter** (chunker) block — set a reasonable chunk size to start (your tool's default is usually a sensible starting point). 3. Connect that to an **Embeddings** block, configured with your LLM provider and the same embedding model whose dimension you used when creating your vector index. 4. Connect that to a **Vector Store** block, configured with your hosted vector database's connection details from earlier. 5. Run this ingestion side first, on its own, for one document. Then check your vector database's dashboard — you should see chunks now appear inside your index. This is the single best confirmation that ingestion is working correctly. **The query side of your flow:** 6. Add a **Retriever** block connected to the same Vector Store. 7. Add a **Prompt Template** block, using a template like the one from the orchestration module (context + question + "say you don't know if unsure"). 8. Connect the Retriever's output and the incoming question into the Prompt Template, then connect that to an **LLM / Chat Model** block, configured with your API key. 9. Add a simple chat interface (most tools provide one built in) so you can type a question and see the final answer. ```mermaid %% title: The blocks you connect in your no-code tool flowchart TB subgraph ING["Ingestion side — run once per document"] direction LR DL["Document Loader"] --> TS["Text Splitter (chunker)"] --> EM["Embeddings"] --> VS1["Vector Store"] end subgraph QRY["Query side — runs on every chat message"] direction LR CHAT["Chat input"] --> RT["Retriever"] RT --> PT["Prompt Template (context + question + 'say I don't know')"] PT --> CM["LLM / Chat Model"] --> UI["Answer in chat UI"] end VS1 --> VDB[("Hosted vector database")] VDB --> RT ``` > 💡 **Tip:** Ask a question you *know* the answer to first — one taken directly, almost word for word, from a document you ingested. If that doesn't work, nothing more advanced will either. Only once easy questions succeed should you try harder paraphrases. > ⚠️ **Watch out:** If you change your knowledge base documents later, remember to re-run the *ingestion* side of the flow — the query side always searches whatever was last stored, not your latest files on disk.
Testing and debugging your setup
Before moving on, run through this checklist against your own working flow — treat it the way you'd treat a pre-flight checklist, not a one-time formality. ```mermaid %% title: The four-question test, and what each failure points to flowchart TD T1["1. Easy, direct question"] --> T2["2. Paraphrased question"] T2 --> T3["3. Unanswerable question"] T3 --> T4["4. Follow-up that needs memory"] T2 -. "fails" .-> F1["→ chunking / embedding setup"] T3 -. "answers anyway" .-> F2["→ stronger 'say I don't know' rule"] T4 -. "gets confused" .-> F3["→ add a conversation-memory block"] ``` 1. **Ask an easy, direct question** with an answer taken almost verbatim from a document. ✅ if it answers correctly and ideally cites the source. 2. **Ask a paraphrased version** of the same question, using different words. ✅ if it still finds the right passage — this tests that your embeddings are actually capturing meaning, not just matching words. 3. **Ask a question with no answer in your documents at all.** ✅ if the assistant says it doesn't know, ❌ if it confidently makes something up. 4. **Ask a follow-up question** that depends on the previous turn ("and what about X instead?"). ✅ if the assistant correctly understands the reference. If a check fails, use the request lifecycle from the orchestration module to localise the problem: - Wrong or no chunk retrieved → check chunking (topic size/boundaries) or the embedding/vector-store setup, not the language model. - Right chunk retrieved but a wrong answer generated → check your prompt template — it may not be instructing the model clearly enough. - Confident wrong answer to an unanswerable question → your "say you don't know" instruction likely needs to be stronger, or you need a relevance-score cutoff. > 💡 **Tip:** Save this four-question checklist — you'll reuse (and expand) it into a full evaluation sheet in this course's evaluation how-to, and it is exactly the debugging habit professional RAG teams use in production. > 🏢 **Business case:** Teams that skip structured testing like this often discover failures from angry users instead of from their own checks — by which point the cost (in trust, and sometimes in real refunds or complaints) is much higher than the ten minutes this checklist takes.

Mastering Each Sub-part 6 topics

How to chunk documents well
Building on the chunking concept from the architecture module, here is a practical, repeatable process for chunking any real document well: ```mermaid %% title: A repeatable chunking process flowchart TD A["Look at the document's natural structure"] --> B{"Clear sections or Q&A pairs?"} B -- "Yes" --> C["One chunk per section or Q&A pair"] B -- "No" --> D["Fixed size (~200–300 words) + a small overlap"] C --> E["Re-read a sample of the actual chunks"] D --> E E --> F{"Each chunk stands on its own?"} F -- "No" --> A F -- "Yes" --> G["Done"] ``` 1. **Look at the document's natural structure first** — headings, numbered sections, FAQ-style question/answer pairs, table rows. Structure is usually a far better chunk boundary than an arbitrary word count. 2. **Default to "one chunk per section or per Q&A pair"** whenever that structure exists. 3. **For long, unstructured prose** (a narrative document with no clear sections), fall back to a size-based split — a few hundred words per chunk is a reasonable starting point — and add a small overlap (a sentence or two shared between consecutive chunks) so an idea that spans a boundary isn't lost entirely to one side. 4. **Re-read a sample of your actual chunks after processing**, not just the settings you chose. Does each one make sense on its own, without the rest of the document around it? If not, adjust and re-run. > 💡 **Tip:** Different document types in the same knowledge base often deserve different chunking rules — a strict Q&A page and a long narrative policy document are not the same shape, and forcing one setting onto both usually under-serves one of them. > ⚠️ **Watch out:** The most common beginner chunking mistake is picking one fixed size for an entire mixed knowledge base and never revisiting it. If your testing keeps surfacing "the retriever found something *close* to right but missing a key detail," re-check your chunk boundaries before you touch anything else.
How to choose and tune an embedding model
Most no-code tools let you pick from a short list of embedding models. For a beginner project, the decision is simpler than it looks: ```mermaid %% title: Choosing and sanity-checking an embedding model flowchart TD A["Use your LLM provider's default embedding model"] --> B["Confirm its dimension matches your vector index"] B --> C["Sanity check: a question and its paraphrase score as similar"] C --> D["Two unrelated sentences score as different"] D --> E{"Agrees with your intuition?"} E -- "No" --> INV["Investigate before building anything on top"] E -- "Yes" --> OK["Keep it — don't switch without re-ingesting everything"] ``` 1. **Use whichever embedding model your LLM provider offers by default** unless you have a specific reason not to — for learning purposes, the differences between major providers' embedding models rarely matter as much as getting the rest of the pipeline right. 2. **Confirm the dimension number** matches your vector database index (as covered in the infrastructure module) before you ingest anything at scale. 3. **Keep it consistent** — once you've ingested documents with one embedding model, don't switch models without re-ingesting everything from scratch; embeddings from different models are not comparable to each other. **A simple way to sanity-check an embedding model's quality, with no coding required:** many no-code tools include a way to preview or test embedding similarity directly. Take two sentences you know *should* be considered similar (a real question and a paraphrase of it) and two you know *should* be considered different (two unrelated sentences), and check that the tool's similarity score agrees with your intuition. > 💡 **Tip:** This "does the tool agree with my intuition" test is a genuinely powerful debugging habit — if a paraphrase you'd consider obviously similar scores low, that's worth investigating before you build anything more on top of it. > ⚠️ **Watch out:** Embeddings capture *topic and phrasing similarity*, not logical truth — "the refund window is 30 days" and "the refund window is not 30 days" will still score as fairly similar, because they're about the same topic in similar words. Don't expect embeddings alone to catch logical contradictions; that's the generator's job, guided by a good prompt.
How to tune retrieval (top-k, threshold, hybrid search)
Retrieval tuning is where most of the *real* quality improvement in a RAG system comes from, once your basic pipeline is working. Three knobs matter most: ```mermaid %% title: Tune retrieval one change at a time flowchart LR A["Run your test questions at the current settings"] --> B["Record pass / fail"] B --> C["Change exactly ONE setting (top-k OR threshold OR search type)"] C --> D["Re-run the same questions"] D --> E{"Clearly better?"} E -- "Yes" --> F["Keep it"] E -- "No" --> G["Revert it"] F --> C G --> C ``` **1. Top-k (how many chunks to retrieve).** Start around 3–5. Too few and you risk missing the right passage entirely; too many and you dilute the prompt with irrelevant text, which can actually make the model's answer *worse*, not better, and costs more per question. **2. Similarity threshold (how close is "close enough").** Many tools let you discard retrieved chunks below a minimum similarity score. Setting this too low lets irrelevant chunks through; setting it too high can cause the retriever to return nothing at all on a slightly unusual phrasing of a perfectly answerable question. **3. Search type (semantic, keyword, or hybrid).** If your knowledge base contains a lot of exact codes, names, or numbers (product SKUs, policy numbers, employee IDs), pure semantic search will under-perform — switch to hybrid if your tool supports it. **A practical tuning method, no coding required:** 1. Take your test question list from the infrastructure module. 2. Run each question through your flow with your current settings and note pass/fail. 3. Change *one* setting (e.g. raise top-k from 3 to 5). 4. Re-run the same questions and compare. 5. Keep the change only if it clearly improves results — resist changing multiple settings at once, or you won't know which change actually helped. > 💡 **Tip:** This one-change-at-a-time method is simply the scientific method applied to your RAG pipeline, and it's exactly how experienced teams tune production systems — you're using the identical process, just without writing code to automate it. > ⚠️ **Watch out:** It's tempting to keep raising top-k "just in case" — resist this. Beyond a certain point, more retrieved chunks reliably increases cost and latency while quality plateaus or even drops.
How to write a grounded prompt template
Building on the prompt-template basics from the orchestration module, here's how to iteratively improve one for real use: ```mermaid %% title: Iterate the prompt template through three versions flowchart LR V1["v1: four-ingredient template"] --> T1["Test: an unanswerable and a partially-answerable question"] T1 --> V2["v2: one specific change (e.g. add a citation instruction)"] V2 --> T2["Test again"] T2 --> V3["v3: one more change"] V3 --> SAVE["Save all 3 versions + a one-line note on each"] ``` 1. **Start from the four-ingredient template** (role/boundary, clearly-marked context, explicit "say you don't know" instruction, the question). 2. **Add a citation instruction** — e.g. "after your answer, note which part of the context you used, like (Source: [document name])." Test that the model actually follows this consistently; if it doesn't, make the instruction more specific and give one worked example directly in the template. 3. **Add tone and audience guidance if relevant** — e.g. "answer in plain, friendly language suitable for a first-time customer," if that matches your use case. 4. **Test edge cases deliberately:** an unanswerable question, a partially-answerable question (context has *some* but not all of the needed information), and a question where two retrieved chunks slightly disagree. Refine your instructions based on what you actually observe, not what you assumed would happen. > 💡 **Tip:** Keep a version history of your prompt template as a plain text file (v1, v2, v3…) with a one-line note on what you changed and why. Prompt templates are exactly like any other important business document — you want to be able to see what changed and roll back if a "improvement" turns out to make things worse. > 🏢 **Business case:** A team once added a well-meaning instruction — "be as helpful as possible" — to their prompt template, only to find it made the model more willing to guess beyond the provided context, undermining their carefully-tuned "say you don't know" instruction. Removing that one line fixed the regression immediately. Small wording changes in a prompt template can have outsized, sometimes counter-intuitive effects — always test after every change.
How to add citations and "I don't know" handling
Two specific, testable upgrades that separate a classroom demo from something you'd trust in front of real users: ```mermaid %% title: Two testable upgrades flowchart TD subgraph CIT["Citations"] C1["Keep the source document name as chunk metadata"] --> C2["Prompt: cite that name in every answer"] C2 --> C3["Test a question whose answer spans two documents"] end subgraph IDK["Honest 'I don't know'"] I1{"Top chunk's score below threshold?"} -- "Yes" --> I2["Return a fixed fallback, skip the model"] I1 -- "No" --> I3["Answer normally"] I2 --> I4["Test 3+ phrasings of unanswerable questions"] end ``` **Adding citations, step by step:** 1. Make sure your document loader or chunker preserves a reference to the original document's name (most no-code tools do this automatically as metadata attached to each chunk). 2. Update your prompt template to explicitly instruct the model to reference that document name in its answer. 3. Test with a question whose answer spans two different source documents, and confirm both are cited, not just one. **Strengthening "I don't know" handling, step by step:** 1. If your tool exposes a similarity/relevance score for retrieved chunks, add a rule: if the top result's score is below a set threshold, skip calling the language model entirely and return a fixed fallback message ("I couldn't find a confident answer to this in the documents I have"). 2. If your tool doesn't expose a score-based rule, strengthen the prompt instruction itself and test repeatedly with deliberately unanswerable questions until the failure rate is low and stable. 3. Log (even just in a personal notes document) every time the assistant either wrongly claims not to know something it should, or wrongly answers something it shouldn't — both are useful, different signals. > 💡 **Tip:** A citation that's wrong (cites the wrong document) is often a more useful bug signal than a wrong answer with no citation at all — it tells you *exactly* which retrieval step to go back and fix. > ⚠️ **Watch out:** Don't declare victory after one successful test of "I don't know" handling. Models can be inconsistent — always test with several different phrasings of unanswerable questions before trusting the behaviour.
How to evaluate your RAG system without writing code
You don't need to write a single line of code to evaluate a RAG system rigorously — a well-organised spreadsheet is enough for a project at this scale, and is exactly what many real teams start with too. ```mermaid %% title: The manual evaluation loop flowchart LR A["Build a test sheet: Question · Expected · Actual · Correct? · Cited? · Notes"] --> B["Mix in: easy · paraphrased · unanswerable · follow-up"] B --> C["Run every question through your flow"] C --> D["Score honestly → overall accuracy + 'I don't know' accuracy"] D --> E["Change chunking, retrieval or prompt"] E --> C ``` **Step-by-step manual evaluation method:** 1. **Build a test sheet** with columns: Question | Expected answer (in your own words) | Actual answer | Correct? (yes/no/partial) | Cited correctly? (yes/no) | Notes. 2. **Include a deliberate mix**: some easy direct-lookup questions, some paraphrased questions, some genuinely unanswerable questions, and — if relevant — a couple of follow-up questions to test conversation memory. 3. **Run every question through your flow**, filling in the Actual answer column honestly, including when it's wrong. 4. **Score each row** using your own judgement (you know your knowledge base well by now) — resist the temptation to be generous; a rigorous eval is more useful than a flattering one. 5. **Calculate two simple numbers**: your overall accuracy rate (% marked "yes"), and your "I don't know" accuracy specifically on the unanswerable questions. 6. **Re-run this exact sheet** after any meaningful change to chunking, retrieval settings, or your prompt template, and compare — this turns "I think that helped" into "here's the number that proves it." > 💡 **Tip:** This spreadsheet method scales up naturally later if you ever do learn to write code — it's the exact same idea (a labelled test set, run automatically instead of by hand) used by professional teams building production RAG systems, covered under the name "offline evaluation" in more advanced courses. > 🏢 **Business case:** A non-technical product manager built exactly this kind of spreadsheet for a support-bot pilot, re-ran it weekly, and used the resulting accuracy trend line to convince engineering leadership to invest further — no code was written, but the evaluation rigour was taken completely seriously because the numbers were consistent and repeatable. This exact test sheet — expanded to your full knowledge base — is one of the required deliverables for your capstone.

Week by week

Wk 1

Understanding RAG — the big idea

Work through Module 1: why plain LLMs fall short, the open-book exam metaphor, real examples, and the business case.

Wk 2

Architecture & components, part 1

Ingestion vs. query pipelines, the knowledge base, chunking, and embeddings.

Wk 3

Architecture & components, part 2 + orchestration

Vector databases, retrievers, generators/orchestrators, and how a question travels end to end.

Wk 4

Essentials, pros/cons & business fit

The essentials checklist, honest pros and cons, the decision framework, and the use-case matrix. Business case memo due.

Wk 5

Infrastructure setup sprint

Prepare your knowledge base, get an LLM API key, set up a vector database, and choose your no-code orchestration tool.

Wk 6

Build & test your first flow

Wire together your first end-to-end RAG flow and run it through the testing checklist. Infra setup documentation due.

Wk 7

Tune, ground, and evaluate

Tune chunking and retrieval, strengthen your prompt template and citations, and build your evaluation sheet. Retrieval tuning report due.

Wk 8

Capstone sprint & submission

Bring everything together into one polished, business-ready RAG assistant, complete your evaluation report and business case memo, and submit your capstone.

What you'll build

Your first working no-code RAG assistant

A working question-answering flow over your chosen knowledge base, built entirely with no-code tools, that a stranger could use. ```mermaid %% title: How the three deliverables build on each other flowchart LR P["Working no-code RAG assistant"] --> R["Evaluation report (measured accuracy)"] R --> C["Capstone: business-ready assistant + business case memo"] ```

RAG evaluation report

A completed manual test sheet (question / expected / actual / correct? / cited correctly?) plus a short written summary of your accuracy rate and what you changed to improve it.

Capstone: a business-ready RAG assistant

A polished, end-to-end RAG assistant for a real business use case of your choosing, with citations, "I don't know" handling, an evaluation report, and a one-page business case memo.

Why take this course

Every concept is taught with a real business example first, jargon second.
No programming background required — every hands-on step uses free, point-and-click tools.
You choose your own real knowledge base on day one and use it all the way to your capstone.

You will build

A complete, working RAG assistant with citations and honest "I don't know" handling.
A tuned retrieval pipeline you improved using your own before/after test results.
A manual evaluation sheet with a real, calculated accuracy rate.
A one-page business case memo you could hand to a real stakeholder.

What you can do afterwards

A working mental model of every RAG component, transferable to any tool you meet later — no-code or code.
The confidence to scope, pitch and justify a RAG project inside a real business.

How to take it

💳 Delivery & fees

One course, four ways to attend. Fees are per course, in rupees excluding 18% GST (the GST-inclusive amount is shown too) with Canadian-dollar fees alongside.

Your fee includes our GPU lab infrastructure — no cloud credits to buy. See how we compare, with sources →