The two-pipeline mental model: ingestion vs. query

Every RAG system, no matter how it's built, is really two separate pipelines that happen to share the same storage:

1. The ingestion pipeline (runs ahead of time, whenever your documents change): take raw documents → break them into chunks → convert each chunk into a numeric representation (an embedding) → store the chunk and its embedding in a searchable index. This happens once per document, not once per question.

2. The query pipeline (runs every time a user asks a question): take the question → convert it into the same kind of numeric representation → search the index for the most similar stored chunks → hand those chunks plus the question to the language model → return its answer.

💡 Tip: If you remember nothing else about RAG architecture, remember this split. Almost every confusing question a beginner has ("why didn't it find my new document," "why is it slow," "why is it expensive") resolves once you ask "is this an ingestion problem or a query problem?"

Think of ingestion as stocking a library and query as a librarian answering a question using that library. You stock the shelves once (and restock whenever new books arrive); every visitor who asks a question just uses whatever is already on the shelves. The rest of this module walks through every component in both pipelines, one at a time.

%% title: The two pipelines share one vector database
flowchart TB
    subgraph ING["INGESTION pipeline — runs when documents change"]
        direction LR
        D["Raw documents"] --> C["Split into chunks"] --> E1["Embed each chunk"]
    end
    subgraph QRY["QUERY pipeline — runs on every question"]
        direction LR
        U["User question"] --> E2["Embed the question"] --> S["Search for nearest chunks"]
        S --> P["Build prompt:<br/>question + top chunks"] --> L["Language model"] --> ANS["Answer"]
    end
    E1 --> VDB[("Vector database")]
    VDB --> S