4.4RAG: making AI answer from your documents
The essentials in 30 seconds
- RAG = retrieve THEN generate: you search the relevant passages in your docs and supply them to the model with the question.
- Grounded answers: fewer inventions, possible citations, easy updates — you reindex, without retraining.
- Lina's reflex: demand sources and an “I don't know” when the info is not in the documents.
Running case — Lina plugs AI into the procedures
Lina's support agent hits a limit: the AI answers “from memory” and sometimes invents procedures. Lina gathers the internal FAQs and procedures in Oran, has them split into chunks, converted and stored for search, then connects the flow: question → retrieved passages → answer from these passages, sources cited. When a document changes, she reindexes. Her chatbot finally knows the company — without inventing.
Sequel to cases 4.1-4.3 — Lina grounds her AI in in-house documents.
1. The RAG principle
A language model does not know your internal documents. RAG (Retrieval-Augmented Generation) solves this simply: instead of letting AI answer from memory, you first search the relevant passages in your documents, then supply them to the model with the question. AI then answers from your content — which is what enables an internal chatbot that knows your company, like Lina's.
2. The mechanism, without jargon
In practice: you split documents into suitably sized chunks, turn them into numerical representations (called embeddings, lists of numbers capturing meaning) stored in a database designed for meaning-based search (vector database). With each question, you retrieve the chunks closest in meaning to the question and inject them into the prompt. No-code tools (dedicated nodes in n8n) build this circuit without coding.
Benefits: answers grounded in your sources (fewer inventions), possible citations, easy updates (you reindex modified documents). One of the most useful AI applications in business — provided you keep a critical mind (see pitfalls).
3. Why not just paste the docs?
Because it does not scale: you don't paste hundreds of pages into every prompt (size limit, cost, drowned information). RAG supplies only the few passages relevant to the question asked. Pasting docs works for a short one-off text; RAG is the real solution for a knowledge base — and Lina verifies this on her procedures.
🔎 Concrete example — 🔎 Concrete example — A documentary chatbot
Lina's circuit for answering on in-house procedures:
Indexing: FAQ/procedures → chunks → meaning-based search
Question → 3-4 passages retrieved
→ injected into the prompt → answer (with citations)
→ reindex on every document update.Reading: the answer reflects the documents, not the model's memory.
Lesson: ask it to cite sources and say “I don't know”.
Section vocabulary
- RAG
- Method that makes AI answer from your documents: search the relevant passages, then supply them to the model to answer on.
Simple example: Lina's chatbot answers on internal procedures instead of inventing.
What it is not: retraining the model on your data.
Reflex: always ask: which passages does this answer build on?
- Embedding
- Numerical representation of a text (list of numbers) used to measure meaning proximity between chunks.
Simple example: Two procedure paragraphs close in meaning have close representations.
What it is not: a readable summary of the document.
Reflex: retain the idea (comparing meaning), not the technique.
- Vector database
- Database designed to store these representations and quickly retrieve the passages closest to a question.
Simple example: Lina stores her split procedures there for meaning-based search.
What it is not: a plain folder of PDF files.
Reflex: check who may query this base (access rights).
- Chunking
- Splitting documents into suitably sized chunks before indexing, for relevant search.
Simple example: Lina splits by coherent procedure, not into arbitrary pages.
What it is not: sending the whole document in each prompt.
Reflex: split by unit of meaning, test, adjust.
- Hallucination
- Invented answer by a model; RAG reduces it by grounding the answer in supplied sources.
Simple example: Before RAG, Lina's AI invented procedure steps.
What it is not: a typo or a software bug.
Reflex: demand sources + “I don't know” and control the critical.
In practice — In practice — Building a mini documentary RAG
- Gather a document base (FAQs, procedures) and split it into coherent chunks, like Lina.
- Prepare meaning-based search (representations + database designed for it, via dedicated n8n nodes).
- Build the flow: question → search for close passages → injection into the prompt → model answer, sources demanded.
- Test on real questions, check that answers reflect your documents, and reindex when docs change.
Ready-to-say script — framing RAG with IT (30 s)
«This is Lina: I want to index our FAQs and procedures for an internal chatbot that cites its sources. Before loading the real documents, I have the scope validated with you (which docs, which access), we test on copies, and we define who reindexes on each update.»
To say before indexing sensitive documents; trace the authorised scope.
Copy-paste template — RAG sheet (fictitious example)
RAG SHEET — fictitious example to adapt
ALLOWED DOCUMENTS: _______________ (FAQs, procedures v. ___)
EXCLUDED (sensitive / confidential): _______________
CHUNKING: by unit of meaning, tested on: _______________
FLOW: question → 3-4 passages → prompt → answer + sources
MODEL INSTRUCTION: “cite your sources; say I don't know if absent”
REINDEXING: by _______________ on each update
DSI/RSSI APPROVAL ON: _______________Small base (10-20 pages)
Ideal for starting: simple chunking, fast tests, visible value within days.
Documents with restricted access
Like Lina: only index the authorised scope and align answer rights with document rights.
Frequently changing docs
Name a reindexing owner: a RAG on outdated docs answers correctly… about the past.
Critical answers (payroll, safety)
Keep systematic human control: RAG reduces inventions, it does not remove them.
✓ Best practices
- Split into coherent chunks before indexing.
- Demand sources and “I don't know”.
- Reindex on every update.
✗ Common pitfalls
- Pasting hundreds of pages into the prompt.
- Believing hallucinations fully eliminated.
- Forgetting human control over the critical.
Key takeaways
- RAG makes AI answer from YOUR documents (not only from its memory).
- Mechanism: split → representations → passage search → supply to the model.
- Benefits: grounded answers, fewer inventions, citations, easy updates.
- Buildable in no-code (n8n and dedicated nodes): one of the most useful AI applications.
48h challenge — apply tomorrow
Gather 10 pages of docs, non-sensitive, split them by unit of meaning and test 5 real questions: do the answers cite your passages?
- I can explain RAG in one sentence (retrieve then generate).
- I know why you don't paste everything into the prompt.
- I have tested 5 questions on 10 split pages.
Frequently asked questions
Why not just paste my documents into the prompt?
It doesn't scale: size limit, cost on each question, drowned information. RAG supplies only the relevant passages: more economical, more precise. As a bonus, you update the base without retraining — you reindex. Pasting works for a short text; RAG is the solution for a knowledge base.
Does RAG totally prevent hallucinations?
It strongly reduces them without eliminating them: AI can misinterpret a passage or answer when it should say “I don't know”. Demand sources, admission of missing info, and keep human control over the critical.
What is RAG for?