5.2Verification, critical thinking & fact-checking
The essentials in 30 seconds
- Proportionate to the stakes: re-read (internal draft) / check the critical points (released) / source everything (published, legal).
- Systematic critical points: figures, dates, names, quotes, references — cross-check and contradict.
- Whoever signs checks: “the AI said so” is never a defence.
Running case — Amine and the figure that looked perfect
Amine gets AI to synthesise a progress report with a rate and a regulatory reference that look “perfect”. Before releasing to the client, he applies his protocol: he asks for the sources, checks the figure against the source and gets AI to hunt for contradiction. The rate was approximate, the reference non-existent. Sequel to case 5.1 — Amine protects what goes out as he protected what goes in.
Sequel to case 5.1 — Amine moves from confidentiality (what goes in) to reliability (what goes out).
1. A proportionate process
After confidentiality (what goes in), reliability (what goes out). Checking is not paranoia but a proportionate process: an internal draft is skimmed; a figure shown to the client is checked against the source; a quote or legal reference is systematically controlled. Amine classifies each output at the right level before releasing it.
2. Three levels, one rule
Your three-level protocol: re-read (internal drafts) / check the critical points (released documents) / source everything (publications, legal, binding figures). Techniques: ask for sources, cross-check two models, hunt for contradiction.
The principle: whoever signs checks — “the AI said so” is no defence. Two models agreeing does not make a truth (same biases possible); two models diverging flag shifting ground.
🔎 Concrete example — Checking does not kill the gain
The balance stays positive: Amine's figured case.
Before: write 2 h, re-read 10 min.
With AI: generate 10 min, check 20 min (targeted on critical points).
-> massively positive balance; two models diverging = shifting ground.
Amine times his checking: faster than a full re-read.Outcome: targeted checking beats full re-reading in time and reliability.
Lesson: check figures, quotes and references whatever the stakes.
Section vocabulary
- Proportionate checking
- Three levels by stakes: re-read, check the critical ones, source everything.
Simple example: Amine re-reads his draft but sources everything in his client report.
What it is not: checking everything identically — or checking nothing at all.
Reflex: which level does this document require: 1, 2 or 3?
- Critical points
- Figures, dates, names, quotes, references: systematic checking.
Simple example: Amine highlights the rate and the reference before release.
What it is not: checking only style and layout.
Reflex: did I highlight each binding fact?
- Model cross-checking
- Same factual question to two assistants: divergence alerts.
Simple example: Amine compares two syntheses of the same report.
What it is not: taking two models agreeing as proof.
Reflex: do both models really say the same thing?
- Contradiction hunt
- Asking what contradicts the claim: anti confirmation bias.
Simple example: Amine asks “which elements contradict this rate?”.
What it is not: only asking for confirmations of his thesis.
Reflex: did I look for the counter-argument?
- Whoever signs checks
- Responsibility belongs to whoever releases, not to the model.
Simple example: Amine signs the report: he checked each figure.
What it is not: “the AI said so” as an excuse.
Reflex: am I ready to defend each fact I sign?
In practice — Building your checking protocol
- Take 3 recent AI outputs (an email, a document, a synthesis) and classify them: which required checking level 1, 2 or 3?
- On the level-2 document: highlight all critical points (figures, names, claims) and check them — time yourself: it is faster than people think.
- Practise cross-checking: ask two assistants a sharp factual question from your profession — compare.
- Write your protocol in 5 lines (what to check, when, how): it joins the 5.5 charter.
Ready-to-say script — requiring sources
“Before release, I check: give me the exact sources (title, author, date, link) of each figure and quote, then list the elements that could contradict this synthesis. Without a checkable source, the fact does not go to the client.”
Prompt to reuse as is on any output meant to be released.
Copy-paste template — 5-line protocol
MY CHECKING PROTOCOL (joins the 5.5 charter)
LEVEL 1 (internal draft): skimmed by: _______________
LEVEL 2 (released): critical points highlighted + checked against the source by: _______________
LEVEL 3 (published/legal/binding figures): full sourcing + 2-model cross-check + contradiction: _______________
SIGNATURE: I only sign checked work. Owner: _______________Solo / in a rush
Amine under deadline: checking reduced to the incompressible core — figures, names, references — never removed, even in a rush.
Small business / no researcher
Assistant with web search + clicked sources: tooled checking replaces the missing documentation unit.
SME in Algeria
Amine in Algiers: local facts checked against primary sources (documents, contacts), not on the model's memory alone.
Team that releases a lot
Reading pair: one generates, the other checks the critical points — whoever signs stays responsible.
✓ Best practices
- Proportionate checking to the stakes (draft ≠ publication).
- Systematically check figures, quotes and references.
- Cross-check two models and hunt for contradiction.
✗ Common pitfalls
- Checking everything identically (or checking nothing).
- Defending yourself with “the AI said so” (no such thing).
- Believing two models agreeing = truth (same biases possible).
Key takeaways
- Checking is proportionate to the stakes: draft ≠ client document ≠ publication.
- Figures, quotes, references: systematic checking, whatever the stakes.
- Two models diverging = shifting ground; two models agreeing ≠ truth.
- Signing commits: whoever releases, checks.
48h challenge — apply tomorrow
Classify 3 AI outputs (levels 1/2/3), check the critical points of the level-2 one and write your 5-line protocol.
- I can cite the 3 checking levels.
- I checked 1 level-2 document (sourced critical points).
- My 5-line protocol is written (ready for the charter).
Frequently asked questions
Does checking not kill AI's time gain?
No: writing 2 hours then re-reading 10 minutes becomes generating 10 minutes then checking 20 — the balance stays massively positive. And targeted checking (critical points) is far faster than a full re-read.
Are there automatic fact-checking tools?
Web-search assistants partly check themselves by citing sources, and specialised tools are emerging. But the final call — is the source reliable? is the context respected? — stays human. No tool replaces the judgement of whoever signs.
What is the responsibility principle of checking?