4.5A/B testing your copy
The essentials in 30 seconds
- The audience decides: A/B testing replaces opinion (“I prefer”) with data — measured conversion decides.
- One variable, one conclusion: change only one element between A and B, with enough data (statistical significance).
- Endless loop: test, keep the winner, retest — copywriting becomes continuous improvement.
Running case — Mehdi and his two subjects
Mehdi debates alone facing his screen: is his “Your guide is ready” subject better than a catchier one? Instead of deciding by taste, he sends each version to half his list and compares opens on sufficient volume. Last episode: he learns to let the audience decide.
Last episode of cases 4.1-4.4 — Mehdi moves from opinion to data.
1. Measure rather than guess
A/B testing compares two versions of copy with real audiences to see which converts best. It replaces opinion debates (“I prefer this headline”) with data: the audience decides, not the ego.
It is the move from intuitive copywriting to measured copywriting. You test what impacts most: email subjects, headlines, hooks, CTAs, prices, visuals. Small changes can produce big conversion gaps.
2. One variable at a time
An absolute methodological rule: change only one variable between A and B. If you change the headline and the button, you will not know what to credit the difference to. One variable, one conclusion.
Another safeguard: statistical significance. A result is only reliable with enough data. A gap over 20 visitors proves nothing; sufficient volume is needed to conclude. Without it, you mistake noise for signal.
3. Continuous improvement
The central metric is the conversion rate: the audience share completing the targeted action. It is what you compare between versions, not impressions or “likes”.
The real gain is a mindset: continuous improvement. You test, keep the winner, test again, learn from your audience continuously. Copywriting becomes an iterative process, never frozen — each test teaches something about the audience.
🔎 Concrete example — Testing two email subjects
You want to know which subject gets opened most.
A: “Your guide is ready”
B: “[First name], the #1 CV mistake (fix it tonight)”
→ Send A to 50%, B to 50%, compare the open rate on sufficient volume.Result: one variable (the subject) changes; the data says which wins.
Next: keep the winner, then test something else (the CTA, e.g.).
Section vocabulary
- A/B testing
- Comparing two versions of copy with real audiences to see which converts best.
Simple example: Mehdi's two subjects sent to 50/50 of his list.
What it is not: picking by personal taste without ever measuring.
Reflex: facing disagreement, ask “shall we test?” instead of debating.
- Single variable
- The test rule: change only one element between A and B to know what to credit the difference to.
Simple example: Mehdi changes only the subject, not the content or CTA.
What it is not: changing headline and button together then concluding on the headline.
Reflex: one variable, one conclusion — the rest identical.
- Statistical significance
- A result's reliability, reached with enough data; a gap over too few cases proves nothing.
Simple example: Mehdi waits for sufficient volume before declaring a winner.
What it is not: concluding on 20 visitors that B beats A.
Reflex: little data = qualitative hint, not verdict.
- Conversion rate
- The audience share completing the targeted action; the main metric to compare between versions.
Simple example: Mehdi compares open rates, not “likes”.
What it is not: comparing impressions instead of the targeted action.
Reflex: always compare on the targeted action, nothing else.
- Continuous improvement
- An iterative approach: test, keep the winner, test again, learn from your audience continuously.
Simple example: after the subject, Mehdi tests the CTA, then the hook.
What it is not: freezing your copy once and for all after a single test.
Reflex: each test teaches something about the audience: note it.
In practice — Set up a first A/B test
- Pick the high-impact element to test (email subject, headline or CTA).
- Create two versions differing only by THAT single variable.
- Send them at equal shares and wait for sufficient volume before concluding.
- Keep the winning version, note the lesson about the audience, then launch a new test.
Ready-to-say script — launching your test (30 s)
“I am testing the subject: version A vs version B, the only changing variable. I send at equal shares, I wait for sufficient volume — no verdict on twenty sends — then I compare rates and keep the winner. Next, I test the CTA.”
To say when framing any test before sending: one variable, volume, then verdict.
Copy-paste template — A/B test sheet
TESTED ELEMENT (subject, headline or CTA): _______________
VERSION A: _______________
VERSION B (only changed variable): _______________
SPLIT (50/50): _______________
COMPARED METRIC (targeted rate): _______________
SUFFICIENT VOLUME BEFORE VERDICT? (yes / no): _______________
WINNER + LESSON ABOUT THE AUDIENCE: _______________
NEXT PLANNED TEST: _______________Small business / tight budget
With no testing tool: alternate A then B at comparable audience and compare raw rates as hints, without over-reading.
SME in Algeria
Small local lists: the strict statistical verdict is out of reach — use direct feedback and received questions as lessons.
E-commerce
Test displayed prices, CTAs and offer visuals first: small changes, potentially big conversion gaps.
Newsletter
Test subjects continuously (strong open lever), then once the winner is known, test the body's hook.
✓ Best practices
- Test high-impact elements first: subjects, headlines, hooks, CTAs.
- Change only one variable at a time: otherwise the result is uninterpretable.
- Wait for sufficient volume: a gap over few cases is noise, not signal.
✗ Common pitfalls
- Changing several elements at once: impossible to know what played.
- Concluding on too little data: mistaking chance for a result.
- Trusting your opinion (“I prefer”) rather than what the audience does.
Key takeaways
- A/B testing replaces opinion with data: the audience decides.
- One variable at a time: one variable, one conclusion.
- Statistical significance: enough data for the result to be reliable.
- Continuous improvement: test, keep the winner, retest, learn.
48h challenge — apply tomorrow
Launch 1 real test with the sheet above: 2 subjects (or 2 headlines) differing only by 1 variable, sent at equal shares. Note the lesson about your audience, whoever wins.
- I can quote the golden rule (1 variable) and the safeguard (volume).
- I know what gets compared (conversion rate on the targeted action).
- I have launched (or planned) 1 real test with the sheet.
Frequently asked questions
What should I test first?
High-leverage, easy-to-change elements: the email subject and headline (they decide whether you get read), then the CTA and hook. A gain on these ripples through everything else.
Is A/B testing useful without a big audience?
Strict statistical testing needs volume. With a small audience, you mostly learn qualitatively (feedback, questions, raw rates as hints). The continuous-improvement mindset stays precious, even without perfect numbers.
Mehdi changes the headline AND the button at once, concludes on 20 visitors and keeps his favorite version. How many mistakes?