Learn which message earns the conversation: openings, value lines and asks, scored separately
Treat each cold email as an opening, a value line and an ask. Tag every send, score each part in comparable cells, and promote only past a preset evidence bar.
Jared Hawkins-Young · Founder, Lost Star TechnologyOctober 8, 20265 min readPart 10 of 14Key takeaways
- A cold email is a set of choices, not a template: an opening, a value line and an ask, each with its own version ID.
- Every send carries its campaign-cell ID plus the three component versions, so each part is scored without being blamed for another part's weakness.
- Components are scored across matched cells first and the whole recipe inside its own cell second, and a change is promoted only after an evidence bar set in advance.
- The bar is higher than most teams expect: telling a 2% positive-reply rate from 3% takes about 3,800 sends per version at 5% significance and 80% power.
Split every cold email into three parts you can test: the opening, the value line and the ask. Tag each send with its campaign cell and the three version IDs, score each part across matched cells, then score the whole combination inside its cell, and promote a change only after it clears an evidence bar you set before the test began. That is how you learn which message earns the conversation, instead of arguing about which email looks best.
Why score email components instead of whole emails?
A whole-email test hides too much. A weak ask can sink a strong opening. A stale signal can sink both, and the copy takes the blame for a problem that lived in the data. If you only compare whole emails, you throw away a good opening because it happened to ship with a bad ask. Component tags let you see that one opening does well wherever it goes, and that one ask drags down everything it touches.
The context row matters as much as the copy. A component is only compared with itself under matched conditions: the same size band, industry, persona and signal type. That is why the cell from part 8 and the signal from part 9 come first. Without them, a component score is an average of unrelated situations.
What does each part of a cold email do?
The opening names the signal and earns the second sentence. The value line connects that signal to one problem you solve, in terms the persona already uses. The ask makes replying easy. Each has its own small library, written and approved by a person, with an ID on every version.
Write the library so the tests stay clean. Each version changes one idea, not five. Keep versions close in length, so length is not the hidden variable. Never reuse an ID: a retired version stays in the record with its result, so nobody rewrites it next quarter and calls it new. And write in the persona's own words. A head of sales talks about ramp, quota and pipeline; a RevOps lead talks about data, routing and tools.
Published data gives you starting priors, not answers. Hunter's State of Email Outreach 2026 reports that decision makers prefer open-ended asks such as "Open to learning more?" over a request to schedule a meeting upfront. Treat that as the first version of the ask to test in your cells, not as a rule.
How do you read component results?
Read components first, then recipes. The table below shows both views for a few cells. Every number in it is illustrative, not a measured result.
| Cell | Component or recipe | Positive reply | vs cell median | Next |
|---|---|---|---|---|
| Logistics · 51 to 200 · head of sales | Opening O-02 “Two new reps in one quarter” | 2.6% | +0.6 pts | Promote |
| Logistics · 51 to 200 · head of sales | Ask C-04 “Are you the right person?” | 2.8% | +0.8 pts | Promote |
| Fintech · 51 to 200 · RevOps lead | Opening O-04 “Built for the next phase” | 1.7% | +0.2 pts | Retest |
| HR tech · 501 to 1,000 · founder | Ask C-01 “Useful to compare options?” | 0.8% | −0.6 pts | Retire |
| Logistics · 51 to 200 · head of sales | Recipe O-02 × V-01 × C-04 | 2.8% | +0.8 pts | Promote |
| Logistics · 51 to 200 · head of sales | Recipe O-02 × V-03 × C-04 | 2.1% | +0.1 pts | Retest |
Two rules keep the reading honest. A recipe that wins is a recommendation for its own cell, never a universal winning email; the same three parts can lose in fintech. And a component that wins in one cell earns a test in a similar cell, not an automatic rollout.
How much evidence is enough to promote a message?
More than most teams expect. With a 2% positive-reply rate as the baseline, telling it apart from 3% at the usual 5% significance level and 80% power takes about 3,800 sends per version, from a standard two-proportion sample-size calculation. Telling 2% from 4% takes about 1,100. A single cell with a thousand eligible contacts cannot settle a small difference on its own, which is exactly why components are scored across matched cells.
The other trap is stopping early. Evan Miller's 2010 essay on A/B testing showed how checking a running test and stopping the moment it looks significant inflates false positives: in his worst-case example, which checks after every observation, a test with no real effect was declared a winner 26.1% of the time instead of 5%.
Decide on a sample size in advance and wait until the experiment is over.
So the bar is set before the test starts, written down, and read once at the end. Anything short of it is labelled "keep learning", not "winner". The outcomes stay simple: promote, retest, or retire.
Two cautions before any test. Clear the delivery path first: a component test on a domain with a placement problem measures the problem, not the copy, which is the whole argument of part 12. And measure positive replies, not opens. Hunter's 2026 data found campaigns without open tracking drew a 7.4% reply rate against 4.4% with it, and privacy features and corporate scanners fire tracking pixels on their own.
Who this is not for. If your cells add up to a few hundred sends a month, you will not reach these thresholds on reply rates. Run fewer, bolder variants, judge them on what the replies say, and keep the tags anyway, so the evidence builds across quarters. The same idea of scoring the artifact rather than the person runs through After the meeting, part 9, which applies it to the sales deck.
Next in the series
Part 11, Time the conversation, applies the same discipline to timing: send window, day of month, follow-up rhythm and season, each learned per cell.
Common questions
- How many cold email variants should I test at once?
- One component at a time inside a cell, with two or three versions of it, and everything else held fixed. More versions split the same sends into smaller groups and push the evidence bar out of reach.
- Should I judge cold emails on open rates or reply rates?
- Positive replies per contacted lead. Open rates are unreliable because privacy features and corporate security scanners fire tracking pixels automatically, and Hunter's 2026 data found campaigns without open tracking drew a 7.4% reply rate against 4.4% with it.
- What is a whole-message recipe?
- The specific combination of one opening, one value line and one ask, scored inside one campaign cell. A recipe that wins in one cell is a recommendation for that cell only, not a universal winning email.
Sources
Written by Jared Hawkins-Young from the system he builds and runs. Diagrams are drawn from the same specs as the software. Published October 8, 2026. Illustrative numbers are labelled as such; every other figure carries its source.