A/B testing cold emails only produces a trustworthy answer when the test has enough sends behind it. At a 3 percent baseline reply rate, proving that a variant lifts replies to 4 percent takes roughly 5,301 sends per variant at 95 percent confidence and 80 percent power. Most outbound tests run a few hundred. Those tests are not close calls, they are coin flips.