Kyle Poyar tracked 25,000 tests. The hit rate is brutal — and the lessons are specific. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Today's newsletter is proudly supported by Dofollow 🎉
In an era where content creation is easier than ever, content authority is the key to ranking high and maximizing SEO. And despite what the Threadbois will tell you, backlinks are still king. That's why I'm excited to partner with Dofollow as my go-to recommendation for sourcing high-quality backlinks. Going from the bottom of page 1 to the top can be the difference of thousands of dollars in MRR. And with so many shady SEO practitioners out there, I'm glad to have a trustworthy team I can confidently recommend. Make sure to mention "Corey" or "Swipe Files."
|
|
Kyle Poyar published the analysis I've been waiting for: 25,000 website tests across top tech companies, and a clean answer to the question every CRO team avoids — how many of these actually move revenue? The headline number: only 10% of tests tied to revenue beat the control. Nine out of ten of your "data-driven" experiments are essentially shipping the existing version with extra steps. Teams are burning cycles on tests that have a 10% chance of doing anything useful, then defending the process because "we're testing." Some specifics from Kyle's data worth sitting with. What's actually winning right now
- Product "sneak peeks" on checkout/registration pages beat plain backgrounds. The old playbook was minimize-distraction at the point of conversion. The new data says the opposite — showing what the user is about to get, inside the registration moment, reduces drop-off. Friction isn't the enemy. Anonymity is.
- Multiple CTAs in hero sections showed signal as a re-testable assumption. The "one CTA above the fold" gospel may not survive contact with current data. Worth A/B testing again, especially for products with multiple buyer intents.
- Annual pricing displayed as a monthly equivalent is now standard among the top 100 B2B SaaS. Nearly 30% have made the switch. The math is the same — $199/month billed annually vs $2,388/year — but the framing makes the price feel smaller. If your pricing page still leads with annual numbers, you're paying a perception tax.
What's failing — and why teams ignore the dataThe most damning anecdote in the post is GoDaddy. They tested less AI-centric messaging on their homepage. It won. Their response: emphasize AI everywhere on the site anyway. This is the part of CRO that doesn't show up in case studies. You don't lose tests because the methodology is bad. You lose them because the org has already decided what the answer is — and the test is theater. If you're running tests where the executive team has pre-committed to a direction, you're not testing. You're collecting evidence to ignore. What to do this monthKyle's top recommendation: run fewer, higher-impact tests instead of pursuing volume. The CRO teams that win are not the ones running 20 tests a month with 2 winners. They're the ones running 5 tests a month, picking variables that genuinely could move revenue, and acting on the results. Concrete moves:
- Audit the last 10 tests you ran. How many tied to revenue (not engagement)? How many beat the control? If the ratio is worse than 1-in-10, your test selection process is the problem, not your execution.
- Stop running button-color, copy-tweak tests. They almost never hit. Test structural changes — page layout, where social proof lives, hero treatment, registration flow.
- Pre-commit to the action. Before launching, write down what you'll do if it wins, lose, or flat. If "we'll ignore it" is the answer for any outcome, don't run the test.
- Track your team's hit rate quarterly. If you're not at 10%+ on revenue-tied tests, you're worse than Kyle's average. Time to recalibrate.
The interesting thing about Kyle's number isn't that 90% of tests fail. It's that most teams don't measure their hit rate at all, so they don't even know they're under average. —Corey
Swipe Files does not have sponsors. We have partners.We partner with trusted top-tier service providers. In fact, I reached out to them — each one has been carefully selected and vetted so I have 100% confidence in recommending them to you. Doing business with them supports this newsletter. 😊 See how these services can help you grow and mention "Swipe Files" or "Corey" for special treatment:
- Conversion Factory — My SaaS marketing and design agency without the BS of unpredictable costs or contracts.
- ReactSquad — The only developer matchmaking service for SaaS startups focused on vetted engineers who focus on React & Node.js.
- xFusion — Get trained, reliable, and affordable customer support reps ready to go for your business.
- Sales.co — Cold email from idea generation to A/B testing, all you have to do is show up for the sales call.
- Videodeck — Done-for-you video production with an expert team at an affordable price.
- Dofollow — My go-to service for sourcing high-quality backlinks in ethical ways to help you rank higher in Google.
- Embarque — Level up your content and SEO at an affordable price, from keyword research to content production to promotion.
- A Team Overseas — Proactive VAs for founders who need more than just order-takers.
- Latin Remote Workers — Save up to 70% on salaries by hiring exceptional talent from Latin America.
- Testimonial Hero — Done-for-you professional video testimonials and case studies.
|
You're just 5 referrals away from unlocking 50% Off
Next Reward
50% Off
Share with 5 friends and become a Swipe Files member for just $249 (normally $499) or get 50% off your next year if you're already a member.
Unlocked Rewards
50% Off
Free Year
Free for Life
|
|
No comments:
Post a Comment