01The owner
Running your own experiments
You own the metric review and you're the call-it-or-not person. The team is watching what you ship, and you'd rather not learn the lessons in production.
A practice ground for product managers
Practice the judgment
behind every A/B test.
An A/B testing simulator for what courses don't teach you. Before a real launch makes you do it live.
The gap nobody prepares you for
Your tool runs the math. Someone still has to make the call.
Picking the right primary. Calling a mixed result. Holding the line on day six when the CEO wants to ship. None of those are math problems.
Most PMs learn them in production, on a real launch, with the team watching. There is a better way.
Why this matters now
The PM bar is rising. Experimentation is the lift.
- 01
Tools handle the math. Not the calls.
Statsig, Datadog, Eppo put PMs in the driver's seat. They tell you the p-value. They don't tell you whether the test was designed right, or whether to ship a mixed result.
- 02
AI handles the busywork. Not the judgment.
Specs, summaries, status updates: those are AI's job now. The hard calls are still yours: when to ship, when to hold, when to revert. That's the work that gets you promoted.
Most PMs build that judgment on a real launch, in public, with the team watching. There is a better way to build it. One where getting it wrong doesn't cost you a quarter.
Who this is for
Three kinds of PMs we built this for.
02The teammate
On a high-experimentation team
Your team ships two or three tests a week and you're not yet the one writing the readout. You'd like to be the PM who can.
03The candidate
Prepping for product interviews
You'll be asked to design a test on a whiteboard and read a mixed result out loud. You'd like to walk in with reps, not theory.
How it works
One scenario. Six phases. End to end.
Every scenario simulates a complete experiment. You walk through the same six phases a real test goes through, and you get scored on the calls you make at each one.
Discovery
You walk into a brief. The team has a change in mind. Stakeholders weigh in. You decide whether to invest investigation budget before designing.
Experiment Design
Hypothesis, metric plan, statistical parameters. You make tradeoffs across MDE, power, and duration. The sample-size calculator updates as you decide.
Monitoring
The experiment plays out day by day. Curveballs appear in real time. You can pause to peek, or hold the line until the planned end date.
Analysis
Final results land. You read the primary, secondaries, guardrails, and data-quality flags before making your call.
Decision
Ship, iterate, or revert. You write a rationale your team could read tomorrow morning. No second chances.
Debrief
The ground truth is revealed. You see what you missed, what you got right, and which judgment habits hold up under pressure.
Featured scenarios
The calls you will recognize.
Each scenario is a complete experiment with stakeholders, data, curveballs, and a real decision at the end. You will probably catch yourself reaching for the wrong call at least once. That is the point.
The Signup CTA
Marketing wants to swap the homepage signup CTA from 'Get Started' to 'Start Free Trial'.
Preview
Control
Get Started
Treatment
Start Free Trial
The Checkout Refresh
The product team consolidated a four-step checkout into a single page to reduce drop-off.
Preview
Control
Step 2 of 4: Payment
Treatment
Address
Payment
Summary
The Onboarding Shortcut
Growth shortened a five-step onboarding flow to two steps to lift activation.
Preview
Control
- 1Welcome
- 2Goals
- 3Preferences
- 4Connect
- 5First action
Treatment
- 1Welcome
- 2First action
The Recommendations Carousel
Marketing replaced the homepage Featured Categories grid with a personalized 'Recommended for You' carousel.
Preview
Control
Featured Categories
Sale
New
Best
Brands
Treatment
Recommended for You
A
B
C
D
The Checkout Button Color
A senior designer wants to change the mobile checkout button from blue to green based on a color-psychology blog post.
Preview
Control
Mobile checkout
Order summary
Payment
Place order
Treatment
Mobile checkout
Order summary
Payment
Place order
More scenarios on the way
What changes
After enough practice, the muscle is yours.
Six PM skills, before and after. Each one is something you'll do on a real launch in the next six months.
PM skill
Before
After
Interpreting a result
Trust the dashboard's verdict
Spot the questions hiding behind the numbers
Defending your primary metric
Bend to whoever pushes back loudest
Walk anyone through your reasoning in sixty seconds
Catching guardrail breaches
Glance, shrug, hope nothing's wrong
Spot the breach the moment it shows
Holding the planned duration
Cave when the CEO asks on day six
Hold the line without sounding like the blocker
Sizing the experiment
Run the calculator, ship the default
Pick MDE and duration that match the decision
Writing the recommendation
Punt the call to next sprint
Write a ship-or-revert rationale anyone can act on
And one tangible thing
A certificate that says you've made the calls.
Finish the track and you walk away with a practitioner certificate. Put it on LinkedIn. Reference it in interviews. It is a record that you have sat through the decisions A/B testing actually rewards, made them under pressure, and seen what happened. Most PMs cannot show that.
Sample
Experiment Lab
Edition · 2026
Practitioner Certificate
This certifies that
Alex Morgan
with Distinction
has earned the Practitioner Certificate in A/B Testing Foundations, demonstrating rigorous experimentation judgment across five end-to-end A/B tests.
Curriculum
A/B Testing Foundations
Scenarios
5 of 5
Issued
May 5, 2026
Who built this
Andres Vourakis
Senior Data Scientist · Founder, Experiment Lab
I'm a Senior Data Scientist with 8+ years in Data and Analytics, with a strong background in Product Analytics, working embedded inside product teams next to the PMs making the calls.
The work has been designing experiments, choosing metrics, making sense of inconclusive results, and translating them into the decisions that actually shipped. The same handful of judgment calls came up, team after team, unrehearsed and under pressure.
I've watched companies buy experimentation tools to help their teams ship faster. The tools handled the math. The judgment calls still landed on the PM, and most weren't ready for it.
Experiment Lab is the practice ground I wished every PM had before their first real launch.
Pricing
One purchase. Lifetime access.
You finish the track in a few weeks. We don't think you should keep paying for that. Buy once, return whenever you need a refresher before a launch or an interview.
Individual
$29$19one-time
Launch price · First 50 buyers
For product managers who want the practice ground for themselves.
Five end-to-end scenarios · self-paced
- A/B Testing Foundations track, all five scenarios
- Certificate of completion
- Lifetime access to the Foundations track
- Personal progress and debrief history
Teams
$59per seat · one-time
For experimentation, growth, and product teams.
- Five-scenario Foundations track for every seat
- Lifetime team dashboard access
- Centralized billing and seat reassignment
- Three-seat minimum · volume pricing as the team grows
Frequently asked