← All articles

Testing & data

Why your 5-game win rate is lying to you

DeckSequence · Updated · 4 min read

"I'm 4-1 with the new list" is five games. Rates from a handful of games swing around for no reason at all, and changing a list because of one is among the most common testing mistakes there is.

Five coin flips

Flip a fair coin five times and you'll get four or more heads 18.8% of the time, or roughly one run in five. Nobody would call the coin rigged after five flips. Yet a player who goes 4-1 in testing will happily call a 50% matchup favorable, and one who starts 1-4 will drop the deck. It's the same mistake.

How much results swing

Here's the range of results a deck that really wins half its games will post 95% of the time:

Games playedWins, 95% of the timeWin rate it can show
50–50%–100%
206–1430%–70%
5018–3236%–64%
10040–6040%–60%

After five games, a 50% deck can show almost any record. After twenty, it can still be twenty points off in either direction. Even after a hundred games, a true 50% deck can look like a 40% deck or a 60% one.

How many games you need

There isn't a single number. It depends on how small a slice of your games the stat covers, and the smaller the slice, the more games you need. As a rough guide:

  • About 20 games for a deck-level number like overall win rate. That's enough to spot a list that isn't working, but not enough to rank two good lists against each other.
  • About 30 games where the card showed up for a per-card rate, since utilization numbers only count those games.
  • About 50 games against a specific deck before your win rate in that matchup means much.

You'll form impressions long before then, which is fine, as long as you treat them as impressions. DeckSequence shows the game count next to every deck's win rate and labels it a small sample until there are enough games behind it, so you can see how much weight a number can take. Sign up and the count builds up as you log.

Keeping your own numbers

If you track results in your own spreadsheet, a few habits help. Put the game count next to every rate, so a win rate over 12 games never gets mistaken for a settled one. Write "no games" rather than 0% when there's nothing behind a number. And leave out games you can't reconstruct properly instead of guessing at them.

What to do in the meantime

Hold off on decisions until you have the sample they need. Rebuilding a list after five games is almost always too early, and so is writing off a matchup after three losses.

While you wait, lean on the numbers that don't need a sample. Draw odds, mulligan rates and the theory floor all come straight from your decklist, and a deck's Odds page in DeckSequence shows them from the moment you add the list.

Run this math on your own deck.

DeckSequence computes your list's opening-hand and mulligan odds, and deals practice hands from it.

Sign up