Testing Bias
Hello MTG Madness!
Today I want to talk more about general Magic mind-set and tournament preparation ! And a tournament begins when testing to find a good deck for a constructed format.
I found that sometimes, even when you’re spending a lot of time with your team testing, you can still miss the deck that you should have played, even when you had all the information about it.
I call it « testing bias » but I should call it the « ugly deck bias » : when a deck doesn’t looks good but actually really is, and you remove it from your testing gauntlet without trying to play it.
On the opposite, sometimes you can feel a deck is very good – you and all your testing group – even when it will actually underperform.
When you’re testing for the Pro Tour and if you’re rigorous, you’ll not have this problem : you’ll spend a lot of time testing and writing down results so you should not miss an important deck, but it can happens sometimes. And it happens a lot more when testing for GPs, because when you have a team, you will usually not spend more than 2 or 3 days testing for it.
1. Ugly deck bias
Let me get back to GP Warsaw, standard, early August.
We had a good testing group, led by Pierre Dagen (since he’s one of the best and most prolific deckbuilders in France) and we were really motivated to rock this standard GP. As I’m personally living in the most lost part of France, far from everything so I was testing on my own on MTGO before I met other French players directly at Warsaw. We spoke on the phone several times before we went in Poland and I told Pierre that I think that UW flash wasn’t good anymore in the metagame we were expecting, and he told me that after a load of testing he found that Esper was beating everyting but Kibler Gruul. Also, Junk was as strong as before and unexpected for this tournament and could be an unexpected challenger.
We found Jund not good nor bad, too average and expected to be picked on. I told Pierre I wanted to test mono-G, just to be sure. But none of us was actually believing that mono-green would be good, despite Raphael Levy texting me to tell me it was actually totally awesome.
It’s probably something more Frenchy, but creature-based decks and aggressive strategies are easily mocked by a lot of players as noobish or mindless decks. It comes from old Magic’s times where creatures had no text and spells had a bunch more. Back in those days, creature decks were almost always one-way decks while control decks were able to outplay opponents and to give more options.
But Creature is by essence the most complicated card type in Magic. Even a vanilla creature offers choices every turn about attacking or blocking. As the amount of creatures in the board increases, the complexity of these attack and blocking steps grows exponentialy because each creature’s decision is related to other creatures. If you ever played limited games with huges boards on each side, you know that sometimes attack steps can be very tricky, especially when you add all the creature’s ability and some tricks to the math. When every creature has at least one or two abilities, requiring choices and you have to think about what your opponenet is gonna play to be one step ahead, it can be the toughest games of Magic you can play.
This « legend » about agressive strategies being bad and for dummies is more harmfull than it seems, even if I don’t believe in it myself I actually found myself saying the same thing when someone talked to me about testing mono-green : « Well, maybe it’s not that bad… but anyway, you’re not gonna test to play MONO-GREEN ? ».
Well, there’s absolutly no sense in it !
I should play the deck I feel has the best winning chances with, and if it happens to be mono-green, well so it is. Just because there’s mostly green dudes in it doesn’t mean that you can play this deck if and only if you came to the tournament without a single test and still drunk from Friday night’s hang out !
2. Too much MD testing bias
The Esper deck I played in Warsaw was indeed the exact opposite, despite being a good choice. Pierre told me it was beating everything but Kibler Gruul, and he was right. Once I landed in Warsaw, I started testing it and I actually was far ahead versus Jund and Patriot. Winning all the Sphinx’s Revelation mirrors plus Jund was a pretty good match up, but the matchup versus Kibler Gruul was pretty ugly and I spent a lot of the time to find a way to beat it post-side. Our tech was to side in a bunch of Vampire Nighthawk and Blood Baron of Vizkopa to fix two problems at once : we found that the matchup versus Gruul was decided by only a few cards. Pre-side, Gruul having or not having a Domri Rade if far more important then who’s on the play. Post-side, there were not only 4 Domri Rade, but also 3 or 4 Burning Earth and those card are auto-lose.
So siding in lifelink creatures was really good because it allows you to gain enough life to play under Burning Earth, and creatures to attack these Domri Rade.
However, it took a lot of room in the sideboard and wasn’t making the matchup positive. Just less bad.
Usually, control is a good strategy because it has around-50% pre-board matchups then sideboard into right fitted solution for the matchup and goes much better in games 2 and 3.
But this time it was actually the opposite situation : Esper was the most control deck in the format but was actually losing 10 to 30% in every matchup after side. Jund was the most impressive : from barely 35% MD, it can reverse it up to 70% by siding out 10 removals and playing discard, planeswalkers and card-advantage instead. But Patriot too improved his matchup by getting rid of blast and removals for more counterspell or win conditions like Assemble the Legion !
UW pike did exactly the same, and even the already bad Gruul matchup can get worse after the inclusion of a four of Burning Earth.
Esper cannot improve it’s matchup that much because it already had the right solutions for it’s opponents, but was taking advantage of the dead removals maindeck.
This is another bias in testing : first you choose a good deck, then you try to find a good sideboard strategy. If you feel like you already had the best deck, you’re not going to change because you don’t find a groudbreaking sideboard strategy.
But post-board games are far more important than pre-board ones : a matchup that is 60% pre-side and 40% post-side is actually 44,8% overall.
I don’t have the right procedure to do it, but you should directly test games with sideboard. The problem is to rightly balance the amount of cards you can side for your tests without still knowing your exact MD list nor the 15 sideboard cards.
In the end, Timothée Simonot and we were playing this Esper deck, we both made day 2, and Tim top 32ed while I hit a disapointing but well disserved top 80 after playing badly.
The deck was truely good but far under our expectations.
For the record, this is our decklist :
ESPER, M14 Standard
