Off Topic: Proper Play Testing

Standard has changed with the loss of the derogatory “F” word. Everybody and their dog are hellbent on cracking the format. Jund decks are the new tech to beat. Personally, I don’t mind the new white elephant. It is good, but not unstoppable as it was combating F explicative. I suppose I could play Jund. Most of the cards I need are in my collection. A couple of trades here, some purchased cards there, and ta da. Unfortunately, I’m not smart enough to just play the best deck and resist the unwitting urge of going rogue. This means I have been play-testing a lot of decks.

Phase 0: The Great Idea

The initial stage of all great thinking. It is those nights in bed dreaming of Baneslayer Angel and Garruk Wildspeakers dancing in a person’s head. A sudden connection is made as we spring awake from our slumber to put some great deck idea together. I betcha that is exactly what happened to Brian Kibler when he put Grove of the Burnwillows and Punishing Fire together. Ideas have to come from somewhere. Mine spawn from the Dark Confidant I tuck under my pillow. Without an idea, there can be no progress.

Phase I: Fishing for Gold

For those new players out there, goldfishing entails playing games of solitaire. Many admonish the practice. True, it is not a great way to play test a deck. The only good thing about goldfishing is it allows a new pilot to get a feel for the deck without the presence of resistance. The trouble begins when a player falls into the false assumption the deck is good by only goldfishing. It is very easy to try out a deck and find it can kill the opponent by turn four. The deck suddenly looks incredible and unstoppable. At game time, the poor plight of the pilot is realized when that 13/13 Plated Geopede slams into a chump blocker or gets whacked by spot removal. Sure, they could have been dead. If they hadn’t blocked with any of three creatures they had in play, they could have been dead. Let’s not forget about the Doom Blade, Path to Exile, Terminate, or Unsummon in their grip.

I like to think of play testing in a similar fashion to the stages drugs go through to get the thumbs up from the FDA (Food and Drug Administration). Goldfishing reminds me a lot of phase I of clinical trials were a drug is initially tested in vitro. The word in vitro in essence means the testing of “A” in a controlled, isolated environment or outside of the natural environment. Before a drug moves on to phase II, a drug must be shown to have some kind of efficacy in an animal model or cell culture. It is also nice if the drug isn’t having lethal effects on all the test animals or cell cultures. Once a drug shows some kind of promise without the nasty side effect of death, the drug can be approved to move to Phase II of clinical trials.

Phase II: Small Deck Trials

The second stage of trials for drugs involves testing in a small population of humans. There is no sense in killing large quantities of people when the toxic effects of the new drug can be demonstrated in small segments. Anyway, the majority of drugs fail in phase II trials. The switch from in vitro to in vivo doesn’t always correlate well. As an example, a lot of animal models use rats. The tiny rodents have a large resistance to toxins. In Phase I trials, the animals are given huge amounts of drug B to kill a cancer. The drug destroys the cancer. At the same body mass to drug ratio needed to treat humans, the drug has been found to be extremely lethal. Over the years, I have met a couple of investors on and off again who are hot on persuading me to buy stock in company X who is going to phase II trials. Idiots.

When testing new decks initially in any format, I don’t believe I need to run the whole gauntlet. With the deck I am trying brew, there is no need in testing further if I can’t beat Jund. Once I can prove I can beat Jund, I will move onto testing the deck further. So, I have been tinkering with a deck I like to call Greenimator. I threw together some cards and goldfished it. It had potential so I began testing it against Jund.

GreenimatorMagic OnlineOCTGN2ApprenticeBuy These Cards
4 Ponder
4 Khalni Heart Expedition
4 Trapmaker's Snare
4 Merfolk Looter
4 Harrow
3 Rhox War Monk
3 Oracle Of Mul Daya
4 Summoning Trap
2 Sphinx of Jwar Isle
4 Empyrial Archangel
4 Progenitus
4 Misty Rainforest
3 Seaside Citadel
2 Plains
4 Island
5 Forest
4 Terramorphic Expanse

Testing, not so good. Jund walked over me like a train over a penny. Slamming down a Empyrial Angel felt good until a Putrid Leech attacked along side a Bloodbraid Elf who flung a Lightning Bolt at me. Progenitus sat there sadly as it became a dumb wall trying to thwart an alpha strike to only have a bunch of Broodmate Dragons fly overhead. All my other creature cards died to spot removal quickly and Sphinx of Jwar Isle just wasn’t big enough to change the tide of the war. Yeah…I didn’t bother testing against the other archetypes. The deck got tweaked and I came up with the following.

Greenimator 2.0Magic OnlineOCTGN2ApprenticeBuy These Cards
4 Rampant Growth
4 Khalni Heart Expedition
3 Trapmaker's Snare
4 Harrow
4 Bloodbraid Elf
4 Captured Sunlight
4 Summoning Trap
3 Bogardan Hellkite
4 Sphinx of the Steel Wind
3 Empyrial Archangel
4 Misty Rainforest
2 Scalding Tarn
2 Plains
2 Island
2 Swamp
2 Mountain
10 Forest

Greenimator 2.0 tested well against Jund. The die roll is extremely important. It can win on the draw, but going first results in a lot of victories. I especially love slapping down a Sphinx of the Steel Wind as it is a big middle-finger to Jund. Bogardan Hellkite does a great job in cleaning up the field and Empyrial Archangel protects against burn and buys a lot of time. Eight damage is a lot to take down the Angel. More importantly, the damage isn’t being dealt to me. Fetch lands and Rampant Growth guarantee I crack the Kalni Heart Expedition. Harrow does the job equally well. Bloodbraid Elf and Captured Sunlight hit additional mana acceleration cards while buying some time. Quick note: Karrthus, Tyrant of Jund can be a cute sideboard card. Play it, steal their Broodmate Dragons and swing for 15.

Possible scenarios:

Turn two: Rampant Growth
Turn three: Bloodbraid Elf into Harrow fetching island and forest then playing Trapmaker’s Snare
Turn four: Summoning Trap for fattie

Turn two: Kalni Heart Expedition
Turn three: Harrow fetch two land, crack the expedition, fetch two more land, play Trapmaker’s Snare to grab Summoning Trap
Turn four: Summoning Trap

Turn two: Kalni Heart Expedition
Turn three: Play and crack a fetch land, play Rampant Growth, crack the expedition.
Turn Four: Play the trap

Turn two: Trapmaker’s Snare
Turn three: Harrow then play Rampant Growth
Turn four: drop a land and play the trap

Phase III: Running the Numbers

Once a drug has shown promise in Phase II trials, it can be fast tracked into testing in larger populations. The significance doesn’t always have to be large. A cancer drug that eradicated the tumors in 10 percent of all patients might be moved forward. The ten seems like a small number, but 10 percent of one million cancer patients saved by the drug can be significant to a huge segment. Still, a large number of drugs still drop out of phase III trials. The testing on a larger population starts producing more data that often reveals the ineffectiveness of the drug. Statistics may show the phase II group produce a bunch of outliers not typical of treatment. Once in a larger population, the drug may start showing more effects of toxicity across a broader span of age groups or ethnicities. The drug may show it only works for breast cancer patients and not any other kind of patients. Even further, the drug may only work on a certain type of breast cancer with already better treatment options.

So, I can beat Jund. I took Greenimator and started testing against other decks. Vampires can be tricky with Gatekeeper of Malakir. It can be troubling unless I let Bloodbraid play sacrificial lamb since I rarely have another creature in play. The nice part is Summoning Trap is an instant and I can play it at the end of their turn. Duress and other discard isn’t too big of an issue. Boros-Bushwhacker is probably the most difficult. Goblin Guide turns into a liability for them though. Whose on the play becomes vital. The deck isn’t unbeatable, but I better darn well mulligan a slow hand. The Red Deck Wins match up can actually be a better scenario. Empyrial Archangel can absorb a lot of damage. Sphinx of the Steel Wind can quickly take me out of range with its first strike and lifelink combo. Boros-Bushwhacker is more of a threat due to Path to Exile.

Phase IV: Real World Testing

My final thoughts about the deck is that it can be very consistent and a powerhouse. A lot of it, I believe, comes down to the die roll and the sideboard. Those extra fifteen card slots will be taken up by other fatties more suitable of taken down certain archetypes. It is hard to believe I can make the deck any faster. Llanowar Elf or Noble Hiearch crossed my mind, but there is just so much darn spot removal running around that they just meet an untimely trip to the graveyard. None of my thoughts matter until I actually take the deck out into the field and have it battle tested. I would say the deck is almost Tier One. It’s just not quite there, but might be a good deck choice in the right environment. It will do well, but not consistent enough win tournaments except against Jund. The future of the deck really depends on the next few sets to see what kind of new behemoths we will have at the deck’s disposal. I think I will still take it out for a spin for fun. Hopefully, Jund won’t mutate and develop a resistance with Thought Hemorrhage.

Even after a drug is released onto the market, the FDA might mandate the use of the drugs are monitored over a period of time. It is to safeguard against any unforeseen consequences. Bacteria could quickly become resistant to the new antibiotic. Heart medications might cause toxicity to kidney function over an extensive amount of time. The drug might not even extend the lifespan of the patients who are taking it. Even after making it to Phase IV, the FDA over the last few years has yanked a couple drugs off the market. Majority of these are because the side effects were found to be as lethal as a Skullclamp.

Scientific Art of Play Testing

Over the years, I have learned a couple of things when testing decks. One, use proxies for all initial testing. The reason being is I destroyed numerous decks during testing. I stole Day of Judgement from my Naya deck, Ponder from my merfolk, and so on. After testing, I don’t have a million cards lying around that I have to find their original home. Proxies can also be easily changed. Originally, I would find some cheap card with a similar same casting cost and write on it. This still involved switching cards when modifying the build. For the sake of efficiency and speed, I just use those extra tip cards inserted in boosters and a permanent marker. I can easily cross out Progenitus and write Empyrial Archangel. Done. No running around trying to find cards. Once the cards get heavily marked up, I just slap on some kind of a label like a white address label to start anew.

Next, take actual data. Most isn’t the same as 70 percent. Keep track of damage. I track my play testing on a excel sheet on my laptop. It is easier and less messy than paper. I can keep track of the matches better. Besides all of this, test the actual deck on the draw and on the play. It should seem rudimentary, but it can be a big failure in testing. The results can be skewed if the on the play or draw isn’t kept track of sincerely. Results look a lot worse when a match up looked 50/50 only because the majority of testing was done on the play. Personally, I begin all my testing on the draw. To me, it is a better sign of the decks potential if it tests well on the draw. Good play testing data keeps me from going to the FNM to have my opponent wondering what the heck I am playing before I am taken down below 20. My opponent: “What does your deck do?” Me: “I play Summoning Trap for a big creature." My opponent: “Cute trick.” Translation: noob.

On a quick final note, good play testing saves money. It makes no sense in shelling out 40 dollars for a Dark Depths when the deck is sub par. Nothing is worse than spending all that money to find a strategy ineffective. As show of good faith, I thought Affinity was the deck to beat for the last Legacy Grand Prix. I spent money to finish off the deck. I only then began practicing to find the deck was not the deck to beat. All that money for nothing.


Posts Quoted:
Clear All Quotes