This thread is for the discussion of my latest article, Flogging the Data: Sizing up the Legacy Metagame. We would be grateful if you would let us know what you think, but please keep your comments on topic.
Private Mod Note
():
Rollback Post to RevisionRollBack
Nihil sub sole novum est.
Author of "Flogging the Data"' econometric article series.
If I understand the article correctly, your findings were:
1) When looking at the Legacy data set as a whole, no one deck from amongst the top 15 stood out as having a statiscally significant advantage. Furthermore, the data did not strongly support the idea that any of the decks really performed significantly better than any of the others.
2) However, when looking at individual matchups, some decks had significantly better matchup percentages against specific other decks, and so should presumably offer a signficant advantage in meta-games that feature those decks.
Point 1) seems counter-intuitive at first glance (surely some decks must be better than others?). However, rather than pointing to an error in your methodology, I suspect this is evidence that the tournament scene is self-correcting and cyclical. That is to say, if Merfolk proves to be powerful and puts up good showings at one tournament, people will play disproportionately more Zoo (which is strongly favored over Merfolk) at the next tournament. How many people switch to Zoo will be largely determined by how dominant Merfolk was. This self-correction will tend to obliterate evidence that Merfolk overperforms.
This process is also cyclical. If many players switch to Zoo and are successful, then other players will play decks that are strong against Zoo. Or, even worse, they may tune their decks and sideboards to make them inherently stronger against Zoo, without altering their general archetype, causing the matchup percentage to vary from event to event. Once again, this will tend to force the overall deck performance towards equality...and this later situation will not be accurately detected by your model.
All of that said, I think this doesn't really say anything more significant than that the Legacy metagame is currently healthy. In order to overcome the equalizing factors mentioned above, a deck has to be so powerful that it maintains a winning matchup percentage despite the presence of dedicated hate. If that happens, then a statiscally significant deviation should show up in your data, and it would be evidence of imbalance. However, since your analysis did not turn up such a signal, I think one could reasonably conclude that this kind of imbalance doesn't exist in the format.
Please do let me know if I've misunderstood something.
Private Mod Note
():
Rollback Post to RevisionRollBack
If God spoke to you, and commanded you to kill your own children, would you do it?
If your answer is "No," then your morality does not come from God's commandments.
If your answer is "Yes," then please, please reconsider.
When looking at the Legacy data set as a whole, no one deck from amongst the top 15 stood out as having a statiscally significant advantage.
Yes, basically no deck has an absolute advantage, but the community has still not really demonstrated that archetype is a useful concept.
Quote from Goryus »
when looking at individual matchups, some decks had significantly better matchup percentages against specific other decks, and so should presumably offer a signficant advantage in meta-games that feature those decks.
Yes. The error intervals are still are fairly large (+/-30%) for some match-ups, and others the sample is far too small to make any good inferences, but match-ups rather than deck win %s are what seem to matter.
Quote from Goryus »
This process is also cyclical.
Definitely. I'll get into this later in a later article, but since you already mention it, but I tried simulating tournaments where players make optimal choices. The image below shows the results.
The results are highly cyclical, but given an "optimal" metagame where all players take into account all deck choices, some decks like Dredge end up with sub-50% win percentages. Affinity doesn't even break 46% win percentage in any tournament.
Quote from Goryus »
without altering their general archetype, causing the matchup percentage to vary from event to event. Once again, this will tend to force the overall deck performance towards equality...and this later situation will not be accurately detected by your model.
This is one of the reasons we need a better way of classifying archetypes--to rule this out as the source of change.
Quote from Goryus »
I think this doesn't really say anything more significant than that the Legacy metagame is currently healthy.
Essentially, yes. It's more a methodological contribution. Makes me wonder why they banned Mental Misstep because all the data is pre-ban.
For the moment, let's assume that a stable cycle like this is what a healthy meta-game looks like in the long term.
So, can you try modeling theoretical meta-games to see what an unhealthy one would look like? Does it gravitate towards all players playing the best deck at some point? Is there a stable amount of hate-decks, or is that also cyclical?
Also, my last comment wasn't intended to minimize the importance of your contribution. What you're doing here is quite useful. I was just trying to intrepret the data you were providing.
EDH/Commander is a social format, right? So why don't people use their social skills to discuss what they like and don't like, instead of adopting a list with 60+ banned cards?
I really like the article as a whole but seem to have some sort of a gut reaction against the conclusion that you are trying to suggest - that we should seek out a better way of classifying decks than the current method (classifying by archetype). The alternatives you suggest as a basis for deck classification are:
Color composition of each deck
% of types present in each deck
Colors included in each deck
The key cards of the deck
Play style of the deck: aggro, control, combo
However, all of these seem to be far less descriptive than archetype.
[g]ut reaction against [...] that we should seek out a better way of classifying decks than the current method (classifying by archetype).
I would argue that there is no method today, e.g. Walls gives no info about the colors, in contrast to BWStoneblade. Its a jumble of card names, pet names (Team America?!), colors, etc. Underscoring the fact that there is no method.
Quote from beast89 »
However, all of these seem to be far less descriptive than archetype.
Not necessarily; if decks were name something like:
There would be a much greater uniformity and names would be more informative.
But the goal isn't description, the goal is statistical significance. If archetypes don't have much bearing on outcomes, archetypes aren't a useful way of thinking about names. It may be a convenient short-hand for a deck-list, but not a useful tool of analysis.
Private Mod Note
():
Rollback Post to RevisionRollBack
Nihil sub sole novum est.
Author of "Flogging the Data"' econometric article series.
Glad to hear someone's interested. It's definitely written, but the editorial team said its unintelligible to all but economists, so they basically vetoed publishing it in its current form. I write for myself more than an audience, and I've been too occupied to essentially rewrite a 19-page article...
What do you think about shocklands being reprinted in Return to Ravnica?
This to my knowledge has not been confirmed yet.
Will it drive their prices up or down?
Probably up. It would mean they become standard legal, this drives average card prices up by 35% according to my estimates.
The reprint will drive prices down by 5%. So that is a net of about 30%.
Private Mod Note
():
Rollback Post to RevisionRollBack
Nihil sub sole novum est.
Author of "Flogging the Data"' econometric article series.
To post a comment, please login or register a new account.
Author of "Flogging the Data"' econometric article series.
1) When looking at the Legacy data set as a whole, no one deck from amongst the top 15 stood out as having a statiscally significant advantage. Furthermore, the data did not strongly support the idea that any of the decks really performed significantly better than any of the others.
2) However, when looking at individual matchups, some decks had significantly better matchup percentages against specific other decks, and so should presumably offer a signficant advantage in meta-games that feature those decks.
Point 1) seems counter-intuitive at first glance (surely some decks must be better than others?). However, rather than pointing to an error in your methodology, I suspect this is evidence that the tournament scene is self-correcting and cyclical. That is to say, if Merfolk proves to be powerful and puts up good showings at one tournament, people will play disproportionately more Zoo (which is strongly favored over Merfolk) at the next tournament. How many people switch to Zoo will be largely determined by how dominant Merfolk was. This self-correction will tend to obliterate evidence that Merfolk overperforms.
This process is also cyclical. If many players switch to Zoo and are successful, then other players will play decks that are strong against Zoo. Or, even worse, they may tune their decks and sideboards to make them inherently stronger against Zoo, without altering their general archetype, causing the matchup percentage to vary from event to event. Once again, this will tend to force the overall deck performance towards equality...and this later situation will not be accurately detected by your model.
All of that said, I think this doesn't really say anything more significant than that the Legacy metagame is currently healthy. In order to overcome the equalizing factors mentioned above, a deck has to be so powerful that it maintains a winning matchup percentage despite the presence of dedicated hate. If that happens, then a statiscally significant deviation should show up in your data, and it would be evidence of imbalance. However, since your analysis did not turn up such a signal, I think one could reasonably conclude that this kind of imbalance doesn't exist in the format.
Please do let me know if I've misunderstood something.
If your answer is "No," then your morality does not come from God's commandments.
If your answer is "Yes," then please, please reconsider.
Yes, basically no deck has an absolute advantage, but the community has still not really demonstrated that archetype is a useful concept.
Yes. The error intervals are still are fairly large (+/-30%) for some match-ups, and others the sample is far too small to make any good inferences, but match-ups rather than deck win %s are what seem to matter.
Definitely. I'll get into this later in a later article, but since you already mention it, but I tried simulating tournaments where players make optimal choices. The image below shows the results.
The results are highly cyclical, but given an "optimal" metagame where all players take into account all deck choices, some decks like Dredge end up with sub-50% win percentages. Affinity doesn't even break 46% win percentage in any tournament.
This is one of the reasons we need a better way of classifying archetypes--to rule this out as the source of change.
Essentially, yes. It's more a methodological contribution. Makes me wonder why they banned Mental Misstep because all the data is pre-ban.
Author of "Flogging the Data"' econometric article series.
For the moment, let's assume that a stable cycle like this is what a healthy meta-game looks like in the long term.
So, can you try modeling theoretical meta-games to see what an unhealthy one would look like? Does it gravitate towards all players playing the best deck at some point? Is there a stable amount of hate-decks, or is that also cyclical?
Also, my last comment wasn't intended to minimize the importance of your contribution. What you're doing here is quite useful. I was just trying to intrepret the data you were providing.
If your answer is "No," then your morality does not come from God's commandments.
If your answer is "Yes," then please, please reconsider.
He said pre ban Misstep. And it is still a deck some of them have done ok at tournaments just not a popular deck any more.
Color composition of each deck
% of types present in each deck
Colors included in each deck
The key cards of the deck
Play style of the deck: aggro, control, combo
However, all of these seem to be far less descriptive than archetype.
BRG Loam Control (Assault - Loam) BRG
W Mono White Control (Martyr - Proc) W
I would argue that there is no method today, e.g. Walls gives no info about the colors, in contrast to BWStoneblade. Its a jumble of card names, pet names (Team America?!), colors, etc. Underscoring the fact that there is no method.
Not necessarily; if decks were name something like:
<Color>-<Strategy>-<Key Cards>
WGr-Aggro-LoamBlade
There would be a much greater uniformity and names would be more informative.
But the goal isn't description, the goal is statistical significance. If archetypes don't have much bearing on outcomes, archetypes aren't a useful way of thinking about names. It may be a convenient short-hand for a deck-list, but not a useful tool of analysis.
Author of "Flogging the Data"' econometric article series.
If your answer is "No," then your morality does not come from God's commandments.
If your answer is "Yes," then please, please reconsider.
BRG Loam Control (Assault - Loam) BRG
W Mono White Control (Martyr - Proc) W
Author of "Flogging the Data"' econometric article series.
Still waiting eagerly for that article!
Especially with so much misunderstanding floating around these forums on the subject of reprints and pricing.
What do you think about shocklands being reprinted in Return to Ravnica? Will it drive their prices up or down?
BRG Loam Control (Assault - Loam) BRG
W Mono White Control (Martyr - Proc) W
Glad to hear someone's interested. It's definitely written, but the editorial team said its unintelligible to all but economists, so they basically vetoed publishing it in its current form. I write for myself more than an audience, and I've been too occupied to essentially rewrite a 19-page article...
This to my knowledge has not been confirmed yet.
Probably up. It would mean they become standard legal, this drives average card prices up by 35% according to my estimates.
The reprint will drive prices down by 5%. So that is a net of about 30%.
Author of "Flogging the Data"' econometric article series.