Here's the thing, they've had a working shuffler on mtgo for a very long time, but for arena they create a new one and suddenly it has a bias that pushes lands drawn slightly closer to what it would be with an optimal average land count.
From what I've heard anecdotally, best of 3 matches seem to be going on mostly as normal, while best of 1's have fewer non games where one player just gets mana screwed or flooded. The data in your link seems to show best of 1's showing the observed effect more, so the anecdotes and data seem to be in alignment.
I'm going to put on my tinfoil hat and turn on the X-Files theme, because I think this is intentional to make best of 1's more viable. The more often you just don't have a chance to play a match because you got mana screwed in one game, the less likely you are to continue with the format, especially for the newer, more casual, and less enfranchised players that best of 1's cater to.
Private Mod Note
():
Rollback Post to RevisionRollBack
The Meaning of Life: "M-hmm. Well, it's nothing very special. Uh, try and be nice to people, avoid eating fat, read a good book every now and then, get some walking in, and try and live together in peace and harmony with people of all creeds and nations"
Onering's 4 simple steps that let you solve any problem with Magic's gameplay
Whether its blue players countering your spells, red players burning you out, or combo, if you have a problem with an aspect of Magic's gameplay, you can fix it!
Step 1: Identify the problem. What aspect of Magic don't you like? Step 2: Find out how others deal with the problem. How do players deal with this aspect of the game when they run into it? Step 3: Do what those players do. Step 4: No more problem. Bonus: You are now better at Magic. Enjoy those extra wins!
I have always been fervent in stating that I wanted to see proof of any rigging in shuffling before admitting it's there, that I'm not going to accept just anecdotal data and the rants of a salty YT crapposter.
This is a sufficient sample size and the data analyzed in a very well-thought out and consistent manner, and the data is presented in a very clear format that makes the point rather clear. I was wrong, there is a problem with the shuffler.
I'm convinced that there is an issue that's going on, and Wizards needs to address this rather quickly because it's a VERY damning issue for them. Until then, about the only thing you can do is arm yourself with the knowledge. For those not wanting to pour through all the technical details, here's a good tl;dr:
-- Limited Decks will constantly be mana-starved, no matter how many you play. Fill up on stuff low on the curve.
-- 22-23 lands is the sweet spot for not getting issues - Play too many more and you get flooded more often, play too many fewer and you get screwed more often compared to expected value.
-- 3 land opening hands are great. 2-land or fewer hands get starved more often, 4-land or more hands get flooded more often compared to expected value.
-- Taking any mulligan seems to put the land/spell ratio back to where it should be.
--By all accounts this appears to be a common mistake in implementing the shuffler algorithm in which the deck is randomized, but not randomized enough. This is why taking a mulligan fixes it. This is an act of incompetence moreso than malfeasance.
-- There is no data to suggest that Arena gives you more copies of specific cards more often than expected.
I am hopeful that the people in charge of this at WotC catch wind of this and are able to affect a fix of some kind. It shouldn't be that hard, from what the report claims.
Top 16 - 2012 Indiana State Championships Currently Playing: GBStandard - Golgari Safari MidrangeBG RBWModern - Mardu PyromancerWBR RLegacy - Good Old Fashioned BurnR
I have not read the entire Reddit post yet. It's not a revelation to me since I had long suspected these things just with my very small sampling size (my own games). The scale of the data set is a lot to digest.
--By all accounts this appears to be a common mistake in implementing the shuffler algorithm in which the deck is randomized, but not randomized enough. This is why taking a mulligan fixes it. This is an act of incompetence moreso than malfeasance.
Interestingly, the Unity document is using the wrong algorithm. WotC was called out on some time ago for using this bad shuffle algorithm on MTGO back in 2014. If we look past the fact that WotC shouldn't be using this algorithm at all, even if it's just the sample hand feature in question, then I would like to believe that WotC knows better. And this understanding of shuffling algorithms carried over into Arena.
If anything, I suspect the hand shaping is the root of the problem. Which would be why mulliganing "fixes" the problem. No hand shaping is occurring thusly no weighting cards to get those artificial hands.
Of course, without looking at the source code, I could be fantastically and gloriously wrong here.
Edit:
Maybe I misunderstood your post. I just had a thought that there is the possibility that the Devs intentionally did not use Fisher-Yates in order to get the hand shaping they're looking for. In other words, they could be using an algorithm that has a known tendency to clump cards in order to reduce, what they consider, bad hands. Mulliganing doesn't do a poor shuffle on an already poorly shuffled deck but literally uses the correct algorithm. If for no other reason that they are not doing the hand shaping so there's no reason to use the garbage shuffled. This could explain the anecdotal evidence that Arena also tends to grab particular 4-ofs early on. The same algorithm that "pulls" lands towards the top also pulls the 4-ofs.
Not sure what you're asking Malumdiabolus. Can you clarify?
Algernone25
Actually, the fact that this problem goes away after a mulligan could be proof that Wizards has nefarious intent. This was one confusing part of the study where he talks about "Maybe the mulligan fixes the problem because it 'shuffles' the deck again, thus making it more random." This doesn't make sense from a programming standpoint (I'm pretty sure at least, I'm not a programmer, Desolator gets into it in his video.) This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
This is the best I can explain it. In paper Magic, you must shuffle your deck for a mulligan. On a computer, instead of having to shuffle that deck to get a new hand, the computer has another identical deck for you, already pre-shuffled. You just set aside the deck you mulliganed and pick up the fresh, pre-shuffled and randomized deck. Mulligan again? The computer has an infinite number of pre-shuffled decks containing all the same cards.
That's why taking a mulligan should have absolutely no impact on the randomness of the deck. If this is an error on Wizards part, it is a TREMENDOUS error. I say it looks more like a cover-up to me. Wizards has known that people complain about the shuffler and constantly tries to reassure them with placating words. Well, data is WAY more placating (or infuriating in this case) than words. Wizards could have run this study themselves with people in house and had a WAY larger sample size. Then they could say "Look everyone, here is your mathematical proof that the shuffler is correct." They didn't, probably because they knew they had something to hide.
If anything, I suspect the hand shaping is the root of the problem. Which would be why mulliganing "fixes" the problem. No hand shaping is occurring thusly no weighting cards to get those artificial hands.
Actually, he accounts for the artificial shaping of hands in his math. At least he claims to, otherwise the study wouldn't be worthwhile. Also, he was taking data from both bo1 games and bo3 games. He stated there was basically no difference in the way the shuffler messes up bo1 vs. bo3. It screws them both up equally, the problems are based more on how many lands you keep in your opening hand, whether playing bo1 or bo3.
Not sure what you're asking Malumdiabolus. Can you clarify?
Algernone25
Actually, the fact that this problem goes away after a mulligan could be proof that Wizards has nefarious intent. This was one confusing part of the study where he talks about "Maybe the mulligan fixes the problem because it 'shuffles' the deck again, thus making it more random." This doesn't make sense from a programming standpoint (I'm pretty sure at least, I'm not a programmer, Desolator gets into it in his video.) This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
This is the best I can explain it. In paper Magic, you must shuffle your deck for a mulligan. On a computer, instead of having to shuffle that deck to get a new hand, the computer has another identical deck for you, already pre-shuffled. You just set aside the deck you mulliganed and pick up the fresh, pre-shuffled and randomized deck. Mulligan again? The computer has an infinite number of pre-shuffled decks containing all the same cards.
That's why taking a mulligan should have absolutely no impact on the randomness of the deck. If this is an error on Wizards part, it is a TREMENDOUS error. I say it looks more like a cover-up to me. Wizards has known that people complain about the shuffler and constantly tries to reassure them with placating words. Well, data is WAY more placating (or infuriating in this case) than words. Wizards could have run this study themselves with people in house and had a WAY larger sample size. Then they could say "Look everyone, here is your mathematical proof that the shuffler is correct." They didn't, probably because they knew they had something to hide.
I believe what Malumdiabolus is trying to ask is if there's a correlation between your player rank (bronze or gold or diamond or mythic) and how often/how much you get the skew from the expected values - that is, the game gives you worse mana issues if you're low rank and gives you better ones at high rank. Even if that's true, I find it likely that such data would be lost in the noise unless you had the extra effort to control for it, which I'm not sure his scraper is capable of.
I'll admit I'm not a programmer either, but I did a bit of reading on the randomization methods that the RedditOP talks about (Fisher-Yates Shuffle and Mersenne Twister) and I have what I think is a good guess of how the shuffler works:
-List cards in the deck from 1 to N, where N is the total number of cards in the deck.
-Use the Mersenne Twister to generate a random number between 1 and N.
-Find that card in the list and place it on the top (or bottom) of the deck.
-Repeat step 2 to generate a new random number, this time between 1 and N-1.
-Find that card in the iist, skipping over cards you've already placed in the list and place that card at the top (or bottom) of the deck.
-Repeat until all cards from the list are placed in a random order.
This looks all well and good from a technical standpoint. I suspect that the issue is caused by two factors - modulo bias and deckbuilding convention.
The Mersenne Twister dones't just pick a number from 1 to 60, it picks a number between 1 and 2^19937-1 which generates a gargantuan number. To make it fit, Arena probably takes the number and divides by your deck size (or deck size remaining to be shuffled) and uses whatever the remainder is as its random number. But since 60 or 40 doesn't evenly divide into that massive number, some remainders are going to be more common than others and that generates a bias, specifically towards the "top" cards in the list. Now consider the fact that when you start making a deck in Arena, it automatically loads lands into the list for you right as you start putting in cards. If you don't use the auto-land filler, lands are almost certainly the last cards you put into your deck.
This results in a state where the game's randomization table is more likely (not by a ton but by enough) to take cards from the top of your list first, and all the lands in your deck are either at the top or the bottom of that list. That's where I think the true issue lies. This might be testable by putting all your basic lands at the front of the decklist and all the non-basics at the back and see if you constantly see one or the other more frequently.
Why a mulligan fixes things, I assume it takes your already shuffled deck as the seed list instead of reverting to the original decklist, and since the lands aren't all at the top or bottom you now get the expected distribution, or one that's within error.
As for wizards not having done this math, I can think of a couple reasons but the biggest one is this: Wizards putting out a claim that the shuffler has no bias just as proof raises a lot of questions behind their logic - similar to putting out a new cereal that's "100% certified asbestos-free". Is it factual, sure. But you've just raised a hell of a lot more concerns than just breakfast cereal.
Private Mod Note
():
Rollback Post to RevisionRollBack
Top 16 - 2012 Indiana State Championships Currently Playing: GBStandard - Golgari Safari MidrangeBG RBWModern - Mardu PyromancerWBR RLegacy - Good Old Fashioned BurnR
Pretty close.
A Fisher-Yates shuffle can randomize a list in place.
Say we have a deck (list) of 4 cards (slots) numbered 0 - 3.
- starting at slot 0
- select a random slot of 0,1,2 or 3 and swap them with slot 0.
- move to slot 1
- select a random slot of 1,2, or 3 and swap it with slot 1.
- move to slot 2
- select a random slot of 2 or 3 and swap it with slot 2.
- ignore slot 3
The method as you describe creates additional work since the algorithm has to do a lot of shifting around with the source list. The Fisher-Yates shuffle takes advantage (in part) of the fast pointer functions of many CPU's. It's also nice and efficient with low RAM requirements.
Not sure what you're asking Malumdiabolus. Can you clarify?
Algernone25
Actually, the fact that this problem goes away after a mulligan could be proof that Wizards has nefarious intent. This was one confusing part of the study where he talks about "Maybe the mulligan fixes the problem because it 'shuffles' the deck again, thus making it more random." This doesn't make sense from a programming standpoint (I'm pretty sure at least, I'm not a programmer, Desolator gets into it in his video.) This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
This is the best I can explain it. In paper Magic, you must shuffle your deck for a mulligan. On a computer, instead of having to shuffle that deck to get a new hand, the computer has another identical deck for you, already pre-shuffled. You just set aside the deck you mulliganed and pick up the fresh, pre-shuffled and randomized deck. Mulligan again? The computer has an infinite number of pre-shuffled decks containing all the same cards.
That's why taking a mulligan should have absolutely no impact on the randomness of the deck. If this is an error on Wizards part, it is a TREMENDOUS error. I say it looks more like a cover-up to me. Wizards has known that people complain about the shuffler and constantly tries to reassure them with placating words. Well, data is WAY more placating (or infuriating in this case) than words. Wizards could have run this study themselves with people in house and had a WAY larger sample size. Then they could say "Look everyone, here is your mathematical proof that the shuffler is correct." They didn't, probably because they knew they had something to hide.
Glad to clarify.
I am wondering if data can prove a player’s rate of actual monetary spending vs grinding for gold daily and causality with the mana error.
In short, are they messing with free to play players so that players who spend money notice better results?
I decided to always mulligan at least once in every game, even if I had a good hand. I still had the default RW deck in my library (I forget the name) and my daily was to play a bunch of RW spells so....
I won 5 out of 7 games. The only losses were against a red Goblin Legion Warboss/Needletooth Raptor deck where I was just overwhelmed with creatures and a Golgari deck that eventually Ultimated.
Notably, except for my Golgari match and a weird Induced Amnesia deck, all of the games felt more normal drawing lands and spells in general. It didn't matter if I kept a difficult 3 mountain hand or a single Plains hand, I drew into my lands roughly when I expected to for that deck.
This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
It is quite possible for a computer to treat a deck of virtual cards the same as a deck of physical cards (e.g. take the 'card' objects, randomize them into a new deck, then randomize the card objects in that new deck into another new deck). Furthermore, true randomization is one of those computer programming issues that is substantially more difficult in practice than in concept, which makes it very believable that they merely screwed something up.
I am wondering if data can prove a player’s rate of actual monetary spending vs grinding for gold daily and causality with the mana error.
In short, are they messing with free to play players so that players who spend money notice better results?
I don't think so. It wouldn't make monetary sense.
Firstly, the difference observed by the study is exploitable; i.e. you can take advantage of it by running slightly fewer lands than in real magic with a reduced consequence, as long as you know not to keep two or fewer lands on your first hand. I don't know what rank or payment status the accounts used to collect that data were, but practicality suggests that they were the minimum needed and thus should have been penalized if your concern is accurate.
Secondly, players would need to notice the difference, and then attribute that difference to their spending habits, in order for them to have an incentive to spend more money and increase profit for WotC. It wouldn't help them to go about this so subtly. If that much thought and deliberation was placed into the decision, then it seems unlikely they would decide that any resulting small increase in revenue would be worth the potential to tremendously damage their reputation if the manipulations were discovered.
On the other hand I can confirm as a software engineer that these sorts of errors are easy to make. That MTG Arena's user base is much larger than many other applications only means so much in terms of being able to assume that it's any more robust in design as a result. WotC are historically not at the top of the heap when it comes to sound software design, and I would wager a guess that it's because they are a tabletop and card game company first and foremost; their management likely doesn't fully understand good software practices or what a digital development team needs to succeed.
I think this is the ansaer I was most hopeful to hear, though it doesn’t tell me if it can be analyzed through the data collected.
Seems doubtful that the study guy would bother to collect how much people spend on Arena. Is that something that's even stored in the game's log anywhere? He was using some kind of Arena add-on software to gather the data. Just seems unlikely that they would develop software to discern how much users spend on Arena.
I see what you're trying to get at Anachronity. However, let me counter with a reason why they WOULD use something like this to make $. Say you rig the shuffler so that the more you win, the worse hands you start getting. Wizards keeps track of how much you spend or don't spend, on Arena. Rather than "rewarding" those who purchase gems, you "punish" those who do not purchase gems. You give them worse hands (or the worse shuffler) as they win more and more gold when they don't spend any money on the game. You punish them by putting their really good deck against it's worst match up opponent more often. Just enough that they can't win as much as they'd like.
What is the solution to a player who is frustrated because their deck just isn't quite good enough? Spend more $ on the game to improve the deck of course! Or spend even more $ and build that deck that keeps beating you. The user does not have to be conscious of the connection between lack of spending $ and losing in this case. All they have to do is get fed up with losing, make the logical choice that spending more $ could make their deck better.
So, let's not put those tinfoil hats away just yet!
You're REALLY gonna need that hat after I reveal this:
DesolatorMagic (A youtuber who has been on this "rigged" shuffler since about the beginning. His theories have been proven right by this study. He's also been collecting data himself.) DesolatorMagic has been keeping track of whether or not he gets to play first. After 300 games, he says it's OBVIOUSLY not a 50/50 chance of whether or not you go first. However, he is going to wait until he gets to 500 games to reveal his data so the usual haters can't say "Not enough data to draw a conclusion." this will happen soon.
I understand that programming is hard, and this shuffler thing could be a mistake. HOWEVER, I would find it EXTREMELY hard to believe that it's hard to program randomness into a coinflip. Is it possible they made a programming error on the coin flip too? If they did, HOLY ***** are they incompetent over there, I mean worse than I suspected. If they didn't, how can this be anything but nefarious evil-doings?
Now, when you add up the "not random shuffler" and the "not random coin flip" together, things look REALLY bad.
Another thing: The study didn't conclude that there was no data to support that cards do clump together. He said, he hadn't thought to focus on that in the beginning of the study. Also, that due to the complexity of tracking the individual decks sequential games, the data is less significant. Also, that he was only tracking ONE set of cards per game, not ALL cards that include multiple copies in the game. He said that due to the number of games tracked (and that he only tracked one set of cards per game) he didn't have enough data to make a solid conclusion on that aspect. He did say there was something off in the data, but he didn't have enough games (high enough confidence level in the results) to make a solid conclusion.
That's very different than "No data to support cards clumping up in multiples."
Quote from the study:
I could show more charts at various positions, or the ones for including all sets of cards, but I don't think it would be meaningfully informative. The trend is that there's something off, but it's weak and only showing as significant because of the sheer number of games tracked. I would not be surprised if there's a substantially stronger trend for cards in certain places in the decklist, but position in the decklist is not something I thought to record and aggregate.
And also
I can view those statistics, but for my main analysis I look at only one set of identical cards per game. Looks like big problems everywhere, here, with the only green cells being ones with few games.
Not sure what you're asking Malumdiabolus. Can you clarify?
Algernone25
Actually, the fact that this problem goes away after a mulligan could be proof that Wizards has nefarious intent. This was one confusing part of the study where he talks about "Maybe the mulligan fixes the problem because it 'shuffles' the deck again, thus making it more random." This doesn't make sense from a programming standpoint (I'm pretty sure at least, I'm not a programmer, Desolator gets into it in his video.) This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
This is the best I can explain it. In paper Magic, you must shuffle your deck for a mulligan. On a computer, instead of having to shuffle that deck to get a new hand, the computer has another identical deck for you, already pre-shuffled. You just set aside the deck you mulliganed and pick up the fresh, pre-shuffled and randomized deck. Mulligan again? The computer has an infinite number of pre-shuffled decks containing all the same cards.
That's why taking a mulligan should have absolutely no impact on the randomness of the deck. If this is an error on Wizards part, it is a TREMENDOUS error. I say it looks more like a cover-up to me. Wizards has known that people complain about the shuffler and constantly tries to reassure them with placating words. Well, data is WAY more placating (or infuriating in this case) than words. Wizards could have run this study themselves with people in house and had a WAY larger sample size. Then they could say "Look everyone, here is your mathematical proof that the shuffler is correct." They didn't, probably because they knew they had something to hide.
I believe what Malumdiabolus is trying to ask is if there's a correlation between your player rank (bronze or gold or diamond or mythic) and how often/how much you get the skew from the expected values - that is, the game gives you worse mana issues if you're low rank and gives you better ones at high rank. Even if that's true, I find it likely that such data would be lost in the noise unless you had the extra effort to control for it, which I'm not sure his scraper is capable of.
I'll admit I'm not a programmer either, but I did a bit of reading on the randomization methods that the RedditOP talks about (Fisher-Yates Shuffle and Mersenne Twister) and I have what I think is a good guess of how the shuffler works:
-List cards in the deck from 1 to N, where N is the total number of cards in the deck.
-Use the Mersenne Twister to generate a random number between 1 and N.
-Find that card in the list and place it on the top (or bottom) of the deck.
-Repeat step 2 to generate a new random number, this time between 1 and N-1.
-Find that card in the iist, skipping over cards you've already placed in the list and place that card at the top (or bottom) of the deck.
-Repeat until all cards from the list are placed in a random order.
This looks all well and good from a technical standpoint. I suspect that the issue is caused by two factors - modulo bias and deckbuilding convention.
The Mersenne Twister dones't just pick a number from 1 to 60, it picks a number between 1 and 2^19937-1 which generates a gargantuan number. To make it fit, Arena probably takes the number and divides by your deck size (or deck size remaining to be shuffled) and uses whatever the remainder is as its random number. But since 60 or 40 doesn't evenly divide into that massive number, some remainders are going to be more common than others and that generates a bias, specifically towards the "top" cards in the list. Now consider the fact that when you start making a deck in Arena, it automatically loads lands into the list for you right as you start putting in cards. If you don't use the auto-land filler, lands are almost certainly the last cards you put into your deck.
This results in a state where the game's randomization table is more likely (not by a ton but by enough) to take cards from the top of your list first, and all the lands in your deck are either at the top or the bottom of that list. That's where I think the true issue lies. This might be testable by putting all your basic lands at the front of the decklist and all the non-basics at the back and see if you constantly see one or the other more frequently.
Why a mulligan fixes things, I assume it takes your already shuffled deck as the seed list instead of reverting to the original decklist, and since the lands aren't all at the top or bottom you now get the expected distribution, or one that's within error.
As for wizards not having done this math, I can think of a couple reasons but the biggest one is this: Wizards putting out a claim that the shuffler has no bias just as proof raises a lot of questions behind their logic - similar to putting out a new cereal that's "100% certified asbestos-free". Is it factual, sure. But you've just raised a hell of a lot more concerns than just breakfast cereal.
Okay, one more thing I was thinking of. I see somewhat what you are saying here. Did they say what kind of generator they are using? Or are these just guesses the author is putting forth because they are the most likely candidates for use?
I see what you're saying now with how the deck order could matter somewhat if they were to divide that huge number by 60. Perhaps mixing up the deck order somewhat will help with this? Not just the lands, but mix up the cmc's and distribute lands into each clump of differenct cmc cards. Also, I'm trying out 2 different sets of lightning strike, placing them in separate piles in hopes of reducing clumps. Can't do this with many cards though...
EDIT!!!! Wait a second now!!! The deck builder automatically sorts everything right back into the usual order as soon as you click done, then reopen the list. That means it's IMPOSSIBLE to try and see if the order of the deck in the deck builder's list has an influence on the randomization factor of the shuffler.
EDIT!!!! Wait a second now!!! The deck builder automatically sorts everything right back into the usual order as soon as you click done, then reopen the list. That means it's IMPOSSIBLE to try and see if the order of the deck in the deck builder's list has an influence on the randomization factor of the shuffler.
It may sort things back into order in the deck building interface, but if you export the decklist to a text file it will be in "cards added" order.
RedditOP did not say he knew what methods WotC used but made some educated guesses because they are apparently near industry standard.
Regarding Dessy, the "who gets to play/draw first" thing is interesting but means nothing unless we have an explanation for why it's uneven, and can test that explanation.
Similarly, what RedditOP has discovered a very probable explanation for the phenomenon, but there's a fair degree of HARKing (Hypothesizing After Results are Known) which means his conclusion is circumstantial. Granted, it's enough circumstantial evidence for me to believe it as fact, but to statistically and scientifically PROVE it as fact requires hypothesis testing that as of yet hasn't been done. (Though I expect once it is done it will confirm the data we've discussed.)
As for the Pay to Play allegations, that's probably another argument entirely, though I'm going to lean on Hanlon's Razor: "Never attribute to malice that which is adequately explained by incompetence. (The first time)"
Private Mod Note
():
Rollback Post to RevisionRollBack
Top 16 - 2012 Indiana State Championships Currently Playing: GBStandard - Golgari Safari MidrangeBG RBWModern - Mardu PyromancerWBR RLegacy - Good Old Fashioned BurnR
So, let's not put those tinfoil hats away just yet!
You're REALLY gonna need that hat after I reveal this:
DesolatorMagic (A youtuber who has been on this "rigged" shuffler since about the beginning. His theories have been proven right by this study. He's also been collecting data himself.) DesolatorMagic has been keeping track of whether or not he gets to play first. After 300 games, he says it's OBVIOUSLY not a 50/50 chance of whether or not you go first. However, he is going to wait until he gets to 500 games to reveal his data so the usual haters can't say "Not enough data to draw a conclusion." this will happen soon.
I hate to be *that person* on statistics, but... This is one of those examples where the average person doesn't really understand it well. I mean, I don't. That said this is a case where I have been corrected regularly by people who know statistics better.
Sure, it *seems* like he should be close to 50/50 in going first. But not really. 50/50 should be the mean of the distribution for a *population* of people after a large number of games. DesolatorMagic is one individual in that sample population. Depending on the particular characteristics of this distribution, there could be individuals who have *1 or even 0* instances of going first after 500 games. That is statistically unlikely, yes, but with a large population that is possible.
DesolatorMagic's experiment is kinda meaningless, since he is only one "sample" in the population distribution. His experiment only proves that his experience is not the mean of the overall population (if indeed the true mean is 50/50), and not that the mean of the overall population is or is not 50/50.
Edit: Basically DesolatorMagic's experiment does not test the true population mean in any way.
DesolatorMagic's experiment is kinda meaningless, since he is only one "sample" in the population distribution. His experiment only proves that his experience is not the mean of the overall population (if indeed the true mean is 50/50), and not that the mean of the overall population is 50/50.
Edit: Basically DesolatorMagic's experiment does not test the true population mean in any way.
You're mostly correct but I'd like to note something. You're right that Dessy is only one data point. However, going first is a binary state - every game in Arena has to have one player go first, so getting a ton of data points from a ton of games will show a near perfect 50/50 of people going first, because half of everyone HAS to go first.
What Dessy's data set is showing (and 500 is probably a large enough number of trials to make a reasonable judgment) is that there is an aberration. It's notable, but it's also useless unless you can identify WHY there's an aberration and HOW that aberration is affecting the play/draw options. Again, hypothesis testing.
Private Mod Note
():
Rollback Post to RevisionRollBack
Top 16 - 2012 Indiana State Championships Currently Playing: GBStandard - Golgari Safari MidrangeBG RBWModern - Mardu PyromancerWBR RLegacy - Good Old Fashioned BurnR
*IN RESPONSE TO TREE IN THE WIND* This is not actually true. Even a single individual, given enough repetitions with something as simple as a coinflip, can show that something is off.
Think of it this way. One individual winning the lottery is no big deal. That individual will think "Oh, I won the lottery. It must not be that hard." Others may think "Well someone ALWAYS wins the lottery eventually. The odds must not be that bad."
However, if the SAME person out of millions wins the lottery, say twice, or 3 times, that is so statistically unlikely that it does demonstrate something could be off.
There is a way to calculate the confidence level of a coin flip come up heads 0 times out of 500 flips. Trust me, the chances of this are astronomical. Now of course, that isn't what will happen. I'm guessing multiple times the number of Magic games that have even been played on Arena. It's been a while since I studied this advanced level of statistical math. However, there is a way to increase confidence of receiving strange results even with only 1 participant who is way off average.
Also, the coinflip/who goes fist thing is easy to replicate. Rumors and anecdotes are how this study even was started in the first place. A study to see how often an individual goes first is easy. Could even be done by a group of say 10 individuals each recording say 500 games each. Then we look at the distribution of how many people are outside the normal distribution then we would expect. If we keep getting individuals who's games vary greatly from 50% chance of going first, it is getting pretty close to definitive proof that something is wrong.
What I'm getting at, if the results are strange enough with DesolatorMagic's experiment, you can't just discount them because "it's only 1 person." If he goes second 55% of the time, that means nothing. 60% of the time, still nothing to worry about. If he does second like 66%, or 70%, that would be VERY strange to have happen in a sample size of 500. Impossible? No. Will it inspire others to start keeping track? Probably.
https://www.reddit.com/r/MagicArena/comments/b21u3n/i_analyzed_shuffling_in_a_million_games/
A video talking about the study.
https://www.youtube.com/watch?v=hR0fhvy9LOQ
From what I've heard anecdotally, best of 3 matches seem to be going on mostly as normal, while best of 1's have fewer non games where one player just gets mana screwed or flooded. The data in your link seems to show best of 1's showing the observed effect more, so the anecdotes and data seem to be in alignment.
I'm going to put on my tinfoil hat and turn on the X-Files theme, because I think this is intentional to make best of 1's more viable. The more often you just don't have a chance to play a match because you got mana screwed in one game, the less likely you are to continue with the format, especially for the newer, more casual, and less enfranchised players that best of 1's cater to.
Onering's 4 simple steps that let you solve any problem with Magic's gameplay
Step 1: Identify the problem. What aspect of Magic don't you like? Step 2: Find out how others deal with the problem. How do players deal with this aspect of the game when they run into it? Step 3: Do what those players do. Step 4: No more problem. Bonus: You are now better at Magic. Enjoy those extra wins!
This is a sufficient sample size and the data analyzed in a very well-thought out and consistent manner, and the data is presented in a very clear format that makes the point rather clear. I was wrong, there is a problem with the shuffler.
I'm convinced that there is an issue that's going on, and Wizards needs to address this rather quickly because it's a VERY damning issue for them. Until then, about the only thing you can do is arm yourself with the knowledge. For those not wanting to pour through all the technical details, here's a good tl;dr:
-- Limited Decks will constantly be mana-starved, no matter how many you play. Fill up on stuff low on the curve.
-- 22-23 lands is the sweet spot for not getting issues - Play too many more and you get flooded more often, play too many fewer and you get screwed more often compared to expected value.
-- 3 land opening hands are great. 2-land or fewer hands get starved more often, 4-land or more hands get flooded more often compared to expected value.
-- Taking any mulligan seems to put the land/spell ratio back to where it should be.
--By all accounts this appears to be a common mistake in implementing the shuffler algorithm in which the deck is randomized, but not randomized enough. This is why taking a mulligan fixes it. This is an act of incompetence moreso than malfeasance.
-- There is no data to suggest that Arena gives you more copies of specific cards more often than expected.
I am hopeful that the people in charge of this at WotC catch wind of this and are able to affect a fix of some kind. It shouldn't be that hard, from what the report claims.
Currently Playing:
GBStandard - Golgari Safari MidrangeBG
RBWModern - Mardu PyromancerWBR
RLegacy - Good Old Fashioned BurnR
Clan Contest 3 Mafia - Mafia Co-MVP
Interestingly, the Unity document is using the wrong algorithm. WotC was called out on some time ago for using this bad shuffle algorithm on MTGO back in 2014. If we look past the fact that WotC shouldn't be using this algorithm at all, even if it's just the sample hand feature in question, then I would like to believe that WotC knows better. And this understanding of shuffling algorithms carried over into Arena.
If anything, I suspect the hand shaping is the root of the problem. Which would be why mulliganing "fixes" the problem. No hand shaping is occurring thusly no weighting cards to get those artificial hands.
Of course, without looking at the source code, I could be fantastically and gloriously wrong here.
Edit:
Maybe I misunderstood your post. I just had a thought that there is the possibility that the Devs intentionally did not use Fisher-Yates in order to get the hand shaping they're looking for. In other words, they could be using an algorithm that has a known tendency to clump cards in order to reduce, what they consider, bad hands. Mulliganing doesn't do a poor shuffle on an already poorly shuffled deck but literally uses the correct algorithm. If for no other reason that they are not doing the hand shaping so there's no reason to use the garbage shuffled. This could explain the anecdotal evidence that Arena also tends to grab particular 4-ofs early on. The same algorithm that "pulls" lands towards the top also pulls the 4-ofs.
Algernone25
Actually, the fact that this problem goes away after a mulligan could be proof that Wizards has nefarious intent. This was one confusing part of the study where he talks about "Maybe the mulligan fixes the problem because it 'shuffles' the deck again, thus making it more random." This doesn't make sense from a programming standpoint (I'm pretty sure at least, I'm not a programmer, Desolator gets into it in his video.) This is not a physical deck of cards that needs shuffling. They are using some kind of generator/algorithm to give you the opening hand and determine the card order sequence. A mulligan should in no way be influenced by how many hands you've seen in the digital world.
This is the best I can explain it. In paper Magic, you must shuffle your deck for a mulligan. On a computer, instead of having to shuffle that deck to get a new hand, the computer has another identical deck for you, already pre-shuffled. You just set aside the deck you mulliganed and pick up the fresh, pre-shuffled and randomized deck. Mulligan again? The computer has an infinite number of pre-shuffled decks containing all the same cards.
That's why taking a mulligan should have absolutely no impact on the randomness of the deck. If this is an error on Wizards part, it is a TREMENDOUS error. I say it looks more like a cover-up to me. Wizards has known that people complain about the shuffler and constantly tries to reassure them with placating words. Well, data is WAY more placating (or infuriating in this case) than words. Wizards could have run this study themselves with people in house and had a WAY larger sample size. Then they could say "Look everyone, here is your mathematical proof that the shuffler is correct." They didn't, probably because they knew they had something to hide.
Actually, he accounts for the artificial shaping of hands in his math. At least he claims to, otherwise the study wouldn't be worthwhile. Also, he was taking data from both bo1 games and bo3 games. He stated there was basically no difference in the way the shuffler messes up bo1 vs. bo3. It screws them both up equally, the problems are based more on how many lands you keep in your opening hand, whether playing bo1 or bo3.
I believe what Malumdiabolus is trying to ask is if there's a correlation between your player rank (bronze or gold or diamond or mythic) and how often/how much you get the skew from the expected values - that is, the game gives you worse mana issues if you're low rank and gives you better ones at high rank. Even if that's true, I find it likely that such data would be lost in the noise unless you had the extra effort to control for it, which I'm not sure his scraper is capable of.
I'll admit I'm not a programmer either, but I did a bit of reading on the randomization methods that the RedditOP talks about (Fisher-Yates Shuffle and Mersenne Twister) and I have what I think is a good guess of how the shuffler works:
-List cards in the deck from 1 to N, where N is the total number of cards in the deck.
-Use the Mersenne Twister to generate a random number between 1 and N.
-Find that card in the list and place it on the top (or bottom) of the deck.
-Repeat step 2 to generate a new random number, this time between 1 and N-1.
-Find that card in the iist, skipping over cards you've already placed in the list and place that card at the top (or bottom) of the deck.
-Repeat until all cards from the list are placed in a random order.
This looks all well and good from a technical standpoint. I suspect that the issue is caused by two factors - modulo bias and deckbuilding convention.
The Mersenne Twister dones't just pick a number from 1 to 60, it picks a number between 1 and 2^19937-1 which generates a gargantuan number. To make it fit, Arena probably takes the number and divides by your deck size (or deck size remaining to be shuffled) and uses whatever the remainder is as its random number. But since 60 or 40 doesn't evenly divide into that massive number, some remainders are going to be more common than others and that generates a bias, specifically towards the "top" cards in the list. Now consider the fact that when you start making a deck in Arena, it automatically loads lands into the list for you right as you start putting in cards. If you don't use the auto-land filler, lands are almost certainly the last cards you put into your deck.
This results in a state where the game's randomization table is more likely (not by a ton but by enough) to take cards from the top of your list first, and all the lands in your deck are either at the top or the bottom of that list. That's where I think the true issue lies. This might be testable by putting all your basic lands at the front of the decklist and all the non-basics at the back and see if you constantly see one or the other more frequently.
Why a mulligan fixes things, I assume it takes your already shuffled deck as the seed list instead of reverting to the original decklist, and since the lands aren't all at the top or bottom you now get the expected distribution, or one that's within error.
As for wizards not having done this math, I can think of a couple reasons but the biggest one is this: Wizards putting out a claim that the shuffler has no bias just as proof raises a lot of questions behind their logic - similar to putting out a new cereal that's "100% certified asbestos-free". Is it factual, sure. But you've just raised a hell of a lot more concerns than just breakfast cereal.
Currently Playing:
GBStandard - Golgari Safari MidrangeBG
RBWModern - Mardu PyromancerWBR
RLegacy - Good Old Fashioned BurnR
Clan Contest 3 Mafia - Mafia Co-MVP
A Fisher-Yates shuffle can randomize a list in place.
Say we have a deck (list) of 4 cards (slots) numbered 0 - 3.
- starting at slot 0
- select a random slot of 0,1,2 or 3 and swap them with slot 0.
- move to slot 1
- select a random slot of 1,2, or 3 and swap it with slot 1.
- move to slot 2
- select a random slot of 2 or 3 and swap it with slot 2.
- ignore slot 3
The method as you describe creates additional work since the algorithm has to do a lot of shifting around with the source list. The Fisher-Yates shuffle takes advantage (in part) of the fast pointer functions of many CPU's. It's also nice and efficient with low RAM requirements.
Glad to clarify.
I am wondering if data can prove a player’s rate of actual monetary spending vs grinding for gold daily and causality with the mana error.
In short, are they messing with free to play players so that players who spend money notice better results?
I decided to always mulligan at least once in every game, even if I had a good hand. I still had the default RW deck in my library (I forget the name) and my daily was to play a bunch of RW spells so....
I won 5 out of 7 games. The only losses were against a red Goblin Legion Warboss/Needletooth Raptor deck where I was just overwhelmed with creatures and a Golgari deck that eventually Ultimated.
Notably, except for my Golgari match and a weird Induced Amnesia deck, all of the games felt more normal drawing lands and spells in general. It didn't matter if I kept a difficult 3 mountain hand or a single Plains hand, I drew into my lands roughly when I expected to for that deck.
It is quite possible for a computer to treat a deck of virtual cards the same as a deck of physical cards (e.g. take the 'card' objects, randomize them into a new deck, then randomize the card objects in that new deck into another new deck). Furthermore, true randomization is one of those computer programming issues that is substantially more difficult in practice than in concept, which makes it very believable that they merely screwed something up.
I don't think so. It wouldn't make monetary sense.
Firstly, the difference observed by the study is exploitable; i.e. you can take advantage of it by running slightly fewer lands than in real magic with a reduced consequence, as long as you know not to keep two or fewer lands on your first hand. I don't know what rank or payment status the accounts used to collect that data were, but practicality suggests that they were the minimum needed and thus should have been penalized if your concern is accurate.
Secondly, players would need to notice the difference, and then attribute that difference to their spending habits, in order for them to have an incentive to spend more money and increase profit for WotC. It wouldn't help them to go about this so subtly. If that much thought and deliberation was placed into the decision, then it seems unlikely they would decide that any resulting small increase in revenue would be worth the potential to tremendously damage their reputation if the manipulations were discovered.
On the other hand I can confirm as a software engineer that these sorts of errors are easy to make. That MTG Arena's user base is much larger than many other applications only means so much in terms of being able to assume that it's any more robust in design as a result. WotC are historically not at the top of the heap when it comes to sound software design, and I would wager a guess that it's because they are a tabletop and card game company first and foremost; their management likely doesn't fully understand good software practices or what a digital development team needs to succeed.
- Rabid Wombat
Seems doubtful that the study guy would bother to collect how much people spend on Arena. Is that something that's even stored in the game's log anywhere? He was using some kind of Arena add-on software to gather the data. Just seems unlikely that they would develop software to discern how much users spend on Arena.
I see what you're trying to get at Anachronity. However, let me counter with a reason why they WOULD use something like this to make $. Say you rig the shuffler so that the more you win, the worse hands you start getting. Wizards keeps track of how much you spend or don't spend, on Arena. Rather than "rewarding" those who purchase gems, you "punish" those who do not purchase gems. You give them worse hands (or the worse shuffler) as they win more and more gold when they don't spend any money on the game. You punish them by putting their really good deck against it's worst match up opponent more often. Just enough that they can't win as much as they'd like.
What is the solution to a player who is frustrated because their deck just isn't quite good enough? Spend more $ on the game to improve the deck of course! Or spend even more $ and build that deck that keeps beating you. The user does not have to be conscious of the connection between lack of spending $ and losing in this case. All they have to do is get fed up with losing, make the logical choice that spending more $ could make their deck better.
So, let's not put those tinfoil hats away just yet!
You're REALLY gonna need that hat after I reveal this:
DesolatorMagic (A youtuber who has been on this "rigged" shuffler since about the beginning. His theories have been proven right by this study. He's also been collecting data himself.) DesolatorMagic has been keeping track of whether or not he gets to play first. After 300 games, he says it's OBVIOUSLY not a 50/50 chance of whether or not you go first. However, he is going to wait until he gets to 500 games to reveal his data so the usual haters can't say "Not enough data to draw a conclusion." this will happen soon.
I understand that programming is hard, and this shuffler thing could be a mistake. HOWEVER, I would find it EXTREMELY hard to believe that it's hard to program randomness into a coinflip. Is it possible they made a programming error on the coin flip too? If they did, HOLY ***** are they incompetent over there, I mean worse than I suspected. If they didn't, how can this be anything but nefarious evil-doings?
Now, when you add up the "not random shuffler" and the "not random coin flip" together, things look REALLY bad.
That's very different than "No data to support cards clumping up in multiples."
Quote from the study:
And also
I see what you're saying now with how the deck order could matter somewhat if they were to divide that huge number by 60. Perhaps mixing up the deck order somewhat will help with this? Not just the lands, but mix up the cmc's and distribute lands into each clump of differenct cmc cards. Also, I'm trying out 2 different sets of lightning strike, placing them in separate piles in hopes of reducing clumps. Can't do this with many cards though...
EDIT!!!! Wait a second now!!! The deck builder automatically sorts everything right back into the usual order as soon as you click done, then reopen the list. That means it's IMPOSSIBLE to try and see if the order of the deck in the deck builder's list has an influence on the randomization factor of the shuffler.
***ADDS ANOTHER LAYER TO MY TINFOIL HAT***
It may sort things back into order in the deck building interface, but if you export the decklist to a text file it will be in "cards added" order.
RedditOP did not say he knew what methods WotC used but made some educated guesses because they are apparently near industry standard.
Regarding Dessy, the "who gets to play/draw first" thing is interesting but means nothing unless we have an explanation for why it's uneven, and can test that explanation.
Similarly, what RedditOP has discovered a very probable explanation for the phenomenon, but there's a fair degree of HARKing (Hypothesizing After Results are Known) which means his conclusion is circumstantial. Granted, it's enough circumstantial evidence for me to believe it as fact, but to statistically and scientifically PROVE it as fact requires hypothesis testing that as of yet hasn't been done. (Though I expect once it is done it will confirm the data we've discussed.)
As for the Pay to Play allegations, that's probably another argument entirely, though I'm going to lean on Hanlon's Razor: "Never attribute to malice that which is adequately explained by incompetence. (The first time)"
Currently Playing:
GBStandard - Golgari Safari MidrangeBG
RBWModern - Mardu PyromancerWBR
RLegacy - Good Old Fashioned BurnR
Clan Contest 3 Mafia - Mafia Co-MVP
I hate to be *that person* on statistics, but... This is one of those examples where the average person doesn't really understand it well. I mean, I don't. That said this is a case where I have been corrected regularly by people who know statistics better.
Sure, it *seems* like he should be close to 50/50 in going first. But not really. 50/50 should be the mean of the distribution for a *population* of people after a large number of games. DesolatorMagic is one individual in that sample population. Depending on the particular characteristics of this distribution, there could be individuals who have *1 or even 0* instances of going first after 500 games. That is statistically unlikely, yes, but with a large population that is possible.
DesolatorMagic's experiment is kinda meaningless, since he is only one "sample" in the population distribution. His experiment only proves that his experience is not the mean of the overall population (if indeed the true mean is 50/50), and not that the mean of the overall population is or is not 50/50.
Edit: Basically DesolatorMagic's experiment does not test the true population mean in any way.
You're mostly correct but I'd like to note something. You're right that Dessy is only one data point. However, going first is a binary state - every game in Arena has to have one player go first, so getting a ton of data points from a ton of games will show a near perfect 50/50 of people going first, because half of everyone HAS to go first.
What Dessy's data set is showing (and 500 is probably a large enough number of trials to make a reasonable judgment) is that there is an aberration. It's notable, but it's also useless unless you can identify WHY there's an aberration and HOW that aberration is affecting the play/draw options. Again, hypothesis testing.
Currently Playing:
GBStandard - Golgari Safari MidrangeBG
RBWModern - Mardu PyromancerWBR
RLegacy - Good Old Fashioned BurnR
Clan Contest 3 Mafia - Mafia Co-MVP
Think of it this way. One individual winning the lottery is no big deal. That individual will think "Oh, I won the lottery. It must not be that hard." Others may think "Well someone ALWAYS wins the lottery eventually. The odds must not be that bad."
However, if the SAME person out of millions wins the lottery, say twice, or 3 times, that is so statistically unlikely that it does demonstrate something could be off.
There is a way to calculate the confidence level of a coin flip come up heads 0 times out of 500 flips. Trust me, the chances of this are astronomical. Now of course, that isn't what will happen. I'm guessing multiple times the number of Magic games that have even been played on Arena. It's been a while since I studied this advanced level of statistical math. However, there is a way to increase confidence of receiving strange results even with only 1 participant who is way off average.
Also, the coinflip/who goes fist thing is easy to replicate. Rumors and anecdotes are how this study even was started in the first place. A study to see how often an individual goes first is easy. Could even be done by a group of say 10 individuals each recording say 500 games each. Then we look at the distribution of how many people are outside the normal distribution then we would expect. If we keep getting individuals who's games vary greatly from 50% chance of going first, it is getting pretty close to definitive proof that something is wrong.
What I'm getting at, if the results are strange enough with DesolatorMagic's experiment, you can't just discount them because "it's only 1 person." If he goes second 55% of the time, that means nothing. 60% of the time, still nothing to worry about. If he does second like 66%, or 70%, that would be VERY strange to have happen in a sample size of 500. Impossible? No. Will it inspire others to start keeping track? Probably.