It's a nice big chunk of data, but it's from 1 person. I am not putting the OP's credibility in question, but we only have his word that the data has been correctly recorded, and mistakes do happen. It would simply be prudent to gather multiple counterpoints from people willing to do the effort before attempting to draw conclusions. If you get a set of 4/5 people doing 100/200 games for decks with various landcounts, then you got a good base to evaluate further. Beyond that I personally couldn't care less where it gets posted

I think the most helpful would be someone willing to do a set for a 26 land deck to juxtapose directly with OP's findings. I chose 23 just to show people it can be done fast, even vs online opponents, but I truly could not muster supplying this amount of data with a classic control deck

You could just run a relatively aggro-ish deck like mid range ramp, but with elves instead of land search. Just a bit on the high end of the curve. Those games would be pretty quick, and there are plenty of targets for t 3, 4, 5, 6 bombs.

I think people are reluctant to do so because 'bolt the bird' in the current meta (at least on steam) is a huge thing. Everyone wants early answers for the more broken openings (thank you constrictor/vehicles) and when you play a 1/3 hoping to skip t3 the following turn, you tend to die a horrible horrible death. My R/G aggro build was fast as hell ( as the data shows) and won ALOT

But uniquely for testing purposes, a true G/C/X midranger with 26 land would probably be ideal, I agree. Although you wouldn't be allowed to run Ruin in their Wake for instance.

@DJ
A RNG very well can produce biased and non regular results.
In fact I remember there being a very good test suite for RNGs called DIEHARD which looks for this sort of thing.
Also even if the personal data of 1700 games is legit, unless this result is repeatable multiple times with similar data , it does not mean the RNG is broken (In a relative sense).

Define biased. Because these weren't slightly biased, which wouldn't have surprised me much, this was like trying to pin point a dot in Paris, and ending up in Dubai (or possibly Berlin.., still far off indeed).

I totally agree with the last bit, in case it wasn't clear. We've got something that seemingly implies the highly unlikely, I'd really like to see if we can confirm it, or discard it.

How often should a working shuffler force you to mulligan to 5 because of lack of land in a 26 land deck, then proceed to give you 11 land out of the first 14 cards? Just happened to me, again...

Dmannn, you really shouldn't be so worried about that kind of thing. I get that it's annoying, but that's not really a meaningful event. The reason why the OP's data is nuts isn't annoying results like the ones you are talking about, it's borderline impossible ones like drawing 12 out of 12 lands on the order of 1 in every 17 games. If you start seeing things like that happen, let us know, and of course collect the data as best as you can. And make sure to collect all of the data, not just the 'interesting' stuff.

How is that not meaningful? If statistical improbabilities happen over and over again, alarms should start going off in your head. Now this was the first time in several games that this happened to me, but I've had anomalies like this go on for days to weeks at a time before. Also, a lot of these things happen when you keep a 5+ land hand. Everybody has recognized that.

That's just not how probability or statistics work. /shrug

I can tell you wether, after a hundred trials, something is likely to be off with the shuffler. 2 to 4 events in a row, even if you think they are improbable, have too much variance to have any statistical meaning. This is why we are gathering data, rather than trying to calculate individual event probabilities, which is a waste of time. Or let me put this another way, every event you'd examine after the fact is likely to have a vanishingly small probability to have happened. But who cares, it happened, so we know it was possible.

Exactly, statiatics and data can show you different variables, but results from a session can determine plenty of answers; especially if we're getting similar dependent variables often.

This thread has caused me to flood more often.

Well, I have a theory that would explain this. Lets assume that duels decks starts of sorted the way they are in the deckbuilder, meaning that we have all the land in the bottom of the deck. Then we shuffle using a random comparator. This introduces a bias for cards originally in the bottom of the deck to end up closer to the top. Now, because most magic games only look at the top ~20 cards of the deck, that bias will add up to some statistical significance. There is a nice visualization of different distributions caused by bad shufflers here:

https://bost.ocks.org/mike/shuffle/compare.html

Things that would make or break this theory are:

Does the amount of bias decrease with number of cards drawn, if it does, it supports my theory.
Are the particular lands in the back of a deck more likely to show up in the deck then other cards? If they do, that supports this theory.

i want Rabble to tell the hummingbird story again, that was awesome

In the larger data set provided by VT2WA we have evidence of the bold (we don't have data for the second part - we did not for example count how often rare lands showed up). There is a statistically relevant downward trend as card count increases. ~ -.0063874% per card drawn, and it's statistically different from 0.

GoboRab's data does not agree with this finding. If anything in GoboRab's data the trend is positive instead of negative, but we can't reject 0 (95% CI -.0084004 to .0193979) - 0 is what we'd expect the trend to be if the shuffler was built correctly, and his data's confidence interval includes 0.

So basically, no we don't have conclusive evidence to support this theory, and we have found yet another way the two data sets contradict each other. Need more data.

Might be that GoboRab's data set is to small to show this possibility. What's the likelihood of that happening?

If someone have time, there is one fairly easy test to do. Build a deck with 50% special lands, and 50% basic lands, and see how many lands you draw, and how many special lands you draw. If my theory is correct, the amount of special lands drawn should be greater then the number of basic lands. Especially for the first few cards in the deck, so just recording starting hands might show some really strange results.

Likelihood is pretty small, t=100 is plenty for good asymptotic results. GoboRab's data is more consistent with a slight positive trend than a negative one. I can calculate a one sided test later ho: trend < 0. But that test will have weak power, as the true trend is supposed to be 0. I can enforce a lower number, but even in V2TWA's data it wasn't very low, just statistically negative.

Your test is not allowed with the Duels deck builder. Sorry.

By itself? Decent. 100 games is a statistically negligible sample. But it's a start and as we pile on more data from more sources, we get closer to the real answer. It's just interesting in how it directly conflicts with VT2WA's data, which could mean a whole collection of things that we would be guessing at until we get more info.

EDIT: Or I'm completely wrong. It doesn't SEEM like a reasonable sample, but DJ knows this stuff better than I.

It depends on the data and its variance. So if we had wild fluctuations then 100 wouldn't be enough, but we don't so it is. This is fundamentally just a calculation of means, the power (think usefulness, or believability) of these tests is strong, so t doesn't have to be that high. The mathematical proof of this is simple, but you can also just trust me on this one or take a look at any stats book and notice how the t-test values become the z-test value around n=80. T=100 should generally be enough, for now. Full blown distribution analysis may prove more difficult, but answering the general question: does the shuffler at least give us results consistent with the land in the deck? should be no problem.

Ok, guys, after playing around 40 games with a 26 land deck with no deck manipulation outside of scry, my land problems dramatically decreased. As soon as I played a deck with evolving wilds, I started flooding again and getting screwy draws that didn't make any sense. It's quite possible that just having evolving wilds (and maybe other manipulators besides scry) in your deck, bugs out the shuffler, causing it to produce weird, anomalous draws. I'd also like to add that the card distribution seemed better with the 26 land deck too. (Not as much card clumping).

YAP. Just hit my first super flood in like 35 games. 9/12 cards in 24 land deck were lands.

Update: after switching decks yesterday, I've experienced multiple super floods, countless unplayable hands/mulligans, and multiple game crashes. ALL of this happened after I changed decks. Plus the lag is way more noticeable. Seriously, WHAT IS GOING ON WITH THIS GAME!?

Update: The game has become unplayable for probably 60% of my games no matter what's in my deck now. I'll get 0 lands, all lands, 4 lands with all 5+ spells, all removal and no creatures, all vehicles and no creatures, etc. Common sense says that DOESNT make any sense. What's the point of deck building if the shuffler just does what it wants? Odds, probability, and algorithms are just tossed out the window in this game. The only reason I'm still playing this game is because I love the strategy and creativity of deck building, but this game is an UTTER joke.