Below is a copy of an article / post I just wrote on BGG
---

New idea, want to try and write about it briefly, but can give more detail if you're interested.

Our initial idea for measuring how much skill and luck are in a game might be to measure a single variable, like "how often does the better player win". This will give us a result where skill and luck are just two sides of the same coin, we might say "this game is 60% skill and 40% luck" but it will always add to 100%.

The problem is that some games with equal amounts of luck don't involve equal amounts of skill (e.g. Chess vs Tic-Tac-Toe). So we want to be able to say that a game is high luck or low luck independently of whether it's high skill or low skill (i.e. all four combinations are possible).

ELO is just a measurement of how likely someone is to win, on a log scale - every 400 points difference makes someone ten times less likely to win. A Magic: the Gathering pro has an ELO of about 2000, so they will beat a beginner 90% of the time. A chess pro has an ELO of about 2800, so they will be beat a beginner 99.9% of the time. But this doesn't mean that the chess pro is more skilled than the Magic pro.

Rather, the maximum achievable ELO in a game is a measure of how much luck there is in the game.

This means we need to measure a different variable to determine how much skill there is in a game. We could look at the number of branches in the decision tree, but that assumes there aren't any shortcuts that allow players to easily eliminate lots of bad ideas at once. We could look at how many players there are at different skill levels, but some games will have many pros and few beginners while others will be the other way around. So I decided to go with something simpler: how much practice / study do you need to do to get good at the game?

The amount of hours it takes to get close to the maximum ELO is a measure of how much skill there is in the game.

But in any game there will always be some pros who will spend more hours than seems necessary trying to get that little bit of extra performance. So if we are to graph a player's ELO vs the amount of hours of practice, rather than looking at the total height or length of the curve, we should look at its shape - is it concave up or concave down?

In a high skill game a player will take a long time at the start before they even get half way to being good, but once they get good, there will still be lots of room to improve - more practice will continue to reward with a higher ELO. In a low skill game, a player can go up very quickly at first, but once they get halfway to good, spending more time practicing won't yield as much results.

Finally, a few examples. Android: Netrunner is a high luck and high skill game, while Hearthstone is a low luck and low skill game. Comparing these two is what inspired this thread in the first place. When you look at results alone, the two games seem similar because the two factors cancel each other out, so how do you prove that there is more diversity in the skill level of top Netrunner players than of top Hearthstone players?

Magic is high luck and low skill. Note that I mean compared to other PvP games. You might think a beginner only having a 10% chance to win against a pro is not much, but if you ever met someone who studied Carcasonne or Dominion heavily it'd probably be even less. PvE games by comparison can have much more randomness, like roguelikes that can either hand you powerful weapons or drop unbeatable bosses on your head.

This is part of what makes Magic addictive, because a beginner has a chance to win but can also rise in skill quickly. Chess is the opposite. I think if it was invented today no one would play it (just look at how Prismata is going). Who has the patience to spend hundreds of hours losing before getting good enough to maybe win a game these days?

Hearthstone being described as low luck feels wrong. Good article though. The end felt abrupt, like we were meant to go somewhere else.