Friday, June 5, 2020

The case against double elimination

The origins of double elimination

Double elimination began as a tournament format in American arcades in the 1990s. The first major "internet tournament" was called B3, and was the precursor to Evo as we know it today. Before B3, there were local arcade tournaments. B3 was the first touranment organized by the nascent fighting game community, by the players, for the players, where people were traveling from across the US and even from foreign countries.

I was part of the B3 planning group, and there were two major constraints that we had to deal with. 

The first was that there was a limited number of arcade machines. There was a large number of people going to show up, and only a handful of machines. We needed to find a format that would accommodate a large number of players, but a limited number of machines. We had negotiated the venue for a day, and the tournament had to finish in one day's play time.

The second constraint was respect for traveling players. We had players coming from all over the US and even abroad. We knew that players had come to play, and we wanted to allow players to play as many games as possible, win or lose.

We discussed a few options:

1) Round Robin
In round robin, every player plays against every other player. Great format for a small number of people. But impractical for a large number of contestants. With 64 players, we'd have 2,016 matches to run. Completely out of the question.

2) Single Elimination
Fast, efficient, and brutal. This is the preferred format of Japanese arcades, although we didn't know it at the time. For a tournament with 64 players, there would be only 63 matches played, a very manageable number. With that said, we felt that it would be deeply unfair for players that had traveled from across the world to potentially play one game and be done.

3) Double elimination.
A reasonable compromise given the constraints. Everyone gets to play at minimum two games. The major disadvantage is that the losers bracket is a long, long morass. All in all, up to 127 matches are played. Another disadvantage is that you can easily get eliminated by a friend that you play with all the time at your local events. Nobody wants to fly across the country only to play someone you play at home.

We ended up going with a hybrid system. We broke up the entrants into two groups of 32, ran qualifying pools, and then put the pool winners into a 32 man double elimination bracket. This ensured that everyone got at least two games in, while cutting down the number of matches to a manageable level. 

Even in the tournament, people found themselves matched up against friends from their region, and so the organizers came up with the idea of "floating" -- adjusting your match and place in the bracket on the fly to prevent some match ups. This was a tremendous amount of work, and the organizing team, led by Tony Cannon, did a heroic job.

Some reading:

B3 was a great success, and after it followed B2, B4, and a string of tournaments that became Evo as we know it today. The pools->double elimination system worked well, players understood it, and it became the standard.

Online vs Offline

Skilled players used to be a small and select group. In order to be good, you had to be a strong, motivated player. You had to have access to core group of strong and motivated players. The "B" tournaments showed us that while there were many people with talent all over the globe, that talent did not translate to tournament success. You had to have talented and a strong group of players that played regular games. And not just regular games, but regular competitive, cutthroat games. 

The rise of online gaming completely changed this dynamic. Online play meant that anyone with motivation could find games at any time. We saw an explosion of success, as single isolated players with no local groups could become tournament powerhouses thanks to regular play and online tournaments. We now see regular tournaments all over the globe -- USA/Canada, Europe, Mexico, Brazil, Latin America all have strong and regular regional online tournaments.

Most of these tournaments, out of historical precedence, have adopted the double elimination format. The double elimination format, which originally was a compromise made in order to accommodate a limited number of physical machines, has become the de facto standard because "that's they way it's always been".

There is a key difference with online tournaments, which is that there is no time constraint on the number of matches. Since each player can play directly with another, the time constraint becomes the number of rounds. This is an important difference, so it bears spelling out:

In a physical arcade with a limited number of machines (1-3), the key determiner of tournament length is the number of matches played. If you have twenty matches to play on one machine, each group of players needs to line up, wait their turn, and play their match. Assuming five minutes per match, those twenty matches will take take 100 minutes, or almost a couple of hours to run!

In an online tournament, those same twenty matches can all be run in parallel. Each person connects directly with their opponent, runs their game, and reports the result. In our example, the twenty matches, that would have taken hours on a physical machine, can all be run in five minutes online.

The difference is colossal. Minutes versus hours. The key constraint that had led to the adoption of double elimination as a tournament standard, has been is completely removed.

Tournament Formats

It's now worth going back to the drawing board, and thinking about what tournament options we have that are available to us for streamed, online tournaments. We have a few guidelines to work with:

* Timeliness The tournament format should be timely. It should run in a predictable amount of time. Streamers often have a time budget, and viewers have a set amount of time. A tournament should neither take an inordinate amount of time to run, nor should it be highly variable.

* Fairness. The tournament format should reward top players. We should expect to see highly skilled players at the top.

* Excitement. The tournament format should culminate in a grand finals. The streamer should be able to build audience excitement over the course of the tournament, following key players, important matches, and build a narrative and story with the audience.
 
* Understandability. The tournament format should be easy to understand for viewers. Viewers join and leave streams at any point in the tournament. They should be able to jump in, and quickly understand the state of the tournament to participate as a viewer.

* Inclusivity. The tournament should be rewarding for new players. They should feel included, welcome, and have fun, regardless of performance.

* Maximizes matches. Last but not least, the format should allow players to play as many matches as possible. Players come to tournaments to test their skill against opponents in a formal setting. As many competitive matches should be played as possible.

The last point is perhaps the most important one. The point of a tournament is not to "weed out" weaker players until one reigns supreme. The point of a tournament is to allow as many people as possible to play competitive games, and to determine the winner!

With these in mind, let's grade double elimination in the online format.

Timeliness: Not great. There are many rounds, and the number of rounds is variable depending on the number of players.
Fairness. Overall pretty fair. Sometimes a strong player will get eliminated by two top players early -- usually if a top player loses their first round, they go and wreack havoc in the losers bracket. But other than that, the results are fairly consistent.
Excitement: The grand finals build up is good. Once the tournament is narrowed to the top 8, it's a good watch.
Understandability: This gets a poor grade. The loses bracket is hard to follow. As a spectator, you're more or less relegated to watching chosen matches from the streamer until the top 8. You constantly have to check the bracket to understand what's going on.
Inclusivity. Another low grade. A full quarter of the players will get eliminated 0-2. That's just not fun. It's not even about losing twice, it's losing twice and not getting to play anymore that's unfun.
Maximizes matches. Very poor. A small number of players get to play many matches, but half the players play only 2 or 3 matches, regardless of tournament size. Not good.


A grade of 2/6. We can surely do better than that! 

Let's consider another format : Swiss into single elim top 8. This is the format that is used in Chess, Magic: the Gathering, and many other events. 

In Swiss, everyone plays a set number of rounds (let's say 8). In each round, each player is matched up against an opponent with the same score. So if you are 0-3 after three rounds, your next opponent will also be 0-3. If you're 2-1, your opponent will also be 2-1. 

After eight rounds of swiss, the top eight players are taken into a single elimination bracket, and a final winner is crowned.

Let's grade this:

Timeliness: Excellent. The tournament always takes the same amount of time, regardless of the number of players. This is because there are always a set number of rounds. The tournament organizer can make the tournament longer by increasing the number of rounds. The top 8 always takes the same amount of time as well, and goes by very fast in single elim.
Fairness. Very fair. Everyone gets to play eight full rounds, so even if you get a bad draw early and lose twice in a row, you can climb out of the bottom bracket by winning your next 6 and make it to the top 8.
Excitement: The swiss part is great, because the streamer can pick a feature match each round to showcase. You always have the full roster of players to choose from. The top 8 format is exciting, with strong players, and an undisputed winner.
Understandability: Much easier to follow mid stream. If someone starts watching mid tournament, all they need to know is what round it is
Inclusivity. Great for new players. As a new player progresses in the tournament, they will face easier and easier opponents. No more getting crushed 0-2 and out.
Maximizes matches. Excellent. Everyone gets at least 8 matches. The very top players will get more than that, but only a few more. The top 8 takes three matches to complete, so the two finalists play 11 matches total.

This system beats double elimination in every dimension. It's a clear win. And, it's supported directly in challonge.

"All right, all right", I can hear you say. "That's all great in theory, but how does it work in reality?"

I'm glad you asked!
It's time to put double elimination to rest. All online tournaments should be Swiss.



Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)