TinSoldier
New member
- Joined
- Jun 4, 2017
- Messages
- 1,568
Hello DomiNations fans! Joe “Muet” Grubb here with Big Huge Games. This spotlight covers a long awaited change to DomiNations: World War Matchmaking.
Why Change Matchmaking? What are the Problems We’re Trying to Solve?
The high level goal is twofold: Improve the competitive integrity of World War and improve the match quality.
Bear with me. I’m trying something new with this Spotlight; we’re going to go deep. We’re going to talk about how the old system worked and how the new system is attempting to solve problems. If that’s not appealing, there’s a tl;dr “what are the changes?” if you skip to the end. It’ll be clearly marked and I won’t fault you for skipping forward, after all you’ve got a Nation to run and World Wars to fight.
Didn’t skip forward? You sure? Cool. Let’s take a look at the problems in the current system and break down each of them:
Did you know that the matchmaking system creates a Rating of your base’s offensive and defensive capabilities? It does. Everything on your base contributes to these Ratings. However, the method that the old system (the current matching system on live) used to compare the final Ratings with other bases wasn’t a great predictor of win rate. There was some directional value in its ability to predict win rate. But, we knew this was an area where we could improve match quality.
That old system compared the absolute value of your Alliance’s Offensive/Defensive (O/D) Ratings with other potential Alliances. We ran some tests internally and discovered that when we match based on a percentage delta between those same Alliance’s O/D Ratings, we actually get a much stronger predictor for winning. So, we’re updating the system to compare Ratings by the percentage delta rather than absolute values.
Fun fact: If your Alliance has a ~10% higher Defensive Rating in the new system, your Alliance’s chances of winning are increased to 60% (Yes, defense matters. Start upgrading your neglected Catapults!).
Additional fun fact: The new system would not consider a 10% delta a “good” match. So don’t worry.
Sandbagging/Imbalance in Alliance Compositions
As with other team-based games, there is an ability to “carry” in DomiNations. If we only checked the percentage delta between two Alliance’s O/D Ratings, we may end up making the following match:
Alliance A: 5 Cold War, 5 Medieval
Alliance B: 1 Cold War with a mix of Enlightenment+.
Seem familiar? I’m sure all of us have been in Alliance B’s position before. The O/D Ratings may look fair, but this is not a fair fight; Alliance B is going to lose this War.
We don’t want to limit Alliances to bringing specific compositions to War. Sometimes you just don’t have enough players at close Ages. But we do want to address the sandbagging issue. So, we’re adding a second comparison to the Ratings formula. The new system now also compares the delta within an Alliance and tries to find other Alliances with similar deltas. The result? The system now knows that Alliance A and B are wildly inappropriate to match. It’ll find Alliance A another Alliance with a large Ratings delta in its players while protecting B from an unfair fight.
Ready for another fun fact?: If your Alliance is sandbagging in the new system, it may result in increased queue times as the system waits to find another sandbagging Alliance for you.
The Leaderboard was not Part of the Evaluation
The old system did not use Glory as part of its calculation. In order to preserve the competitive integrity of our Leaderboards, we felt that it was important to start using Glory. In order to do that, Glory gained/lost needed to be updated.
We had several discussions about potential directions to take Glory and what systems to use.
A system like Elo (initially popularized as a chess ranking system) provides a single rating that can be used to predict the winner between opponents. Since Glory is serving as our Leaderboards, using a system that would suggest the higher ranking Alliances are more likely to win made sense. Afterall, if you’re Rank 1 on the Leaderboards, you should have earned it. Elo also does some other great things for Glory like awarding you more Glory if you happen to beat an Alliance that is higher ranked than you. That was a tough War and you deserve it.
Elo is somewhat of a barebones rating system, however. Even chess has been modifying parts of the formula based on player ratings. So, we also considered more robust systems like Glicko-2, a more complicated evolution on the Elo formula that introduces a confidence interval. This confidence interval is a way for the system to determine how accurate it thinks an Alliance’s rating is. The interval can then modify points gained/lost in the system based on this confidence. For example, if your Alliance is 20-0 in War and the system says you have a rating of 18,000 Glory, what are the odds that your Alliance is probably better than 18,000? Probably high. Glicko-2 would take this into account and likely accelerate your subsequent Glory gained.
While more complex systems may have offered some extra bells and whistles, we ultimately decided that we could get most of what was important to DomiNations by using a modified version of Elo we’re calling DOM MMR (DomiNations Matchmaking Rating). It’s pretty close to standard Elo with some targeted modifications to K-factor and streaking.
It’s a bit of an oversimplification, but you can think of K-factor as a constant in the Elo formula that sets a maximum on the number of points that can be won or lost from a given match. There’s a handful of problems with this value being constant. Even chess has been modifying the K-factor of players based on their rating.
So what modifications are we making? DOM MMR will dynamically change an Alliance’s K-factor based on the following:
Think of it as a season reset.
Queue Functionality
The old queue would search the list of available Alliances at the moment your Alliance entered queue. If you were not matched, you were then put in the bucket for other Alliances to potentially pair with when they entered queue.
In theory this system sounds fine. And it worked okay for the most part. But, this created a rigid queue. Your potential pool of matches never expanded except when other Alliances entered the queue, regardless of how long you waited.
This is changing in the new system to better incorporate industry best practices of time vs. match quality. The result is a system that is constantly evaluating the entire pool of Alliances against each other. The longer you wait, the less restrictive the system will be.
Is this Match Fair to My Alliance?
Some of you that are Alliance leaders may have noticed that your Alliance would sometimes instantly get matched with what appeared to be a poor match. What gives? Why? To the forums! The matchmaking is awful! Well, not quite. This was because the old system didn’t have a concept of handshaking. It compared Ratings at the time your Alliance entered queue and if they were within an acceptable range of another Alliance, bam. Matched.
Moving forward, the new system will require the match to be acceptable from the perspective of both Alliances. This isn’t an in-game option or agreement that is made. Instead, you can think of it as the system viewing the potential match from both Alliance’s perspective to see if it thinks it is fair.
What is fair to my Alliance after being in queue for only 10 seconds may be different from what another Alliance is looking for after 5 minutes.
Sparse Queues
I know that this will be one of the least popular changes. I apologize in advance for killing your Alliance’s favorite queue. The reality is that DomiNations can’t—and has never truly been able to—support seven different queues. Think of it this way: Each queue added to DomiNations splits the player base. Each of those splits greatly reduces the number of potential matches your Alliance can make and reduces the quality of those matches. Theoretically, the most optimal system would be to force everyone into a single XvX queue. We can’t really do that and aren’t entertaining the idea. But four is better than seven.
We’re reducing the queues to the following:
10v10
20v20
30v30
50v50
We feel that this division of queues will flow more Alliances into more pools and increase the odds of finding your Alliance a better match. It should be noted that 50v50 is not and has never been a popular queue. But, we felt it was important to preserve 50v50 for community events like the Continental World Cup (https://continentalwar.wordpress.com/). If you find your Alliance jumping into 50v50 expect long wait times and extremely relaxed matching requirements.
Conclusion (“What are the Changes?” tl;dr is here!)
In short, this is a complete overhaul to World War matchmaking. Standard PvP battle matching is not being touched. The changes are as follows:
Thanks for reading!
- Muet
Why Change Matchmaking? What are the Problems We’re Trying to Solve?
The high level goal is twofold: Improve the competitive integrity of World War and improve the match quality.
Bear with me. I’m trying something new with this Spotlight; we’re going to go deep. We’re going to talk about how the old system worked and how the new system is attempting to solve problems. If that’s not appealing, there’s a tl;dr “what are the changes?” if you skip to the end. It’ll be clearly marked and I won’t fault you for skipping forward, after all you’ve got a Nation to run and World Wars to fight.
Didn’t skip forward? You sure? Cool. Let’s take a look at the problems in the current system and break down each of them:
- Method for comparing the Offensive and Defensive Ratings of a base
- Sandbagging/Imbalance in Alliance compositions
- The Leaderboard was not part of the evaluation
- Queue functionality
- Is this match fair to my Alliance?
- Sparse queues
Did you know that the matchmaking system creates a Rating of your base’s offensive and defensive capabilities? It does. Everything on your base contributes to these Ratings. However, the method that the old system (the current matching system on live) used to compare the final Ratings with other bases wasn’t a great predictor of win rate. There was some directional value in its ability to predict win rate. But, we knew this was an area where we could improve match quality.
That old system compared the absolute value of your Alliance’s Offensive/Defensive (O/D) Ratings with other potential Alliances. We ran some tests internally and discovered that when we match based on a percentage delta between those same Alliance’s O/D Ratings, we actually get a much stronger predictor for winning. So, we’re updating the system to compare Ratings by the percentage delta rather than absolute values.
Fun fact: If your Alliance has a ~10% higher Defensive Rating in the new system, your Alliance’s chances of winning are increased to 60% (Yes, defense matters. Start upgrading your neglected Catapults!).
Additional fun fact: The new system would not consider a 10% delta a “good” match. So don’t worry.
Sandbagging/Imbalance in Alliance Compositions
As with other team-based games, there is an ability to “carry” in DomiNations. If we only checked the percentage delta between two Alliance’s O/D Ratings, we may end up making the following match:
Alliance A: 5 Cold War, 5 Medieval
Alliance B: 1 Cold War with a mix of Enlightenment+.
Seem familiar? I’m sure all of us have been in Alliance B’s position before. The O/D Ratings may look fair, but this is not a fair fight; Alliance B is going to lose this War.
We don’t want to limit Alliances to bringing specific compositions to War. Sometimes you just don’t have enough players at close Ages. But we do want to address the sandbagging issue. So, we’re adding a second comparison to the Ratings formula. The new system now also compares the delta within an Alliance and tries to find other Alliances with similar deltas. The result? The system now knows that Alliance A and B are wildly inappropriate to match. It’ll find Alliance A another Alliance with a large Ratings delta in its players while protecting B from an unfair fight.
Ready for another fun fact?: If your Alliance is sandbagging in the new system, it may result in increased queue times as the system waits to find another sandbagging Alliance for you.
The Leaderboard was not Part of the Evaluation
The old system did not use Glory as part of its calculation. In order to preserve the competitive integrity of our Leaderboards, we felt that it was important to start using Glory. In order to do that, Glory gained/lost needed to be updated.
We had several discussions about potential directions to take Glory and what systems to use.
A system like Elo (initially popularized as a chess ranking system) provides a single rating that can be used to predict the winner between opponents. Since Glory is serving as our Leaderboards, using a system that would suggest the higher ranking Alliances are more likely to win made sense. Afterall, if you’re Rank 1 on the Leaderboards, you should have earned it. Elo also does some other great things for Glory like awarding you more Glory if you happen to beat an Alliance that is higher ranked than you. That was a tough War and you deserve it.
Elo is somewhat of a barebones rating system, however. Even chess has been modifying parts of the formula based on player ratings. So, we also considered more robust systems like Glicko-2, a more complicated evolution on the Elo formula that introduces a confidence interval. This confidence interval is a way for the system to determine how accurate it thinks an Alliance’s rating is. The interval can then modify points gained/lost in the system based on this confidence. For example, if your Alliance is 20-0 in War and the system says you have a rating of 18,000 Glory, what are the odds that your Alliance is probably better than 18,000? Probably high. Glicko-2 would take this into account and likely accelerate your subsequent Glory gained.
While more complex systems may have offered some extra bells and whistles, we ultimately decided that we could get most of what was important to DomiNations by using a modified version of Elo we’re calling DOM MMR (DomiNations Matchmaking Rating). It’s pretty close to standard Elo with some targeted modifications to K-factor and streaking.
It’s a bit of an oversimplification, but you can think of K-factor as a constant in the Elo formula that sets a maximum on the number of points that can be won or lost from a given match. There’s a handful of problems with this value being constant. Even chess has been modifying the K-factor of players based on their rating.
So what modifications are we making? DOM MMR will dynamically change an Alliance’s K-factor based on the following:
- New Alliance - New combatants in any ranking system are hard to properly evaluate. It is near impossible to properly evaluate a new Alliance with only 2 Wars compared to an Alliance that has 300 under their belt. Therefore, new Alliances will have a higher K-factor for their first several Wars. The result should help get sharks out of the kiddie pool and help new Alliances find a win faster.
- Streaking - Winning or losing several Wars in a row will also cause the system to modify your K-factor. Sometimes you’re just not as good as you used to be, you know? Or maybe your Alliance recruited some top-tier players and it’s time to step into the big leagues.
- High Glory Alliances - K-factor modifications sound great but K-factor can break down at the extreme ends of a ranking system. Therefore, Alliances with particularly high Glory will have their K-factor treated differently. The main difference is that streaking won’t apply here.
Think of it as a season reset.
Queue Functionality
The old queue would search the list of available Alliances at the moment your Alliance entered queue. If you were not matched, you were then put in the bucket for other Alliances to potentially pair with when they entered queue.
In theory this system sounds fine. And it worked okay for the most part. But, this created a rigid queue. Your potential pool of matches never expanded except when other Alliances entered the queue, regardless of how long you waited.
This is changing in the new system to better incorporate industry best practices of time vs. match quality. The result is a system that is constantly evaluating the entire pool of Alliances against each other. The longer you wait, the less restrictive the system will be.
Is this Match Fair to My Alliance?
Some of you that are Alliance leaders may have noticed that your Alliance would sometimes instantly get matched with what appeared to be a poor match. What gives? Why? To the forums! The matchmaking is awful! Well, not quite. This was because the old system didn’t have a concept of handshaking. It compared Ratings at the time your Alliance entered queue and if they were within an acceptable range of another Alliance, bam. Matched.
Moving forward, the new system will require the match to be acceptable from the perspective of both Alliances. This isn’t an in-game option or agreement that is made. Instead, you can think of it as the system viewing the potential match from both Alliance’s perspective to see if it thinks it is fair.
What is fair to my Alliance after being in queue for only 10 seconds may be different from what another Alliance is looking for after 5 minutes.
Sparse Queues
I know that this will be one of the least popular changes. I apologize in advance for killing your Alliance’s favorite queue. The reality is that DomiNations can’t—and has never truly been able to—support seven different queues. Think of it this way: Each queue added to DomiNations splits the player base. Each of those splits greatly reduces the number of potential matches your Alliance can make and reduces the quality of those matches. Theoretically, the most optimal system would be to force everyone into a single XvX queue. We can’t really do that and aren’t entertaining the idea. But four is better than seven.
We’re reducing the queues to the following:
10v10
20v20
30v30
50v50
We feel that this division of queues will flow more Alliances into more pools and increase the odds of finding your Alliance a better match. It should be noted that 50v50 is not and has never been a popular queue. But, we felt it was important to preserve 50v50 for community events like the Continental World Cup (https://continentalwar.wordpress.com/). If you find your Alliance jumping into 50v50 expect long wait times and extremely relaxed matching requirements.
Conclusion (“What are the Changes?” tl;dr is here!)
In short, this is a complete overhaul to World War matchmaking. Standard PvP battle matching is not being touched. The changes are as follows:
- The matching logic has changed how it compares base power between Alliances. The new logic is proving to be a strong predictor of win rate and should allow us to make more fair Wars.
- The matching logic now understands imbalances within an Alliance’s composition. It understands “sandbagging” and will try to find Alliances opponents with similar compositions.
- How Glory is calculated has been redesigned.
- All Alliances are being pulled back toward 12,000 Glory when the update goes live. The pull is relative based on your Alliance’s Glory and it shouldn’t affect the standings much. Think of it as a season reset.
- Reduced the number of queues from seven to four:
- 10v10, 20v20, 30v30 and 50v50
- The behind-the-scenes functionality of the queue itself is changing to constantly evaluate the entire pool of Alliances in queue.
- The longer you wait in the queue, the more relaxed your matching requirements become.
Thanks for reading!
- Muet