In a previous post, we shared our decision to introduce a ship limitation system for Season 14 of Clan Battles and talked about how it impacted gameplay. Now, we want to go into detail about how and why we decide to limit certain ships.
Why implement ship limitations?
The ship limitation system was first introduced in Update 0.10.1. with the purpose of being able to shake up the established “meta” during a Season and introduce more variety to team compositions.
An established “meta” during a Clan Battles season often translates into teams almost exclusively gravitating towards a select few ships and strategies that have most often proven to secure wins. This phenomenon causes most battles to play out very similarly to each other throughout the season. It can sometimes happen, however, that a go-to meta is not firmly established from the very start of a season, and two or more different ship + strategy combinations can be spotted on the battlefield before the final verdict on the “ultimate” option is settled upon by players. This initial uncertainty helps to keep Clan Battles interesting, so in an effort to promote it, we introduced a ship limiting system to allow us to make live adjustments to what ships can and can’t be used as the meta evolves, thereby forcing Clans to adapt on the fly with new, creative approaches.
How we use the ship limiting system
How do we keep track of the Clan Battles meta and how do we draw conclusions from it?
Statistics
Firstly, and most importantly, we analyze ship statistics. This is our main tool when it comes to implementing limits to Clan Battles.
As a rule, we only start working with statistics after the first two weeks of a new season have passed. By that time, we can be relatively sure that a meta has already formed and we can start poring over the data and making decisions on specific ships.
After implementing limits to the live server, we immediately go back to the statistics. If we see that teams react by using new tactics and more varied ship selections, then we can consider the chosen limitation a success. If we see that there is no significant impact from the imposed limit and the meta settles down again, then we start looking into additional restrictions.
Win rate and popularity
Win rate and popularity are the two most important statistics we look at for these types of decisions. Our approach is quite simple: if a ship is turning out to be both very widely used and effective, we limit it. This applies to most if not all Clan Battles Leagues. If a ship is only proving very popular and efficient in one League, then we most likely will not limit it, as it does not significantly affect the big picture of the Season. However, if this is the case, we keep a close eye on that particular ship’s performance and only introduce limits if the trend spreads to the other Leagues.
It's important to note that we only consider popularity and effectiveness together: if a ship is very popular but does not have an abnormally large win rate, or vice versa, we will not limit it. Let’s look at an example:
Let’s say ships A and B are standing out in our statistical analysis for two different reasons:
- Ship A appears in 90% of battles and has a 48.9% win rate
- Ship B appears in 10% of battles and has a 60% win rate
Judging by its win rate, Ship A is not giving teams a significant advantage in battle, despite its popularity. This most likely means that players will eventually migrate to different options. If we choose to limit Ship A based on its high popularity alone, its players will probably not bother organically searching for new, stronger options, and instead directly settle for Ship B. In this case, limiting Ship A would simply hasten the stagnation of the meta since Ship B will not have an effective alternative. Likewise, it makes little sense to limit Ship B: despite its high win rate, only a small number of battles were played on it, so it's not having a significant impact on the overall picture of the season.
In such a situation, even if the meta seems to be settled, it is likely to change soon, so we do not implement any limits. Where we DO begin to see a problem is in this other example:
- Ship A appears in 10% of battles and has a 40% win rate
- Ship B appears in 90% of battles and has a 51.1% win rate
In this case, a meta has clearly been established with little chance of it organically changing. Ship B outperforms the alternative in effectiveness, so her popularity, that is already very high, is unlikely to drop. Therefore, we will likely choose to impose a limit on Ship B and encourage teams to develop new strategies with other ships.
These are, of course, simplified examples, since there are always going to be more than just 2 ships to choose from, and there may be several alternatives to them as well. However, this example is an effective demonstration of our approach to ship limits in general.
Other tools
Although statistics are very accurate and allow us to quickly find even minor deviations in the season’s meta, our scope when working on deciding on ship limitations is broader. We play Clan Battles ourselves, we watch replays and streams, and we analyze player feedback from various platforms, such as regional forums, YouTube, or Twitch. Feedback from players competing in the top leagues — Hurricane and Typhoon — is especially valuable, as they are some of the most hardcore and dedicated World of Warships players.
It's important for us to know how the season is going, not only in terms of the raw, emotionless data. Perception, as well as expert assessment of the season from both our team and the wider Clan Battles playerbase, are essential considerations before any final decision is made. Doing this allows us to either confirm or question the conclusions drawn from our statistical analysis. For example, if data shows that a ship is becoming too popular and effective, but our own game experience and that of our players show that this ship’s presence or absence in battle does not give a significant tactical advantage, we will not immediately restrict the ship and will continue to monitor it, as there is a chance that its popularity and efficiency statistics will return to normal levels on their own.
Another important tool is the survey based on the results of the previous Clan Battles Season. It is filled in by randomly selected players who participated in the respective season. This survey is the final assessment of what players liked about the season, what they didn't like, and what could be improved. This is more of a long-term indicator, which can allow us to pre-determine restrictions on certain ships before a new season begins. This was the case with Graf Spee in the 14th season of Clan Battles, which we immediately banned from the start based on the info collected about the 10th season. In addition, the survey helps us come up with ways to improve Clan Battles on a larger scale by introducing new functionalities. For example, we introduced Mercenaries, the cross-server matchmaker, and the system of ship limits itself largely based on the results of Clan Battles surveys.
Additional details
When is a win rate too high?
While the popularity of a ship is a relatively straightforward indicator, its win rate is more nuanced. We do not always evaluate it based solely on its absolute value, but other factors are also important to consider. Let's look at a few examples:
1) Some ships have a 100% win rate, but only a few clans choose them.
Yes, this is the highest possible figure, but if a ship has it, it’s only because that ship in particular has been taken into VERY few battles throughout the season, so it is therefore statistically insignificant.
2) A ship has a 55% win rate and it is chosen by 15% of clans in total from all Leagues.
We already consider this win rate to be high, since the number of clans in the sample is quite large, and the deviation from the standard of 50% is noticeable.
3) A ship is chosen by 90% of clans in total from all Leagues, and it has 50.5% of victories.
Even such a seemingly insignificant deviation from 50% is already considered high. In this case, the vast majority of battles in the season will see this ship on both teams, but precisely because of this, the outcome of the battles will not affect the ship’s overall win rate — in one team, the player on this ship will win, and on the opposite team, the player on the same ship will lose, so the statistic will tend to drift toward 50%. As small as it is, the deviation from 50% is affected by a very small number of battles, where the ship is only in one of the teams and, accordingly, provides it with an advantage, which indicates its excessive effectiveness.
Why are some ships limited in number, while others are completely banned?
When deciding whether to limit the number of a ship or completely prohibit it from going into battle, we use the following principle: if a ship is overly popular and efficient when it is alone in its team, then it will be banned, because limiting the number in this case will clearly not fix the problem. These ships include all battleships and aircraft carriers, since ships of these classes are most often alone, with a maximum of two in a team. Another suitable example is Graf Spee in the 10th season of Clan Battles. This ship was too effective and popular not only in groups, but also when alone. Therefore, with the advent of the system of ship limitaions at the start of the 14th season, which was also on Tier VI ships, Graf Spee was completely banned at the start of the season.
There are also ships that are overly efficient and popular in groups of two or more of the same ships in a team. A good example occurred in the 12th season of Clan Battles, when it became very popular to stack three or four Mogadors in a team. In this case, we do not pursue the goal of completely banning these ships, but just limiting their number. Usually, the win rates of these ships after the introduction of restrictions decreases to acceptable values, and individually they are no longer as effective, and, accordingly, not many clans continue to choose them. However, if the ship continues to show a high percentage of victories even alone, we can completely ban it with the next wave of restrictions.
Why did the current season repeat the format of the previous one without new restrictions?
The 16th season of Clan Battles began in Update 0.11.1, taking place in the same format as the previous one.
This is due to the fact that season 15, as shown by the survey data on Clan Battles, turned out to be one of the most successful in terms of popularity and other metrics. We decided to hold the next season in the same format and evaluate whether it will be just as successful. Moreover, it will help us collect more statistics on Clan Battles on Tier X ships — the most popular Tier for this battle type. The next season, regardless of the performance of the 16th, will be played in a different format.
At the start of the season, we also did not introduce any additional restrictions on the most popular ships in Season 15: for example, Ohio, Kremlin or Smaland. As statistics showed, these ships, despite their popularity, were not overly efficient. Their presence or absence in the teams did not have a significant impact on the win rate of these teams. This suggests that the most popular ship spread of the season was not overly effective, and teams using different ships and tactics were not behind in terms of wins. Therefore, we decided not to limit these ships with the start of the 16th season of Clan Battles.
Additionally, we want to talk about Petropavlovsk. Last season, as the statistics showed, the ship was quite popular and effective. However, according to player feedback and our own game experience, in comparison with other ships, it did not stand out in terms of perception. Therefore, we were in no hurry to ban Petropavlovsk, but continued to monitor it. This season, the stats and player feedback showed that the ship had become almost indispensable for most teams — her popularity was close to 100% in all leagues, and she was still quite effective. Therefore, as announced on the development blog, we decided to ban the ship.
We continue to keep an eye on other ships that were mentioned as well, but currently, they have no noticeable deviations in terms of popularity and efficiency. If during the course of the season we decide to introduce additional restrictions, we will definitely announce them on the Dev Blog.