New Ranking System

Everything dealing with the video game developed by Cyanide!
seriousjest
Posts: 485
Joined: 09 May 2016, 18:30
Facebook: seriousjest
Twitter: seriousjest
Contact:

Re: New Ranking System

Postby seriousjest » 08 November 2016, 14:30

I'm not trying to be sarcastic or otherwise aggressive in asking this; I am asking for real. To my understanding, the point of the Champ Ladder is to find the 16 best BB coaches to play off for the Championship, right? It's not just to have casual fun, try out a bunch of different teams, etc. That's what the Casual ladder is for, no?

So I don't understand this argument about people demanding the ability to play different teams. Only 1 of your teams will be able to qualify. If you are strapped for time, concentrate on your best one. I say this as someone who has multiple teams in the Champ Ladder on PS4, but I have had the free time to play all of those games. I didn't start a new team until the team I was presently playing was where I wanted it on the ladder.

Anyway, this is my viewpoint. I just don't understand the argument that we need to provide coaches with less time the ability to play a bunch of different teams and have them all be competitive. I invite someone to educate me as to what I'm missing, although I really don't want to get into a flame war.
PS4 Champion Cup co-admin, co-host of "Live from the ManCave!!!," and your favorite Blood Bowl coach's favorite Blood Bowl coach

Image

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 08 November 2016, 14:50

I mean that after around 60 games, if you have a win rate of 50% (i.e. if half of your future games are likely to be wins and half to be losses) you are likely to gain no points on long term independently from the number of games that you play (ofc variance may help you). This because on a given number of additional game played (10-20-30) you tend to lose with losses all the new points that you gain with wins.

If your win rate is, says, 55% (11 wins and 9 losses each 20 games) you are likely to gain the points of 2 wins each 20 games. Thus, a coach with a 55% win record (that is, imho, a poor record) may potentially grind the leaderboard. Nothing against Zippy, but it is the case of the first Dark Elf of the leaderboard (even if he had a much better record some weeks ago).

What I suggest, is that after at a certain number of games, coaches should need a much better win rate to expect gaining points in the leaderboard on the long run. For example, I wouldn't complaint that where I am (around 90 games) if I lose a game I need 2 wins (or even 3!) to return to the same amount of points than before the loss.

In conclusion, I was just suggesting to tweak the actual system in a way to make faster the diminution in returns of wins.

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 08 November 2016, 14:51

You have to cater to more than just the hardcore otherwise there's nobody for the hardcore to play against.

Also, the goal you stated isn't the sole goal. It also provides an environment where you can expect to finish a game you start along with a bunch of other reasons for people to play there.
Image

User avatar
VoodooMike
Posts: 1614
Joined: 14 July 2009, 07:44
Location: Calgary, Alberta
Contact:

Re: New Ranking System

Postby VoodooMike » 08 November 2016, 17:37

Well, ELO (or, on-the-fly system) can not happen then, so i guess it is a moot point. I would just say that, of course, it would be team based, you would have different ELO with every team. Coach rankings could be done by ELO by team !
Yeah, and I'm telling you that the point of ELO is to predict which of two players in a game is more likely to win, and how much more likely, based on comparing their ratings. OCC tracked ELO for the participating teams. The correlation between ELO differences and match outcomes was weaker than the correlation between TVPlus rating differences and match outcomes. That means TVPlus, which can be calculated on a total record rather than needing a per-match cumulative calculation, is a better metric for exactly what ELO is meant to do when it comes to Blood Bowl, and requires less effort to calculate. To that end, ELO is worthless to BB.
Hard cap system is great in my opinion, because the average player will have a goal to strive for. I am certain that there is enough data to see what number of games would be best. Too little, and we would have world cup ladder team rerolling bs. Too high, and you lose prospective players.
There is enough data, and the data says "low numbers" which makes it vulnerable to random variance if you try to accommodate the majority of participating coaches even playing with the fervor they did in season one. I wholeheartedly believe in a soft cap and very explicit contributions for further games played, but I think a hard cap would discourage play. More play = more stable and more accurate performance gauging... and we want it to be accurate. People who rode the variance to a record higher than they deserve won't risk more play, but if there's a benefit to more play people will see the benefit > cost so long as they believe their performance can be more or less maintained.
I see where you are coming from that the system that is not hardcapped "might" be better, if only for the illusion that you could somehow miraculously qualify if you do not play a lot. And, do not be fooled, it is an illusion.
No, you misunderstand the point. The point is that the more games someone plays the more accurate their performance record will be with the team they're playing... their performance record will regress toward their mean performance value. Having games played contribute to the ranking simply motivates people to establish a more long-term and thus more accurate performance record.

The key there is to have a benefit to playing more games, in terms of ranking, but not an overwhelming benefit such that playing more games completely trumps performance as a contributor to ranking. There are different ideas on how to do that, and it's an area where there continues to be debate and discussion..
Similarly, for those that insist for a (much) lower number of games. The lower the number of games, the higher the influence of variance on the leaderboard. World Cup had precisely the issue of rewarding those picking up 20 consecutive terrible opponents. And this is far from unlikely, my orcs in Open met 24 terrible players before their first loss! With the actual system, I'm quite sure that the win rate of the coaches with 60 or more games in the leaderboard reflects their real value.
This is all true. The problem is that the number of games being played, per coach, in the roughly two month season tends to be a lower number... lower than 20, even. It is true that variance will be an issue, but it is unlikely to be as much of an issue as it was in WCQ due to the concession limits. Personally, I think the best plan is to have a soft cap at around 30, but with the pre-30 curve set to get very close to maximum at around 20... such that the penalty to the performance at the point where season 1 averages is only around, say, 10% ... then have the 100 games after the soft cap contribute a bonus of up to 10% to the performance rating. That means every game helps your rating to a small degree.. but it encourages people to play enough to establish a sufficiently accurate rating.
Is the 'we' here significant? For quite a while I have noticed that Dode and Mike arent at each other quite like the old days. Is Mike actively helping out to improve the Cyanide offering?
I attribute that to Dode wholeheartedly embracing the same religion - the worship of the almighty data - and to his no longer being a forum moderator. Because we no longer disagree on facts, and we value facts over opinions, we find ourselves routinely in agreement (where subjective opinions aren't key, that is).

The "we" in question was "the people" but realistically there are only two numbers people in the community. Everyone else has plenty to say and is very interested in opining, but is unlikely to ever do any actual work on formulas or calculations... and certainly won't bother learning to do analysis.
To begin with I had assumed that the brave knight sent to slay the dragon had returned defeated after multiple failures and instead was trying to become more dragon-like himself, but there has also been a noticeable softening in said dragon recently, so I'm left wondering :)
I don't know which is the knight and which is the dragon in your analogy. I think everyone, with the possible exception of the FUMBBL die-hards, wants Cyanide to succeed with their Blood Bowl offerings. There's a difference between what people think of a product and what they hope it can or will be. To that end we're all trying to improve Cyanide's offering... we just don't all agree on how to do so.
Anyway, this is my viewpoint. I just don't understand the argument that we need to provide coaches with less time the ability to play a bunch of different teams and have them all be competitive. I invite someone to educate me as to what I'm missing, although I really don't want to get into a flame war.
I don't understand the argument either, frankly. If you want a shot at a high ranking - high enough to qualify for the season's final tournament, then pour your effort into playing a single team and playing it hard. If you're just in CCL to have fun then do whatever you want (except concede, of course).. play 3 teams... 10 teams... whatever. Don't worry about your ranking.

Be cautious that such heady concepts do not blow Scram Lyche's mind in the process, though.
Friendly Reminder: Correlation does not equal Causation - tattoo it on the inside of your eyelids if it'll help.

User avatar
Koadah
Posts: 1211
Joined: 08 April 2009, 16:17
Contact:

Re: New Ranking System

Postby Koadah » 09 November 2016, 00:24

Anyway, this is my viewpoint. I just don't understand the argument that we need to provide coaches with less time the ability to play a bunch of different teams and have them all be competitive. I invite someone to educate me as to what I'm missing, although I really don't want to get into a flame war.
I don't understand the argument either, frankly. If you want a shot at a high ranking - high enough to qualify for the season's final tournament, then pour your effort into playing a single team and playing it hard. If you're just in CCL to have fun then do whatever you want (except concede, of course).. play 3 teams... 10 teams... whatever. Don't worry about your ranking.

Be cautious that such heady concepts do not blow Scram Lyche's mind in the process, though.
The fewer coaches that you have in the league, the worse the match ups will be. There will also be longer waits for a match. Especially later in the season.

The less prolific/multi team coaches will see their teams grow more slowly than the prolific, single team guys. The teams may not get far before the season ends. That will get even worse when you make the seasons shorter. If there is nothing else to play for, they may as well just start their teams in Open Ladder.

20-30 games seems like a fair number. Whether you got there by heavy stats or gut feel. :twisted:
CaRBB

User avatar
VoodooMike
Posts: 1614
Joined: 14 July 2009, 07:44
Location: Calgary, Alberta
Contact:

Re: New Ranking System

Postby VoodooMike » 09 November 2016, 01:33

The fewer coaches that you have in the league, the worse the match ups will be. There will also be longer waits for a match. Especially later in the season.
This assumes that the only reason people play in CCL is the hopes of a high ranking rather than it being a superior play environment for serious coaches. One of two things will be true (or, more likely, a combination of the two):

1) Some coaches are not playing for ranking but rather are playing for an environment with far fewer concessions... in that case there's no reason we shouldn't see new teams later in the season, albeit fewer (since all teams are new at the start).

2) Serious coaches who are playing to rank or even to qualify for the tournament will be playing the same team over time, and to some degree will have similar development levels as other serious coaches, having started at roughly the same time, and only having a limited time to develop. Those teams get to face each other, and on reasonably even terms.
The less prolific/multi team coaches will see their teams grow more slowly than the prolific, single team guys. The teams may not get far before the season ends. That will get even worse when you make the seasons shorter. If there is nothing else to play for, they may as well just start their teams in Open Ladder.
The concession rate in CCL is dramatically lower than the concession rate in COL, and the concession rate is one of the major complaints about BB2. There's no "might as well have made the team in COL" when you can play that same team and actually complete games quite a bit more often in CCL... you end up in COL in the long run, but why not take the good games while you can?

Shorter seasons mean that playing one team for a season is less of a commitment for the people who are "altaholics", but still gives time to play to rank with a serious team. So long as it isn't compressed to the point that random variance is a larger factor than realistic performance, it sounds like it'd work well for everyone ;)
Friendly Reminder: Correlation does not equal Causation - tattoo it on the inside of your eyelids if it'll help.

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 12 November 2016, 21:30

I assumed that you would have had a fair idea by now how the season 2 formula would work.
I do. I just wasn't clear what you meant by "new system". There are a couple of options I'm looking at, and either 26-3-1 or 25-3-2 would be sufficient to beat 91-8-13. As I said, ANY coach would have to do very well to beat a win% of 84.8 with over 100 games played.
So a 88% win rate on 30 games will soon beat a 84.5% win rate on 110 games? Seems slightly broke to me. In my opinion it requires some calibration. Personally, I can eventually understand a 88% win rate on 30 games beating a 75% on 112 games (79-8-24), for example. The aim for me should be a system in which a coach with a 29-0-1 record hasn't an excessively large gap (but still a gap) between what he can gain and what he can lose on the leaderboard by playing an additional game. It is the only way to avoid "sitting on record" and "I lose, I reroll" situations.

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 12 November 2016, 21:45

So a 88% win rate on 30 games will soon beat a 84.5% win rate on 110 games? Seems slightly broke to me. In my opinion it requires some calibration. Personally, I can eventually understand a 88% win rate on 30 games beating a 75% on 112 games (79-8-24), for example. The aim for me should be a system in which a coach with a 29-0-1 record hasn't an excessively large gap (but still a gap) between what he can gain and what he can lose on the leaderboard by playing an additional game. It is the only way to avoid "sitting on record" and "I lose, I reroll" situations.
And this is the issue that is HIGHLY subjective (so "broke" doesn't apply): the value of playing more games vs playing to a very high standard. The previous (i.e. current) version overvalued playing games, and you are now claiming (on the basis of nothing more than a single datapoint) that the new version is too far the other way.
Image

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 12 November 2016, 22:02

The aim for me should be a system in which a coach with a 29-0-1 record hasn't an excessively large gap (but still a gap) between what he can gain and what he can lose on the leaderboard by playing an additional game. It is the only way to avoid "sitting on record" and "I lose, I reroll" situations.
I think you've misunderstood the dynamic here. What matters is not how much he gains or loses relative to his current score, but what he gains or loses relative to someone else's score.
Image

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 12 November 2016, 23:03

The aim for me should be a system in which a coach with a 29-0-1 record hasn't an excessively large gap (but still a gap) between what he can gain and what he can lose on the leaderboard by playing an additional game. It is the only way to avoid "sitting on record" and "I lose, I reroll" situations.
I think you've misunderstood the dynamic here. What matters is not how much he gains or loses relative to his current score, but what he gains or loses relative to someone else's score.
When you play in the leaderboard and want to maximize your chance to qualify, you don't watch "who I can catch?" but rather "I'm more likely to improve or worsen my actual rating by playing an additional game?". Atm, with 1 game lost = 1 game win in terms of points (this is what I observe in my position, around to 70-80 games played), there is no reason (apart your AV7 precious elfs risking to be killed) to don't play one additional game if you expect to have a chance to win the game higher than 50%. From here the grinding.

Now imagine a system in which win rate dominates. I sit on a 25-3-2 record. My actual win rate is 88.33. If I lose next game, my record drops to 85.5. If I lose, I will able to return to a 88.33 win rate only with a record of 32-3-3, i.e. with 7 more won games. With a loss costing me 7 wins, on a 25-3-2 record I'm likely to play one additional game only if I think to have a chance higher than 88% (7/(7+1)) to win next game. From here the stall on "lucky" records and the rerolls after losing a game.

I want also to point out that a player with a long term win rate of 80% has a 16% chance to obtain a 25-0-3 record or better (89%+ win rate) and a chance of 2% to obtain a 90-0-12 record or better (88%+ win rate). Thus, a higher win rate with x games isn't necessarily "playing at higher standard" than a lower win rate with 2x or 3x games. There is no way to have a rating system taking in consideration the higher variance generated by a lower number of games played?


Return to “General”

Who is online

Users browsing this forum: No registered users and 5 guests

cron