New Ranking System

Everything dealing with the video game developed by Cyanide!
Fashbinder
Posts: 60
Joined: 21 March 2016, 12:28

Re: New Ranking System

Postby Fashbinder » 14 November 2016, 03:07

I've been told we can't since that involves calculations that require more than the at-the-second record of a person's W/D/L.
Bit of a shame, imagine if mike came up with some brilliant system* that people unanimously thought was the best and it revolutionised matchmaking for every head to head competition in any game/sport but it used more than wdl, we wouldn't get it :(

*(not one that gives a new player against a 20 year vet an equal chance of winning the game)

User avatar
VoodooMike
Posts: 1614
Joined: 14 July 2009, 07:44
Location: Calgary, Alberta
Contact:

Re: New Ranking System

Postby VoodooMike » 14 November 2016, 03:43

I know. I was talking specifically about this post, which was itself a counterexample for the probability-based comparison Veggente was talking about. I know the current formula doesn't use probability distributions, obviously.
But are you aware of why it's not relevant to seasonal ranking, and why things like TrueSkill are also not relevant to it? The way you're discussing it with Veggente, who doesn't even understand the concepts you're discussing with him, I'm not convinced you are.
That's not what I was thinking, actually.
Then why talk about it at all? Seasonal performance is not about using a sample of games to determine the overall ability of a team.. if we cared about that we wouldn't restrict ourselves to the performance during a given season. We'd end up with teams playing in the final tournament based on how they generally play, not how they performed during that season... so we'd have to wonder why we were bothering with seasons at all.
Your level of condescension is starting to irritate me mike, if you can't ditch your rude arrogance and insults, don't bother speaking to or about me.
Or what, you'll cry? In both instances where you and I have interacted in this thread it has been you who initiated it, and you who specifically made it a conflict. Maybe you need to work on stopping yourself from engaging in conflict with people you can't handle being in conflict with?
And if anyone thinks they can dissuade me from voicing my opinion by using snide comments and cheap put downs.. well think again, I ain't going to be bullied away from the discussion.
Awesome, keep up the good work. The same goes for me pointing out that your opinions are uneducated and ill-considered... especially when you voice those opinions toward me by name ;)
Calm down.
I'm perfectly calm - I know what I'm talking about and you don't... also, you have no direct influence on the future of CCL. I'm not sure why you think I'd be upset over you making erroneous statements. I'm mostly amused when someone goes on a rant and says a bunch of stuff that is totally out to lunch... and I like to point it out when they do, unless there's more fun to be had by not doing so.
But, if you want to compare the performance (measured simply as a win rate) of someone with 112 played games with the performance of someone with 30 played games, you are comparing theoretical performances.
Nope. You're comparing actual performances - that's what seasonal play is. You can certainly theorize that the performance during a season is not indicative of one coach's true skill, but that's irrelevant when ranking them for that season - we rank them on their time-limited performance.

To reduce the randomness associated with the outcome of games we need to set a minimum number of games being played before we take their limited performance seriously... or we end up with someone winning one game and being top ranked because of it. Other than that taking games played into account is just a preference - one that Focus/Cyanide opted for. I personally think the amount that games played contributes should be explicit and tightly controlled, but that's not how the current formula works.. which is why I don't like the current formula... but I don't inherently disagree with the idea of having games played contribute to ranking.
What I tried to say in my previous posts, and probably I wasn't able to (given your reaction), is: if you compare the theoretical performance of players, you should account for the fact that the central tendency of the player with 112 games is more likely to be close to his theoretical performance (or his "theoretical central tendency", surely not a good name) than the central tendency of the player with 30 games.
You can't "account for it" - that's just how probability distributions work. You set ONE confidence range and that's the range you'll have for everyone... if you try to change the range for different people you're being a crazy person who doesn't understand statistics.

Likewise, who the hell is trying to compare theoretical performance? This is a seasonal system! If team A that is normally a really good team, plays crappy during a season.. should they go to the playoffs anyway because we feel they're still good enough to be one of the playoff teams? No, we base it on that season's performance.
Because of that, a given win rate with a lot of games should be valued more than a slightly higher win rate with much less games.
Well, that's one opinion I suppose. Lucky for you the way you'd accomplish that is by making games played contribute to the ranking value... and that, for different reasons, is what already happens. Ta da, you already got what you wanted... so why are you arguing with dode?
*(not one that gives a new player against a 20 year vet an equal chance of winning the game)
As a 20 year vet you just want to win every game you play? That makes you sound like a douchebag single-player gamer, not a veteran multiplayer competitive gamer. What being a veteran player should do is get you ranked higher than a neophyte, and that can be done even under a system that lets you play that neophyte and both have to play your best to have a chance to win.
Friendly Reminder: Correlation does not equal Causation - tattoo it on the inside of your eyelids if it'll help.

User avatar
dode74
Posts: 7040
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 14 November 2016, 06:54

But are you aware of why it's not relevant to seasonal ranking, and why things like TrueSkill are also not relevant to it? The way you're discussing it with Veggente, who doesn't even understand the concepts you're discussing with him, I'm not convinced you are.
Yep. Seasonal ranking is about how you have actually have done rather than assessing how "good" you "really are".
Then why talk about it at all?
It was simply a counter-argument to his own (very flawed) version of a probabilistic argument.
Image

User avatar
Koadah
Posts: 1211
Joined: 08 April 2009, 16:17
Contact:

Re: New Ranking System

Postby Koadah » 14 November 2016, 08:15

Calm down.
Similarly, it doesn't matter where someone's theoretical performance is... for any given season of a sport you're looking at how they performed during that season which means the data you have is not a sample of games from that team, it is the sum total of the relevant data, meaning you have the full population of data and the resulting value is not a measure of central tendency it is the de facto final value.
But, if you want to compare the performance (measured simply as a win rate) of someone with 112 played games with the performance of someone with 30 played games, you are comparing theoretical performances. Any formula that discriminates between these two players is based on an hypothesis of what would have been the performance of the player with 30 games if he had played 112 games.

What I tried to say in my previous posts, and probably I wasn't able to (given your reaction), is: if you compare the theoretical performance of players, you should account for the fact that the central tendency of the player with 112 games is more likely to be close to his theoretical performance (or his "theoretical central tendency", surely not a good name) than the central tendency of the player with 30 games. You are more likely to outperform (or to underperform) on 30 games than on 112 games. However, if you play a lot of series of 30 games (or reroll when you lose 1 game in the first ten) there are very good chances that your best attempt will be the one in which you largely outperform your theoretical value (ofc we have not the absolute certitude of that...). Because of that, a given win rate with a lot of games should be valued more than a slightly higher win rate with much less games.
Is a team that is 50% at 112 games better than a team that is 50% at 30 games? I don't think so. Should a team that is 40% at 112 be ranked lower than a team that 40% after 30?

The coach of the 30 game team may have only played 30 games all season. The coach of the 12 game team may have played 224. It is the team that is being ranked not the coach.

I'd call 100% at 112 better than 100% at 30 games though. ;)
CaRBB

Miraskadu
Posts: 231
Joined: 08 March 2016, 03:05

Re: New Ranking System

Postby Miraskadu » 14 November 2016, 09:16

Is a team that is 50% at 112 games better than a team that is 50% at 30 games? I don't think so. Should a team that is 40% at 112 be ranked lower than a team that 40% after 30?

The coach of the 30 game team may have only played 30 games all season. The coach of the 12 game team may have played 224. It is the team that is being ranked not the coach.

I'd call 100% at 112 better than 100% at 30 games though. ;)
I would say the 50% win rate at 30 games is def worth than the 50% at 112, the assumption that the team (not coach on purpose) will be able to hold the same win rate for another 82 is a rather daunting one. For the same reasoning you count a 100% win rate at 112 games better than a 100% win rate at 30 games.

For both the CI is larger on the 30 games played. Just that it is pretty obvious that at 100% the upper half of the interval has to be cut of since it becomes non-sensical (winrate above 100%, I would love that :shock: )

You can make the argument that the whole formula for season 1 is valuating more the amount of games played to highly, and even dode came around to that conclusion after season 1. And is adjusting it the other way for season 2, according to him.

The new formula might be shooting to low, hence people will try to go for perfect records again. But then it can be changed into season 3, and at some point we should hit a sweet spot or close to it, for balance between running for the perfect record and just amassing games. And with 6 weeks seasons, the overturn won't be to bad. So maybe half a year to a year to find something that works.

All I would ask if it would be possible is to give people easier access to the league data, so we can actually have a discussion on somehow equal footing here. Just looking at page 1 or 2 of the leaderboard and trying to evaluate the ranking system is not a good approach, but with no access to data it is all the 'normal' people have. It is not that I'm not trusting dode posts the overview data, but it is hard to argue statistics and the like if one person has access to the data and the other person doesn't.

User avatar
dode74
Posts: 7040
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 14 November 2016, 09:48

All I would ask if it would be possible is to give people easier access to the league data
I'm working on it. Learning PHP first ;)
Image

Miraskadu
Posts: 231
Joined: 08 March 2016, 03:05

Re: New Ranking System

Postby Miraskadu » 14 November 2016, 10:01

All I would ask if it would be possible is to give people easier access to the league data
I'm working on it. Learning PHP first ;)
Ok, did actually not expect that answer. :shock:

But sounds great. And thanks for doing the work to make it possible.

So when can we expect it, tomorrow? :mrgreen: :mrgreen: :mrgreen:

Fashbinder
Posts: 60
Joined: 21 March 2016, 12:28

Re: New Ranking System

Postby Fashbinder » 14 November 2016, 11:20

As a 20 year vet you just want to win every game you play? That makes you sound like a douchebag single-player gamer, not a veteran multiplayer competitive gamer.
Yes every game i play i want to win in the same way that tom brady wants to win every football game. I think its safe to say those on the competitive ladder are also quing for a match with the intention of winning.
What being a veteran player should do is get you ranked higher than a neophyte, and that can be done even under a system that lets you play that neophyte and both have to play your best to have a chance to win.
I expect to lose to magnus carlson at chess, federer at tennis and phelps at swimming but im sure if there was only one competition in the respective field that the could play in on a daily basis they'd be pretty disappointed if i could beat them 50% of the time as it would make their hardwork and training irrelevant. I think it'd be fantastic in an open/new players ladder for people who play casually but would destroy competitive blood bowl.

User avatar
dode74
Posts: 7040
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 14 November 2016, 12:59

Ok, did actually not expect that answer. :shock:

But sounds great. And thanks for doing the work to make it possible.

So when can we expect it, tomorrow? :mrgreen: :mrgreen: :mrgreen:
Haha, might be a little longer than that! While I do have a fair amount of free time I also have a real life, like everyone else.

But I'm not averse to giving people data access at all, it's more a matter of what and how, and how often it needs to be updated. Since I have no intention of doing this the hard way I don't regularly update it and upload it yet. What I have in mind will hopefully make all that easy once I've put in the groundwork.
Image

Miraskadu
Posts: 231
Joined: 08 March 2016, 03:05

Re: New Ranking System

Postby Miraskadu » 14 November 2016, 13:24

Haha, might be a little longer than that! While I do have a fair amount of free time I also have a real life, like everyone else.
Real life what is that? you mean the game with great graphics but shit game play? :mrgreen:
As long it is not the 'old' cyanide soon(TM) all is fine.
But I'm not averse to giving people data access at all, it's more a matter of what and how, and how often it needs to be updated. Since I have no intention of doing this the hard way I don't regularly update it and upload it yet. What I have in mind will hopefully make all that easy once I've put in the groundwork.
Expected more an answer along the lines of, we can't do that due to privacy issues, ie caoch names, log in data etc., my hope would be just to have access to the raw data, even with coach name replaced, and even if it is just with the first coach gets is 1 the second 2 and so forth, and maybe do something similar for the team names. I don't see it as needed for team names, since it doesn't give you much valuable data for getting into an account, while the coach name is linked to the BB2 account.

But by replacing the individual coach names with unique numbers it still allows to parse the data for certain coaches with interesting records, while keeping the identity of everyone playing bloodbowl intact.

On how up to date the data should be, no good idea. End of season 1 data for both COL and CCL, would be a good starting point, and should give people enough to work with in the first go.


Return to “General”

Who is online

Users browsing this forum: No registered users and 7 guests

cron