New Ranking System

Everything dealing with the video game developed by Cyanide!
Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 13 November 2016, 12:51

I've not assumed the centroid is the representation of the player at all, simply pointed out what the distribution is based on the sample size. The numbers given show the range in which the real value sits to 95CI. The central tendency is based on the data itself and the magnitude of the uncertainty is based on how much data we have for each player.
When we treat it like that the 95CI range (i.e. 1.96 z-score) for the 112 game player is 78.15-91.48 and the range for the 30 game player is 76.8-99.8. The 30-game player is more likely than less likely to be a better player than the 112 game player under that model.
You can't claim what you claim in quote 2 if you don't assume that the central tendencies used to compute the confidence intervals are comparable. And they are not comparable (without any correction) because one is extrapolated from 30 games and the other from 112. You completely ignores the fact that the win rate of the player with 30 games is more likely to NOT be his central tendency than the win rate of the player with 112 games. Your calculations are right, but your inferences are wrong.
How have you come to that conclusion? Your assumption is that they continue the win streak at the same rate, but that assumption is flawed in that it doesn't account for its own uncertainty (which is what TrueSkill does, conservatively). It's 95CI to be within that distribution for each player. If you look at the 90CI ranges then the lower (left) bound of the range is higher for the 30-game player than for the 112-game player.
I work with the idea that players have a latent win rate (the measure of their real skill level) and that their "empirical" win rate is a reflection of this latent win rate. The higher the number of games played, the more likely that the "empirical" win rate is equal to the latent win rate (i.e. the skill of the player).
Furthermore, there needs to be some sort of incentive for people playing fewer games to allow them to at least try to compete with the higher games-played players, and if that means a devaluation of games played in order to incentivise play at the lower end of the scale then so be it.
Then the model that fairly ranks players independently from the number of games that they played also answer to the need to give some sort of incentive for people playing fewer games. I'm fine with that, we can end here.

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 13 November 2016, 12:59

You can't claim what you claim in quote 2 if you don't assume that the central tendencies used to compute the confidence intervals are comparable. And they are not comparable (without any correction) because one is extrapolated from 30 games and the other from 112.
That's why the distributions are emphasised rather than the tendencies. The size of the sample determines the magnitude of the distribution. I am not comparing central tendencies but comparing distributions.
You completely ignores the fact that the win rate of the player with 30 games is more likely to NOT be his central tendency than the win rate of the player with 112 games. Your calculations are right, but your inferences are wrong.
No, I'm including that. If you account for single tail probabilities and use a z-score of 1.285 for a 90CI, for example, the lower end of the range for 30 games is higher than for 112 games, meaning that there's a 90% chance (single tail) 30-game is better than ranking x and a 90% chance 112-game is better than ranking y, where x > y. Therefore 30-game is more likely to be better than 112-game under that model.
I work with the idea that players have a latent win rate (the measure of their real skill level) and that their real win rate is a reflection of this latent win rate. The higher the number of games played, the more likely that the real win rate is equal to the latent win rate (i.e. the skill of the player).
I agree with that statement, but have no idea how you are concluding that 30-game is more likely to be further to the right. The distributions I gave were dependent on games played.
Image

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 13 November 2016, 13:15

That's why the distributions are emphasised rather than the tendencies. The size of the sample determines the magnitude of the distribution. I am not comparing central tendencies but comparing distributions.
The values of yours 0.05 and 0.95 intervals depend on the value of the central tendencies, you have to start from somewhere! If you want to know the central tendency of my win rate, would you rather measure it over 30 games or over 100 games? Why?
Last edited by Veggente on 13 November 2016, 13:21, edited 2 times in total.

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 13 November 2016, 13:20

The values of yours 0.05 and 0.95 intervals depend on the value of the central tendencies, you have to start from somewhere! If you want to know the central tendency of my win rate, would you rather measure it over 30 games or over 100 games? Why?
More games gives a smaller distribution. That's what I've been saying. If the distributions I'd given were the same size you'd have a point, but they are not. The 30-game distribution is larger than the 112-game distribution precisely because the sample size is smaller. The recorded win% is used as a central tendency to give a range within which your real performance value lies, that range decreasing in size (and adjusting the position of the central tendency) as more games are played.
Image

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 13 November 2016, 13:21

More games gives a smaller distribution. That's what I've been saying. If the distributions I'd given were the same size you'd have a point, but they are not. The 30-game distribution is larger than the 112-game distribution precisely because the sample size is smaller. The recorded win% is used as a central tendency to give a range within which your real performance value lies, that range decreasing in size (and adjusting the position of the central tendency) as more games are played.
You should account for both the variance around the tendency and the variance of your indirect measure of the tendency.

Cripple Cup now, out for 7 hours :)

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 13 November 2016, 15:55

Which you do by choosing an appropriate confidence interval...

Have fun ;)
Image

User avatar
VoodooMike
Posts: 1614
Joined: 14 July 2009, 07:44
Location: Calgary, Alberta
Contact:

Re: New Ranking System

Postby VoodooMike » 13 November 2016, 21:06

You can waffle on for paragraph after paragraph about this and that and tennis matches, I just have to list up a few numbers to show that what you are trying to defend is batshit crazy.
Except that you don't have anything... you're throwing around numbers that have no meaning without being given specific context and claiming they make a point. Is the point that you don't understand math? You made that point earlier by suggesting a linear, cumulative system that would (if I can suss out what you're bitching about) make an even more lopsided ranking than the one you're complaining about.

Not that you'll read this far, but had you actually read my posts you'd find that I don't like the formula being used... at all. What I argue for is the concept of using games played as a modifier on the general record of people in the ranking. I also try to get it through your thick skull that losses giving points is totally irrelevant mathematically, what matters is the numeric relationship between the points for wins, draws, and losses (and any other conditions like concession wins and concession losses, that may be added). Over and over you seem unable to wrap your head around that.

The reason your mind gets blown so easily is that we're running too much intellectual electricity through too small a machine. Like putting an iPod directly on the power grid.
The win rate of the player with 30 games is more likely to be far on the right tail of the probability distribution than the win rate of the player with 112 games.
That's not true at all and shows a pretty disctinct lack of understanding of probability distributions. We don't know where on the distribution the true value falls, we simply know the probability that it falls within certain ranges... that's the entire point of a probability distribution. The value we have is most likely to fall exactly in the center, in both cases, because it is the mean of the observed values, and the mean of our calculated distribution is, y'know, the mean of observed values.

Everything you and Dode are talking about, for the most part, is total shite.. there's no use of probability distributions in this system, and I've been told we can't since that involves calculations that require more than the at-the-second record of a person's W/D/L.

Similarly, it doesn't matter where someone's theoretical performance is... for any given season of a sport you're looking at how they performed during that season which means the data you have is not a sample of games from that team, it is the sum total of the relevant data, meaning you have the full population of data and the resulting value is not a measure of central tendency it is the de facto final value.. no CIs or probabilities.. it is exactly that.
You can't claim what you claim in quote 2 if you don't assume that the central tendencies used to compute the confidence intervals are comparable. And they are not comparable (without any correction) because one is extrapolated from 30 games and the other from 112.
Sure you can. That's what confidence intervals are, and they take into account sample sizes. The measure of central tendency is fine for everything - it's the mean value, and it makes up the middle of the distribution if you're treating it like a sample, which I see no reason to be doing. The mean is the mean even if you have 30, 112, or 20,000,000 values. Your CI range will just shrink as your sample size increases.
That's why the distributions are emphasised rather than the tendencies. The size of the sample determines the magnitude of the distribution. I am not comparing central tendencies but comparing distributions.
Not in your ranking formula you're not. Not in ANY of the ranking formulas discussed for CCL, in fact. The only place you'll see distributions being used for ranking is in that ranking system I proposed for full TVPlus back in the days of old. Likewise, I don't see there being a good reason to do any of that for seasonal play - you'd need to give a good reason for doing so rather than explaining why you're not going to use seasonal performance as your measure of ranking for a season and rather plan on using theoretical true performance only sampled by a season for ranking for a season.
The values of yours 0.05 and 0.95 intervals depend on the value of the central tendencies, you have to start from somewhere! If you want to know the central tendency of my win rate, would you rather measure it over 30 games or over 100 games? Why?
It absolutely doesn't matter... you really, really don't get what a measure of central tendency is but seem to say the words over and over. It is the MEAN of observed values, and because of that it doesnt matter how many games you've played - that only affects the confidence intervals.

The number of datapoints in a sample does not affect the mean, it affects the CI.. the "spread" of the distribution. You're banging your head against the wrong wall here, champ.
You should account for both the variance around the tendency and the variance of your indirect measure of the tendency.
The quoted statement is meaningless drivel coming from someone who doesn't understand basic statistics.... and given the things dode has been posting during the last page of this thread, I can only assume he's drunk.
Friendly Reminder: Correlation does not equal Causation - tattoo it on the inside of your eyelids if it'll help.

User avatar
dode74
Posts: 7041
Joined: 11 December 2008, 11:18
Location: Nr. Reading, UK
Contact:

Re: New Ranking System

Postby dode74 » 13 November 2016, 21:44

Not in your ranking formula you're not. Not in ANY of the ranking formulas discussed for CCL, in fact.
I know. I was talking specifically about this post, which was itself a counterexample for the probability-based comparison Veggente was talking about. I know the current formula doesn't use probability distributions, obviously.
you'd need to give a good reason for doing so rather than explaining why you're not going to use seasonal performance as your measure of ranking for a season and rather plan on using theoretical true performance only sampled by a season for ranking for a season.
That's not what I was thinking, actually.
Image

User avatar
Scram Lyche
Posts: 143
Joined: 22 July 2015, 10:04

Re: New Ranking System

Postby Scram Lyche » 13 November 2016, 22:20

Your level of condescension is starting to irritate me mike, if you can't ditch your rude arrogance and insults, don't bother speaking to or about me.

My stance on this remains. The leaderboards are favouring those with 30 hours a week to Bloodbowl. And if anyone thinks they can dissuade me from voicing my opinion by using snide comments and cheap put downs.. well think again, I ain't going to be bullied away from the discussion.

Talking about a game, this ain't personal, so don't go there.

Veggente
Posts: 21
Joined: 10 July 2016, 23:31

Re: New Ranking System

Postby Veggente » 13 November 2016, 22:59

Calm down.
Similarly, it doesn't matter where someone's theoretical performance is... for any given season of a sport you're looking at how they performed during that season which means the data you have is not a sample of games from that team, it is the sum total of the relevant data, meaning you have the full population of data and the resulting value is not a measure of central tendency it is the de facto final value.
But, if you want to compare the performance (measured simply as a win rate) of someone with 112 played games with the performance of someone with 30 played games, you are comparing theoretical performances. Any formula that discriminates between these two players is based on an hypothesis of what would have been the performance of the player with 30 games if he had played 112 games.

What I tried to say in my previous posts, and probably I wasn't able to (given your reaction), is: if you compare the theoretical performance of players, you should account for the fact that the central tendency of the player with 112 games is more likely to be close to his theoretical performance (or his "theoretical central tendency", surely not a good name) than the central tendency of the player with 30 games. You are more likely to outperform (or to underperform) on 30 games than on 112 games. However, if you play a lot of series of 30 games (or reroll when you lose 1 game in the first ten) there are very good chances that your best attempt will be the one in which you largely outperform your theoretical value (ofc we have not the absolute certitude of that...). Because of that, a given win rate with a lot of games should be valued more than a slightly higher win rate with much less games.


Return to “General”

Who is online

Users browsing this forum: No registered users and 7 guests

cron