I know that. That's what I was trying to get at by proxying win rates within certain ranges. There could easily be an effect (albeit a small one, or we'd see it in the aggregate numbers to a lesser extent), but in this data set it's not there. Any effect there may be is masked by the noise.

The point you just erased is actually a good one — you can't really run linear regression on this even with perfect data, as the win rates of more experienced players include some games played as less experienced players. I don't know the best way to take that out formally (or whether it's possible without ordered data for each player). But my intuitive understanding of it as this point is captured above: there may be a weak effect in the 15 to 30 range, and thereafter it looks like players win at slightly more than the aggregate rate... but by slightly, I do mean slightly.

Possible explanations:

1. They are exempt from the lynch the first few days, which is beneficial whether they are a wolf or a villager. New players get passes and certainly dont get lynched on day 1 or day 2.

2. As villagers, they are an asset. Most of the time, they are really obvious. They are mislynched far less than the average I would imagine.

3. Their wolf range is estimated to be very narrow, they are seen as not capable of very much. If they are somewhat smart they can with this. See Kukra first wolf game, see Gabe first wolf game.

Yes, BBB, I was wondering about that. Another possibility is that after the first couple games, players start experimenting... and their experiments tend to suck.

so subbing in a middle figure for the ranges, 4, 8, 13, etc. and 100 for the final entry, and running an LSR generates an equation of winning chance = 48% + 0.001%*(#games), so is essentially flat

i'm sick, so not really on top form but a couple of points:
1) it shouldn't be a linear model, obviously since it's bounded by 100%
2) i have no idea without giving it some thought as to how to run this more properly - it probably needs more weighting
3) i dunno how to get the R^2 out of my spreadsheet atm
4) but it will be quite low, but also, that's not too much of a problem - if things are fitting too well, i would be more worried

Gotta milk the enthusiasm before they join the jaded has beens.

I agree it's positive. It's just incredibly weak. I don't know what the confidence interval on that 1% is, but it can't be all that narrow and the effect is so small — a 1% increase in the win chances at game 110 versus game 10? Even if that's real, I think the more important point is that while experience probably does matter, it matters much less than most of us would like to think. Or so the data seem to tell us.

If we could eliminate the first and second games, maybe we'd get a stronger effect, probably on the order of 2%, but unless we have a conceptual reason for doing that I don't like it.


As an aside, playing with data like this is something I spent about three years doing when working for a professional expert witness. I promise you, I can make these numbers prove anything you want them to, and can coach you for testimony that will survive any cross-examination and convince any jury. But that won't make it correct.

Running regression on those proxied incremental rates while excluding one- and two-game players does reveal a trend: 47.8% + (0.072% x games) (R² = 0.076). That would mean that the player with 110 games would be expected to win at a 7% greater rate than the one with ten games. The problems, though, are:

• jiggering the data until we find a pattern, without having a conceptual basis for it, is a poor methodology that will always yield a stronger pattern than actually exists; and
• the top end of the data are determined by very few players.

It's been too long, so I've forgotten... but I'll bet one of the young whippersnapper math geeks here can remind us what the conceptually preferred way to model correlation of a probability against a single independent variable is.

If I were creating testimony, I'd start with that and then play around with games played, log of games played, square root of games played... But ultimately we'd present something really simple (and totally specious) to the jury.

OK, still to do (I need to go to bed): looking at villager win percentage as a function of games played. Eyeballing the data, it looks like there may be a much clearer pattern there: my sense of it as that the most experienced players are winning noticeably more than 49% (the average) of their village games, but possibly less than the 46% average of their wolf games.

i don't think anyone thinks luckayluck is the best wwer around.

edit:
http://en.wikipedia.org/wiki/Logistic_regression

Does the data include the dates that the games were played? I'd be interested in seeing what percentage of players with 40+ games have a better winrate in their second 20 than their first 20. I'd imagine it's more than 50% but by a small amount.

As for the weird effect with players doing better in their first 2 games I do think there is some merit in the fact that they get a lot of leeway in their first two games, and some wolves can be flat-out bizarre and live a long time just because no one knows what to make of it. After a couple games, perhaps they are given more respect, which actually makes players less likely to clear them?

I don't think that this stuff proves that there is no skill in ww. I think it shows that players do most of their improvement quite rapidly, and beyond that point their improvements mostly just track the overall improvement of the player pool.

The data unfortunately doesn't include dates, no.

Maybe I can get working on that plus data of when people were lynched / killed next time I have free time on my hands :P

Well as long as you have the game names in there you can grab the dates pretty easily from the archive thread.

After watching an excellent video about the math behind figuring if you are a winning poker player I have to say.

"LOL sample size" to all of this.

I wonder how many vanilla 13ers we would have to play to produce a sample that is w/i +/- 5% of our actual skill.

I think it might be as many as 1k but I have no idea how to figure these things out.

Someone who is great at stats should be able to tell us.

Edit: Mets, Wahoo wanna try?

just because you can't definitely prove which players are winners or losers in the long run doesn't mean that there aren't winners and losers in the long run

Agreed 100% and I never said that.

I just don't think we have anywhere near the amount of games to prove which are which.

Especially considering these stats include a large portion of mish-mashes.

I mean does anyone think 100 hands would be a large enough sample to decide if someone is a winner or a loser? and that doesn't even add in the complexity of 8-50 other players.

hands of poker don't have the same variance as a game of werewolf

my results are about 1 standard deviation above average

OTI doesnt want to hear this. DUCY?

Some "shocking" numbers though. I mean, look at SH, Nez for example.

Ok so is 1k hands a big enough hand sample size?

10k hands?

100k hand?

We need someone with stats knowledge to tell us how many games is meaningful.

it's much less like looking at one player's stats and much more like looking at the database of a range of players, all with varying samples - so like the graphs they produced when they were proving the ap/ub cheats were cheating, say.

Also, if you think about it, a game is so dependent on the whole lineup, that any one player's impact is going to be small, so even if you could model it, it would look like:
P(win) = 50% +/- skill +/- teammates
where skill includes any experience factor as well.
I suspect the impact of skill would turn out to be only a few percentage points at best, and that means that presumably you need a big sample to discern the difference.

Agreed.

Which is why we need a stats expert to help us even come close to finding a suitable sample size.

I do think some players are far superior to others, but with so many players responsible for the result of any given game and all mishmashes being balanced differently, it could take a million games to statistically determine an individual player's skill level.

A ton more.

assuming there is player skill, it seems like the best way to demonstrate it would be to have a large sample of rather smaller games, since it seems intuitive that the smaller the game size, the more "impact" a good player is going to have