I did a little experiment to try to gage the accuracy of the model. I split the predictions into buckets of 10% each and measured the accuracy of results in each bucket. E.g. for all players where a win was predicted between 50%-60%, the optimum result would be an average win of 55%.
These are the results:
Code:
Bucket(Predicted win %): 0-10 Win: 2 Loss: 35 Actual Win %: 5.41
Bucket(Predicted win %): 10-20 Win: 32 Loss: 146 Actual Win %: 17.98
Bucket(Predicted win %): 20-30 Win: 83 Loss: 214 Actual Win %: 27.95
Bucket(Predicted win %): 30-40 Win: 132 Loss: 279 Actual Win %: 32.12
Bucket(Predicted win %): 40-50 Win: 239 Loss: 278 Actual Win %: 46.23
Bucket(Predicted win %): 50-60 Win: 277 Loss: 239 Actual Win %: 53.68
Bucket(Predicted win %): 60-70 Win: 280 Loss: 132 Actual Win %: 67.96
Bucket(Predicted win %): 70-80 Win: 214 Loss: 83 Actual Win %: 72.05
Bucket(Predicted win %): 80-90 Win: 146 Loss: 32 Actual Win %: 82.02
Bucket(Predicted win %): 90-100 Win: 35 Loss: 2 Actual Win %: 94.59
Are these results meaningful?