TrueChess Computer Analysis: Fischer is strongest ever. Botvinnik > Kasparov

[EGarrett]

Originally Posted by Do it Right

Oh stop being so damn obtuse. Do you honestly not realize what excluding draws does? I refuse to believe you're that dumb and prefer to believe you're simply trolling me.

You need to control yourself and discuss things like an adult. That is not rational or acceptable behavior.

As far as I see, you cannot in one breath mock the study for leaving out drawn games, then go on to list several problems you see with that type of analysis, all of which come from drawn games.

Now, there is one problem that immediately does jump to mind with leaving out those drawn games, which is that it limits the sample size for each player. But you can increase the sample size by simply looking to the tables with a greater number of years included...and you will get more moves while still avoiding the two major potential problems. This appears to be the approach taken in the study also.

And if you don't see the point the above game illustrates as it applies to decisive games then add a move where one of the 'players' hangs a piece and they continue to play weak, but computer approved, moves until one resigns. Wow, such accurate and amazing play from the winner!

The above game would include a massive negative mark for the loser. And as for the winner, the computer would not see any blunders, however, the list was not done solely by rate of blunders. The other major ranking category (as far as I see) is "percent better than average grandmaster move." A player who just took a hung piece then won quickly would not gain any blunders, but he also would not gain any percentage points in terms of making moves that are rated better than the average grandmaster.

So, in the above example, it would seem that the game would be a huge negative mark for the player that hangs the piece, but would not be a huge gain for the winning player, unless his method of victory was far more accurate than any other Grandmaster who was finishing such a "gimme" game. That seems fair.

But I am of course, interested in discussing this further or other potential problems. I noticed that there are a few games of Claude Bloodgood on wikipedia, do you feel he would be rated too highly for a reason other than the above?

Originally Posted by wlrs

It's so much harder to play "accurately" when your opponents are constantly putting pressure on you (which I guess is one of the points DIR is trying to make).

This must certainly be true. One thing that is listed in the study is the "complexity score" for each player, meaning how often they put themselves (or were put into) difficult positions.

"Ranking a Player's Relative Accuracy of Play
A Complexity Table (see graph above), constructed from all the positions analyzed, is used to level the playing field between different players. The more complicated the position, the higher the expected raw error will be."

So it would seem that the study was more forgiving for people who played more difficult positions, thus we shouldn't have to worry that much about how the different amounts of pressure effected each player's accuracy, (well, at least the pressure of difficult positions).

[Noir_Desir]

You cannot try and estimate the "correctness" of games and then leave out the games that end in the "correct" result, which with 99% certainty is established to be a draw. This isn't just a matter of sample size. It also skews the game selection towards weaker opponents.

[Do it Right]

EG the reason I am being somewhat less than tactful is that the whole concept of simply ignoring draws immediately invalidates the primary reported results. I feel it is also invalid for numerous other reasons as well but ignoring draws is obvious and clearly invalidating. It's like if you said: "Hey I have this really shocking theoretical physics concept I'd like to share! I think I'm onto something huge here. Ok so first off, assume you can travel faster than the speed of light..."

At that point all that's left is



Continuing to humor the rest of the 'study', with study obviously being used in the loosest possible sense given the obvious lack of any expertise evident from the immediate blatant biasing his own results, is just pointless. Though I did offer other reasons the study was invalid aside from just the draw issue. The problem is that with enough broken logic and self biasing you can 'prove' just about anything. I suppose it's a bit of a pet peeve of mine when people do just that and then present it in a format that can easily mislead less scrutinizing or knowledgeable individuals into believing it's legitimate.

[EGarrett]

Well if the concern is that he was sloppy, foolish or biased in not including draws at all, that's not the case. While re-reading the page, I see now that he included a version of the results with drawn games counted as well. Kasparov ranks higher on these lists, those Fischer is still clearly #1.

http://www.truechess.com/web/champsdraws.html

I don't think that the person conducting the study showed himself to be lazy or ignorant in how he conducted his study. The amount of information provided is exhaustive, and there were a number of allowances made to try and minimize the skewing of the results due to things like different pressure and complexity and the effects of draws (by including the option to view the lists with or without drawn games).

Now of course, these draw-included lists could open things up your other complaints, however, when both sets of data give similar results, I think that should be considered valuable information.

[Do it Right]

His main report was excluding draws. That shows either bias or stupidity. Simple as that.

But your comment actually made me bother to check more in the site and this is damn hilarious. His own results invalidate themselves and support exactly what I was talking about earlier his particular sort of study measuring the mediocrity of the competition as opposed to the quality of the player. You're right that he lists Fischer as the #1 player, draws included. The funny thing is the fine print. That's for a 1 year result, and he lists Fischer's prime at 1968. Well before he hit his prime by any rational measurement. The only thing that's interesting about 1968 is that Fischer played a lot of relatively terrible players. He played in two tournaments where many of his opponents weren't even GMs. Crushing two tournaments where he was vastly far ahead of the field was Fischer's peak performance, and shows he was clearly the best player ever. That's some high class statistical analysis!

If you really want to delve into this ridiculous era vs era business try Jeff Sonas' work. I think it also has a number of flaws, but he is at least a trained statistician and his work has been developed with the input of many strong chess players: http://chessmetrics.com/cm/CM2/PeakL...00000000010100

And you can still be happy, for one year peak performance - Fischer reigns supreme. Unsurprisingly that year is 1972 though. 1968 is just well...

[EGarrett]

Originally Posted by Do it Right

His main report was excluding draws. That shows either bias or stupidity. Simple as that.

I think you realize that your statement here is highly suspect.

It is only bias if he was trying to skew the results toward a desired conclusion apart from what the data would otherwise suggest. But Fischer is still #1 even when draws are counted, and there doesn't seem to be any other major change in the ratings that he might be trying to hide.

It is only stupid if one can show that the results including draws are far more accurate and do a better job of showing what he wants to show. However you yourself have argued that including draws introduces several potential problems.

Also, it would only be ignorant if he was not aware that draws might provide a different story. He freely provided his raw data alongside a table made for the draws that anyone could access, so he did not lie or ignore information. The only clear conclusion seems to be that his interest was in answering the question of who was the most accurate human player outside of opening theory, and he felt that excluding draws was the best way to do it, though he freely provided the other information for those who might disagree.

You may feel he is incorrect, but he has not shown himself to be biased, stupid, ignorant, or dishonest. Furthermore, it would be better for you to focus on actual logical evaluation and discussion and not personal attacks. When you do so, the only person who looks bad is yourself.

But your comment actually made me bother to check more in the site and this is damn hilarious. His own results invalidate themselves and support exactly what I was talking about earlier his particular sort of study measuring the mediocrity of the competition as opposed to the quality of the player. You're right that he lists Fischer as the #1 player, draws included. The funny thing is the fine print. That's for a 1 year result, and he lists Fischer's prime at 1968. Well before he hit his prime by any rational measurement.

That was the best year for Fischer that his algorithm found. He didn't choose it.

The only thing that's interesting about 1968 is that Fischer played a lot of relatively terrible players. He played in two tournaments where many of his opponents weren't even GMs. Crushing two tournaments where he was vastly far ahead of the field was Fischer's peak performance, and shows he was clearly the best player ever. That's some high class statistical analysis!

The study's ranking method compared each player's move to what the computers (Crafty, double-checked by the stronger Rybka) thought was the best move, and to the quality-level of the best move that the average grandmaster would play in that position.

This does not give an inherent advantage to someone playing against weaker opposition. Again, they are compared to what the computer would pick, and what the average grandmaster would've picked, not to their opponent.

If anything, playing against weaker opposition could've caused the studied player to not think as hard about his moves, which would have significantly harmed in the ratings. The opposite of what you seem to be claiming.

If you really want to delve into this ridiculous era vs era business try Jeff Sonas' work. I think it also has a number of flaws, but he is at least a trained statistician and his work has been developed with the input of many strong chess players: http://chessmetrics.com/cm/CM2/PeakL...00000000010100

I know the site. It focuses on wins and losses amongst players, not on the strengths of their moves, thus I find it unsatisfying for comparing raw quality of play.

[Do it Right]

1. The premise of the study is absurd - using software weaker than many of the players involved to measure the quality of the players' moves with zero compensation for the peer bias.

2. The author of the study is absurd. I don't know why he chose to exclude draws. I don't really care. It shows something is not sound in his head or that excluding draws was a better match with what he (in your words) "wanted to show." HELLO - DO YOU NOT UNDERSTAND TURNING YOUR DATA AROUND TO BETTER MATCH WHAT YOU WANT TO SHOW IS BIASING? YOU ARE NOT THIS STUPID SO STOP FRIGGIN TROLLING ME BECAUSE I'M APPARENTLY VERY SUSCEPTIBLE TO IT!

3. The results of the study itself are completely absurd. Fischer 1968, having played against practically no top players, was apparently his prime year and also the single best year and performance by any player of all time. Here's some of the amazing play he faced from that year:



An interesting and dynamic position. Fischer is black. His opponent makes it really tough on him. White to move and helpmate:

Spoiler:

1. Bg4?? Bxg4 2. Kxg4 Rxg2+ 3. Kh3?? Rg1 4. e6 Nf5 5. Rd3??????? g5 0-1.

AMAZING PLAY FROM FISCHER! How could he manage to find such tough moves.

That quality of play was not uncommon for the games he played in 68. He was slumming it. Look over some of the other games for the strongest year for the best player of all time.

You, egarrett, are the reason 'studies' like this are so dangerous. I am at least 90% sure you actually believe in the validity of this study, though 10% of me still feels this is just some slightly elaborate trolling. In any case given that 90%, so long as there's enough grey area - I doubt anything on this earth could convince you otherwise. It's a pity which is why I find 'studies' like this so offensive.

[Do it Right]

You can also see more evidence of this 'study' simply picking the players with weakest competition. The 5th strongest player year of all time was Capablanca 1915. As a result of WW1 he was left playing mostly casual and weaker fields. Many of his opponents were not even masters. He obviously did incredibly well conceding only 8 draws in 41 games, winning the rest. The quality of games was generally low due to the opposition, yet once again - wow that was apparently the peak of his career and the 5th best performance of a year by any world champion ever. It's just completely absurd. It should be obvious how and why his methodology is broken.

I would posit that the reason he abandoned the project was the he started to find 'unexpected' results. He stated (in 2009) he planned to add DeLaBourdanais, McDonnell, Anderssen, Rubinstein, Pillsbury, Schlechter, Keres, Korchnoi. I've no doubts he ended up "discovering" that a player like Pillsbury was likely the strongest player ever!

[EGarrett]

Originally Posted by Do it Right

1. The premise of the study is absurd - using software weaker than many of the players involved to measure the quality of the players' moves with zero compensation for the peer bias.

Not true. From my understanding, he used Crafty because he was able to modify it easily, and had it function to simply point to areas for the far stronger Rybka to investigate...not unlike a bloodhound that sniffs out potential stashes of drugs for the DEA agent to apply a more studied judgment.

http://www.truechess.com/web/projectdetails.html

"Each move with a "raw error" of at least 1.25 pawn was subsequently examined (with no time limit) with Rybka (version 2.3.2a running by itself on a quad-core Q6600 computer), the strongest program available at the time, and analyzed until it was determined whether the move exceeded 0.75 pawns, the (arbitrary) blunder-threshold for this project."

Another relevant comment he made...

"The goal was to be as correct as possible, not to adhere slavishly to the computer's evaulation. For example, I overrode the computers' decision and classified Fischer's 29...Bxh2 in the 1st game of the 1972 Fischer-Spassky match as a blunder -- true, the resulting position is not completely lost, but 29...Bxh2 turned an easy draw into a fight for survival (a fight that Fischer lost)."

2. The author of the study is absurd. I don't know why he chose to exclude draws.

"I prefer to look at the data with draws excluded. (Although the Summary Rankings are presented both ways.) I think it is misleading that certain players should have their blunder rates dramatically lowered because of a fondness for the "grandmaster" draw."

Makes sense to me.

It shows something is not sound in his head or that excluding draws was a better match with what he (in your words) "wanted to show." HELLO - DO YOU NOT UNDERSTAND TURNING YOUR DATA AROUND TO BETTER MATCH WHAT YOU WANT TO SHOW IS BIASING? YOU ARE NOT THIS STUPID SO STOP FRIGGIN TROLLING ME BECAUSE I'M APPARENTLY VERY SUSCEPTIBLE TO IT!

You're getting confused with language. What he "wanted to show" was who, outside of progress in opening theory, was the most accurate player in human history. Not one specific person. I thought that was clear.

3. The results of the study itself are completely absurd. Fischer 1968, having played against practically no top players, was apparently his prime year and also the single best year and performance by any player of all time. Here's some of the amazing play he faced from that year:



An interesting and dynamic position. Fischer is black. His opponent makes it really tough on him. White to move and helpmate:

Spoiler:

1. Bg4?? Bxg4 2. Kxg4 Rxg2+ 3. Kh3?? Rg1 4. e6 Nf5 5. Rd3??????? g5 0-1.

AMAZING PLAY FROM FISCHER! How could he manage to find such tough moves.

That quality of play was not uncommon for the games he played in 68. He was slumming it. Look over some of the other games for the strongest year for the best player of all time.

I have already explained that playing against weak opposition does not offer any inherent advantage in this study, and if anything, would be a disadvantage. You have ignored my explanation and have offered a reply that simply repeats the same error.

If you feel that playing against weak opposition would help someone in the TrueChess study, which ranks them by comparing their choices to computers and the quality of what the average grandmaster would be expected to play in the same position, then please explain how you feel this to be the case, as, otherwise, it is quite clearly not the case at all.

You, egarrett, are the reason 'studies' like this are so dangerous. I am at least 90% sure you actually believe in the validity of this study, though 10% of me still feels this is just some slightly elaborate trolling.

I've been familiar with the study for awhile, and I have a general grasp of how it functions, which is the basis for my posts. It would seem that what you've been posting is focused instead on personal attacks and misunderstandings.

That being said, I think you are stuck on trying to judge "greatness" by the resume of wins and losses, and assuming that the study must reflect the conclusions suggested by this instinct, and assuming it must be flawed when it doesn't. However, the approach you want to instinctively use is inherently flawed because a great player can't control the quality of his opposition, and simple wins and losses cannot judge the actual accuracy of play.

Assume for example, that there was some hidden human super-genius who lived in Guatemala and played at over 3000 ELO strength, in other words, at the level of an engine. All his moves, let's say, are Rybka's first choice on a high-level CPU given an hour to think. But this person only played against random enthusiasts he could find at the chess clubs there, none of whom were over 2000 in playing strength. By your measure, this freakishly good human being is not the strongest or most accurate player ever, because he's just beating up on club players...despite the fact that he would slaughter any human being that's ever lived in match. However, the analysis used by the TrueChess study would be able to identify his talent-level (given a database of his moves of course) and would accurately label him as by far the best human ever, even though he didn't ever play on a grand stage or against high-level opposition. The actual fact of the matter, in these cases, can be reached by the TrueChess method, whereas yours falters.

[EGarrett]

Originally Posted by Noir_Desir

You cannot try and estimate the "correctness" of games and then leave out the games that end in the "correct" result, which with 99% certainty is established to be a draw. This isn't just a matter of sample size. It also skews the game selection towards weaker opponents.

I'm not entirely clear on your reasoning here. It almost seems like you are arguing that since the ultimate "solution" to chess is 99% likely to be a draw, that the most accurate games must also be draws. Given that one player in a game can blunder horribly while the other player plays perfectly (and that a perfect player does not become less perfect just because he's playing someone extremely weak), I don't think your reasoning here is valid. Remember, the objective here was not to find the best played games by both parties involved, but to find the players who played their own side the best regardless of what their opponents did.

If I misunderstood you, please re-state what you said so I can see it more clearly.

Also, regardless, if you would prefer to see the results with draws included, they are located here.

http://www.truechess.com/web/champsdraws.html

The results are pretty similar. Fischer remains #1 on almost every list, though Kasparov ranks a bit higher.

[DaMaGor]

Originally Posted by EGarrett

Not true. From my understanding, he used Crafty because he was able to modify it easily, and had it function to simply point to areas for the far stronger Rybka to investigate...not unlike a bloodhound that sniffs out potential stashes of drugs for the DEA agent to apply a more studied judgment.

http://www.truechess.com/web/projectdetails.html

"Each move with a "raw error" of at least 1.25 pawn was subsequently examined (with no time limit) with Rybka (version 2.3.2a running by itself on a quad-core Q6600 computer), the strongest program available at the time, and analyzed until it was determined whether the move exceeded 0.75 pawns, the (arbitrary) blunder-threshold for this project."

Another relevant comment he made...

"The goal was to be as correct as possible, not to adhere slavishly to the computer's evaulation. For example, I overrode the computers' decision and classified Fischer's 29...Bxh2 in the 1st game of the 1972 Fischer-Spassky match as a blunder -- true, the resulting position is not completely lost, but 29...Bxh2 turned an easy draw into a fight for survival (a fight that Fischer lost)."

"I prefer to look at the data with draws excluded. (Although the Summary Rankings are presented both ways.) I think it is misleading that certain players should have their blunder rates dramatically lowered because of a fondness for the "grandmaster" draw."

Makes sense to me.

You're getting confused with language. What he "wanted to show" was who, outside of progress in opening theory, was the most accurate player in human history. Not one specific person. I thought that was clear.

I have already explained that playing against weak opposition does not offer any inherent advantage in this study, and if anything, would be a disadvantage. You have ignored my explanation and have offered a reply that simply repeats the same error.

If you feel that playing against weak opposition would help someone in the TrueChess study, which ranks them by comparing their choices to computers and the quality of what the average grandmaster would be expected to play in the same position, then please explain how you feel this to be the case, as, otherwise, it is quite clearly not the case at all.

I've been familiar with the study for awhile, and I have a general grasp of how it functions, which is the basis for my posts. It would seem that what you've been posting is focused instead on personal attacks and misunderstandings.

That being said, I think you are stuck on trying to judge "greatness" by the resume of wins and losses, and assuming that the study must reflect the conclusions suggested by this instinct, and assuming it must be flawed when it doesn't. However, the approach you want to instinctively use is inherently flawed because a great player can't control the quality of his opposition, and simple wins and losses cannot judge the actual accuracy of play.

Assume for example, that there was some hidden human super-genius who lived in Guatemala and played at over 3000 ELO strength, in other words, at the level of an engine. All his moves, let's say, are Rybka's first choice on a high-level CPU given an hour to think. But this person only played against random enthusiasts he could find at the chess clubs there, none of whom were over 2000 in playing strength. By your measure, this freakishly good human being is not the strongest or most accurate player ever, because he's just beating up on club players...despite the fact that he would slaughter any human being that's ever lived in match. However, the analysis used by the TrueChess study would be able to identify his talent-level (given a database of his moves of course) and would accurately label him as by far the best human ever, even though he didn't ever play on a grand stage or against high-level opposition. The actual fact of the matter, in these cases, can be reached by the TrueChess method, whereas yours falters.

Cute idea for a short story, though it's been done, albeit with New Jersey instead of Guatemala. It's incredibly unrealistic, though -- no one, not even a supergenius, approaches their potential without competing against strong opposition.

To know a thing you must know its limits. Anyone can be plenty accurate against weak opposition. Strength and accuracy, among many other things, are defined by where they begin to become insufficient.

[TomCowley]

Blunders
A move is a blunder when: (A) the "best" move is rated at least 0.75 pawns better by Rybka [2.3.2a] and at least 1.25 pawns better by Crafty [20.14]. (B) the player is not making a desperate move because he is "lost" [i.e., the position is no worse than -2.00 pawns]; and (C) the opponent has at least a reasonable chance of saving the game [i.e., the opponent's subsequent disadvantage is less than 1.50 pawns]. (I analyzed with Rybka until I was sure the opponent actually had reasonable practical chances to at least draw.).

This is a big problem. Obviously you shouldn't give the first **** about a "blunder" that changes the eval from +14 to +12 or something, but because there doesn't seem to be an eval cutoff for what moves are counted, you don't get dinged with a blunder when you're clearly winning unless it blows the game, even if the move is actually clearly suboptimal. This instantly creates a bias in blunders/move in favor of people playing terrible opponents because they'll be playing in the freeroll zone much more often because of the skill difference, not to mention the opponents probably not resigning as fast as GMs who are getting beaten as badly.

By that judging standard, I could wind up as the most accurate player in history, possibly even with a perfect score, as long as I only played people who kept hanging pieces quickly.

[John_Douglas]

I don't know why these discussions constantly flare up every few months. You can't directly compare players from different eras who played under completely different conditions and against different classes of opposition.

Each of the great players of the past deserves respect for his achievements, but it's silly to try to compare them and debate who is stronger than who.
/end thread

[tchaz]

itt we learn that Botvinnik in the 1, 2 and 3 year periods up to 1932(!!!) was a stronger player than any 1, 2 or 3 year period of Kasparov's career

Spoiler:

butnah

[EGarrett]

Originally Posted by DaMaGor

Cute idea for a short story, though it's been done, albeit with New Jersey instead of Guatemala. It's incredibly unrealistic, though -- no one, not even a supergenius, approaches their potential without competing against strong opposition.

Whether the hypothetical has been proposed before, whether it's realistic, or whether the player in question was playing to their potential are all completely irrelevant. A hypothetical does not become less accurate or useful because it isn't original, nor does it become accurate or useful because it is original. It also wasn't proposed that a person could ever approach the strength of a top-level of Engine. Only that a player who could beat any human being but who had a sample against weaker players would not be recognized by Do It Right's preferred standard but could be found by TrueChess's algorithm. Lastly, the study was not meant to measure someone's potential accuracy, only the actual accuracy of the moves they played. So whether they were pushed by their opposition doesn't matter, and not being pushed (i.e. playing weaker players) would seem to be a detriment to their placing in the study instead of an edge.

To know a thing you must know its limits. Anyone can be plenty accurate against weak opposition. Strength and accuracy, among many other things, are defined by where they begin to become insufficient.

Unless I've misunderstood you, what you seem to be saying isn't true at all. One player may beat a weaker player in 30 moves, while a stronger player will find an even more economical win. Both player's skill level was more than sufficient to win, neither player's limits were tested by their opposition, but nonetheless, we can find the difference in accuracy between the two winning players.

Originally Posted by TomCowley

This is a big problem. Obviously you shouldn't give the first **** about a "blunder" that changes the eval from +14 to +12 or something, but because there doesn't seem to be an eval cutoff for what moves are counted, you don't get dinged with a blunder when you're clearly winning unless it blows the game, even if the move is actually clearly suboptimal. This instantly creates a bias in blunders/move in favor of people playing terrible opponents because they'll be playing in the freeroll zone much more often because of the skill difference, not to mention the opponents probably not resigning as fast as GMs who are getting beaten as badly.

An interesting point that I will be thinking about. This notion of "free-rolling" seems to be a double-edged sword, in that I doubt Fischer (or Kasparov) would avoid a potential thin loss of edge 5 moves down the line if they were playing against a grade-school child, in favor of playing a clear attacking move, so...at least at this first consideration, we'd have to compare the relative gain of playing more moves that don't count for blunders with the loss of not wasting energy calculating super-complex variations against opponents who would never see them.

By that judging standard, I could wind up as the most accurate player in history, possibly even with a perfect score, as long as I only played people who kept hanging pieces quickly.

You can't gain ground in the "percent better than average grandmaster move" category by doing this. The study doesn't only judge by number of blunders. Also, if we look at the "draws included" versions of the lists, a number of the grandmasters already have a 0.0 for their average number of blunders, so you would need to score extremely well in the other category to make the top list.