Afternoon everyone.

My first post, so be gentle.

A little about my background so you can understand where I am coming from......well, back in the 1980s I owned a ZX Sinclair Spectrum (I'm in the UK so our American cousins may not know what I mean) and bought a backgammon program made by an outfit named Psion. £14.95 for a cassette, a princely sum in those days........around 4 hours work at average pay, so not cheap.

Having played it a few times, it was fairly clear the program was cheating so I very quickly disassembled the code and ascertained this to be the case.

It was very crude, simply roll the computer double after double if the gamestate flag was less than zero.

Fast forward to the 1990s.......I bought a backgammon program from a one-man company in Scotland (can't remember their name).

I demonstrated that was cheating too by replicating the seeding but the owner insisted that the different behaviour displayed by his bot on identical dice rolls was due to statistically similar positions requiring the advancement of the random number generator by one so throwing the whole thing out of synch.

Er, yeah right Hamish, you keep supping at the Scotch my wee friend.

So here we are in 2012.

GNU, no less.

I've whooped the a$$ off of about just about every backgammon computer program out there and yet GNU tans my hide repeatedly.

The cheating seems to be a little more sophisticated......roll the user a lot of 2, 1's rather than making it obvious by rolling the bot double after double.

(A little like Spades card programs, keep dealing the user the Ace of Spades and he/she can't bid nil which is the most powerful bid)

Hey ho, programmers get craftier, c'est la vie.

To cut a long story short (too late!)........I create a file called dice.csv in Excel using a simple +INT(RAND()*6)+1 formula that populates 999,999 cells.

I tell GNU I want to read the dice from a file, point GNU at that file and away we go and in an 11 point game I get beaten 11-3.

Hmmm, I've told GNU in advance what every roll will be, surely it isn't reading the whole file and playing according to what it knows is going to be rolled next?

Tell you what, let's change settings to manual dice and play the exact same rolls.

I won 11-5, GNU played differently even though I made sure I played identically as before.

A genuine question......what's going on?

Did I do something wrong?

This doesn't even make sense. As soon as it played a different move, you couldn't be playing identically. And your contention is that it actually plays different moves in the same positions and same rolls (and same settings) based on whether or not the rolls are manual or from a file?

Snowie, GnuBg and XG are three programs that nobody whoops. If you are tromping them, you are literally the greatest backgammon player that ever lived. Of course, I am sure this is not your claim, but you must be a strong player nevertheless. How have you fared in tournaments against some of the Giants of Backgammon? No Giant of Backgammon has ever accused one of the three major bots of cheating.

I have heard that GnuBg will misplay its defense of massive backgames, and allow you to pump up the cube to astronomical levels. Supposedly, it misjudges its cube decisions, and beavers your double. Then it gives you the cube right back, and you can beaver. Now, you cube again. Etc. By this method, some have claimed to establish a winning record against GnuBg. It all comes with one big victory. But if you play straight, I know of no one who has a winning record against any of the three bots listed above (except in the short term, of course). XG, for instance, plays with a PR of around 0.5. The best players in the world are not even close. They average between 2.0 and 3.0, generally closer to the latter figure.

The test you describe is certainly interesting. I would like to see you try it again, being absolutely certain to duplicate rolls and play. If you save the two sessions, you should be able to see where they diverge. An easy way to examine the sessions is to save them as HTML. Choose File -> Save... -> HTML. Beforehand, go to Settings -> Export, and put a checkmark in every checkbox, in particular, in “Show evaluation parameters.” I have mine set to display a board diagram every “1” move. (Don’t be concerned if the board diagrams won’t display. You have to generate the PNG files first. See http://www.gnu.org/software/gnubg/ma...ml#HTML-export. Choose Settings -> Export -> Style to generate the board images.)

You should be certain that the same level is being used in both sessions. If one session plays using out-of-the-box settings, and the the other uses “world class” or “supremo” settings, it is expected that GnuBg will make different decisions in the different sessions.

Out of the box, GnuBg plays a weak 1-ply game. Change these three settings to get a stronger 2-ply result:

Settings > Analysis... > Analysis Level: supremo
Settings > Players... > GNU Backgammon: supremo
Settings > Rollouts... > Play Settings > First Play Both: supremo

If you can get it to screw up, I would like to see the files. I might try to duplicate your results on my version of GnuBg. (Even here there could be a problem. I am running an unofficial release of GnuBg, given to me to patch a bug I found. I am a bit ahead of the general public with the version I am running. It will get the patch in the next official release of GnuBg.)

Another thing you can do to see why GnuBg beats you is to activate Tutor mode, and set a very low threshold. Go to Settings -> Tutor, and add a checkmark to the checkbox for Tutor Mode. Be sure the Warning level is Doubtful. Next, go to Settings -> Analysis, and set the threshold for Doubtful to 0.02 (or less). Now you are set to begin playing. GnuBg will interrupt everytime it thinks you make a mistake. Not all of GnuBg’s suggestions will be correct. Rollouts, or 4-ply analyses, will overturn some of them. But for the most part, it will reveal to you just how often you are making mistakes. Get ready to be humbled.

After each game or session, be sure to choose Analyze -> Analyze match or session..., and review your results.

Here is a link to a discussion of other tests you can use to check GnuBg.

Am i nuts or does GNU cheat?
http://forumserver.twoplustwo.com/13...-cheat-910882/

Hope this helps.

The question whether GNU (or any other backgammon bot) cheats cannot be answered without being clear about what “cheat” means. To the person who wrote that nobody “whoops” GNU, or Snowie or any of the other bots, that is true, but only where the quality of the dice rolls is either (i) the same for both players, or (ii) more favorable for the bot. The bots are programmed to analyze board situations at hyper speed, and to make the optimal decision based on whatever they “read” from the board.

In my book, “cheat” means giving yourself an unfair advantage where you have some control over the situation. Based on my experience, the bots I have used (GNU and eXtremeGammon) both cheat, and to the extreme. I have come to this conclusion after playing and analyzing (with the GNU analysis function) more than a few thousand matches. My conclusion is NOT based on a mathematical proof or detailed examination of the software code that underlies the program. Rather, I rely on data that is generated by the GNU analysis that is available to ANYBODY who is willing to (i) play 7-game (or longer) matches, (ii) analyze the match, and (iii) open the analysis and look at the data shown on the “Luck” tab.

First a word about “luck”. If both players have fantastically good dice, then neither is lucky. The same is true if the quality of the dice rolls each player receives is extremely poor. Likewise, if the dice rolls for player A are mediocre and the rolls for player B are extremely poor, then player A has a significant advantage because of the dice, and for that reason should be considered the “luckier” player. Indeed, if the “dice roll differential” in a 7-game match is greater than 25%, the player with the advantage should ALWAYS win unless he/she/it is both a complete idiot and seriously drunk. Again, this isn’t based on a mathematical analysis, but on extensive experience.

What does “dice roll differential” mean? This is my own label for a statistic that is extremely useful if you are interested in knowing whether you won/(lost) a match because you played better/(worse) than your opponent, or whether the dice played such a big role that it didn’t really matter how well/(poorly) you played; you would’ve won/(lost) anyway.

This statistic comes from the data on the “Luck” tab from a GNU-generated analysis of a match. Again, matches of seven games or more are needed so that the data will be reliable. On the “Luck” tab there is a line item “Luck total EMG (MWC)”. “MWC” stands for “match winning chances”, which is what interest most/(all?) people. If the quality of the dice rolls for each player was exactly the same (fantastic! – mediocre – utterly deplorable), then subtracting the MWC number for one player from that of the other will result in zero (0). Again, it doesn’t matter whether the dice were great, average, or crap. What does matter is the difference between the two numbers, what I call the “dice roll differential”.

When doing the math, it’s important to remember that you’re dealing with “absolute values”. Here are some examples to illustrate:

- A (+27.488%), B (+1.237%). Subtract 1.237 from both players to get a differential of 26.251. B had little if any chance.
- A (8.589%), B (-34.912%). Add 34.912 to both players to get a differential of 43.501. Again, B has no chance.
- A (36.701%), B (36.792%). Subtract 36.701 from both players to get a differential of 0.91. Dice played no role, the better player will win.
- A (-23.632%), B (-98.256%). Add 98.256 to both players to get a differential of 74.264. Even though A had “bad dice”, B’s dice were far worse and A should win unless A tries to lose.

Based on my experience, the dice rolls generated by GNU consistently favor the bot, and by a substantial margin. “Substantial” means a dice roll differential of at least 25%. Is this “cheating”? It most certainly is. Why? Because the dice rolls are generated by the same application that plays the match.

If there was $$ involved, you can be sure that this would not be the case, or else GNU (and everybody behind it) would be living in some country that is immune from prosecution for fraud. Indeed, you can make LOTS of $$ if you can find a (hopefully wealthy) person who is absolutely convinced that GNU “plays fair”. Don’t bet on the outcome of the backgammon matches. Instead, bet on the “dice roll differentials”. If you bet that GNU will have an advantage of 20% or more, and if you can find someone who will take that bet, then you will quickly become a very rich person.

Gnubg allows you to enter your dice manually; set the bot to "world class" setting and play a bunch of games with manual dice (play honestly !). When you're lazy with the process you'll use the mersenne twister generator provided by gnu again.

A question for "the_wiz" --

Your contention is that XG cheats. (I use XG and Snowie, but not GNU, so I'll stick to XG.)

Are you aware that all world-class players of which I'm aware use XG regularly? (They do.)

Are you aware that none of them think XG cheats? (They don't. Many of them collaborated on XG's development.)

Why do you suppose this is? Normally, one would think that players with the best understanding of the game would be more likely, rather than less likely, to spot something unusual. We seem to have a conundrum here, in that the better someone plays, the less likely they are to spot a cheating program. Seems very counter-intuitive, but I'm sure you must have an explanation. Please enlighten us.

The Dunning-Kruger effect

It is interesting that you specifically mention XG. I purchased their product, joined their site, … and had much the same experience as described in my message to which you responded. Indeed, I used the product ‘tools’ (there are – supposedly – tightly integrated with the site) extensively, and noticed a number of discrepancies. As I did with GNU I tracked these over a long period of time before putting questions to the site administrator/(owner?), a Swiss gentleman (his first name begins with ‘X’) living in the French part of the country. (I am an American expat living in the German part.) The questions were supported with concrete data generated over a substantial period of time, and relied on the statistics generated by XG’s own tools.

The result? My questions and concerns were NOT addressed. Because I thought there might have been a misunderstanding because of the language, I asked the questions again, … with the same result.

Your remark about ‘all the world-class players’ using XG is also interesting. I don’t presume to dispute that. Indeed, I had an email exchange with one of them; his initials are PS, his ‘endorsement’ of the product played a big role in my decision to purchase XG. I emailed him (with copies of the aforementioned messages to the site administrator) because I thought he should know how my questions had been handled. I considered it appropriate to inform him about this because he endorsed the product, and not just passively, but indeed, quite strongly. His response was very interesting. While he didn’t want to get involved with the math, he was quite adept at a ‘marketing spin’ commentary that was as useful as a roll of used toilet paper in a dysentery ward. That was understandable; he has a stake in the site and the number of persons who visit there. To question the integrity of the site – as I did with my questions – would go against what he apparently perceives to be his interests. I’m sure he checked the numbers in my analysis and saw that my questions had substance and were not without foundation. Instead of addressing the questions, he chose the same path as the site administrator. That is regrettable.

With all due respect to the ‘experts’ and ‘champions’ – both those who are real and those who are mere pretenders – backgammon is a game that by its nature relies on statistics and numerical data. If an expert deigns to share his/her opinion with me, I will certainly take that on board. However, where that opinion doesn’t stand up to scrutiny and is contradicted by the numbers, then I have to question why that opinion was offered in the first place. Unavoidably the question arises, is this person really good, … are have they just been extraordinarily lucky?

Hi

assumed Gnu Bg cheats why and how is this so?
Its open source. Open for anybody to participate in its programming and free so noone pays for it.
And about XG...Im with Mr. Robertie.
If all the Bg cracks get tricked by XG its Ok for me too :-)

If I were a rigtard and thought I had evidence of a program cheating, I'd prattle on endlessly and never get around to showing the evidence.

@ Tom Cowley

+1

- never come across 'rigtard' before though. Would that be a righteous ******, or a rigid turd, or just a typo?

I assumed it meant a ****** who thought games were rigged (when they weren't).

Yeah, for all the "online poker is rigged" people who've posted to 2p2.

Official rec.games.backgammon Software Complaint Form
-----------------------------------------------------
[x] it has a biased random dice generator
[x] intentionally coded by the author to give "better" rolls to the
computer to create the illusion of an superior backgammon
program;

I haven't formalised my suspicions, presented a falsifiable hypothesis,
designed an experiment, gathered data and made reasonable conculsions
because I (check all that apply):

[x] don't know how;
[x] can't be bothered;
[x] already KNOW I'm right so there's no point doing any of that stuff;

A couple of more questions for "the_wiz":

How good, in your opinion, would XG be if it didn't cheat?
-- still better than the best humans by far
-- slightly better than the best humans
-- equal, but not better than, the best humans
-- pretty good, but not in a class with the top humans
-- mediocre, probably below average in the championship division of a major tournament
-- not even championship level caliber.

Just curious, but where would you rank yourself on the same scale?

Could you name some human players who are legitimately good, as opposed to folks on a run of good luck, whose judgement you would respect?

The better you play the luckier you get.

Not that is it important, but I am French and living in the US.

My position about Dice complainer is, and will always been the same:

• for GammonSite check www.gammonsite.com/dice.asp

• for eXtreme Gammon check the help file, dice section

• for XG mobile check http://www.xg-mobile.com/help.html#dice

I know by experience than trying to convince that particular kind of person that the program is fair is just a waste of time for the user and myself.

A note about GNUBG: of all the program it is the only one which is open source, you can yourself look at the code, compile it yourself. Claiming it cheats is the most ridiculous claim as you can check by yourself without having a to play a single game. Still people claim it cheats, unbelievable!

This will be my only contribution to this thread

Back in the day, before the Age of Reason, when people saw something astonishing occur, like the moon rising, they had to explain it somehow and any idiocy would do. I well remember that when I first learned the game and downloaded gnubg, it seemed as if some extraordinary sorcery was at work; the dice would just dance to its tune as it crushed me. I never caught the full-blown affliction of paranoidicia cretinalis, but I couldn't entirely banish the human tendency towards magical thinking, and spent more time raging against the dice gods than I did in thinking about my position. When I realised this was getting me nowhere, I decided to forget all that and learn the game instead.

XG came along at just the right time, and with its help I have become a reasonable player. I'm glad Xavier doesn't spend too much of his time responding to drivel, because I want him to put his exceptional talents into improving XG for the future. At the moment it is a wonderful player but a poor teacher, and I am greedy to learn. But if it wasn't for that apparent sorcery of the dice I first saw with gnubg, I would never have fallen in love with the game.

@Robertie:

I had assumed that you had read and understood my earlier comments before you took exception to them. From your comments that appears not to be the case.

My original comment a week or so ago was quite clear as to what I considered to be “cheating” – giving yourself an unfair advantage where you have some control over the situation. In the context of the theme for this particular thread, and as I tried to explain in the comment, “cheating” means the bot giving itself an advantage in the quality of the dice rolls it receives relative to that of the non-bot player.

My conclusions are based on an extensive number of 7-game matches where the dice rolls are generated by the bot. Based on my own experience, the bot had a significant advantage (+25% “dice roll differential” as that term is defined in my comment) in the quality of the dice rolls it received in a significant number of matches (+80%). It was also interesting to notice how seldom the dice roll differential was less than 10%, i.e. where the dice were relatively balanced for each player. This latter factor has been replicated in my experience with online backgammon sites.

You asked my opinion about how good XG would be if it didn’t “cheat”. My original comment states, “the bots are programmed to analyze board situations at hyper speed, and to make the optimal decision based on whatever they ‘read’ from the board”. I certainly don’t consider that cheating; that is simply playing better. I also believe that the vaunted backgammon expert Bill Robertie (who I thought you might be) would have understood this. It’s a pity that his moniker has been appropriated by an imposter; otherwise, why the inane question?

As for your question, “Could you name some human players who are legitimately good, as opposed to folks on a run of good luck, whose judgment you would respect?” I understand what “legitimately” means but fail to understand how it fits. Indeed, given a group of world class players playing against each other in a tournament, it is those who are “on a run of good luck” who are most likely to succeed. Unless they’re playing with loaded dice, then how could their success be considered “illegitimate”?

I find your question “where would you rank yourself” on your “scale” to be a bit odd, and can only assume you intend it to mean what it implies, that unless I (or anybody else) play at least at a “top human” level then question or doubts about the “top quality” tools you endorse are inappropriate, even where they are supported by objective and fair explanations, as mine have been. Please forgive my impertinence, but that comes off as more than a little arrogant.

As for those persons whose judgment I would respect, then from a backgammon context I would include the genuine Bill Robertie. The gentleman ‘PS’ mentioned in an earlier comment once complimented me on my understanding of some fundamental, basic principles. When I first began playing backgammon I was fortunate to discover a site that had some well-written articles by Mr. Robertie that got me started on the right foot. It’s too bad his name as been appropriated by someone who seems to lack the class the genuine Mr. Robertie no doubt possesses.

Might I suggest that before you take exception to a comment on your forum, you first make an effort to understand what was written. I don’t ask this for my benefit, I will not waste any more of my time here and won’t read what you may write. But perhaps someone else who may comment in the future may be spared unnecessary prattling.

Ok, I have some time to waste, so I'll cut all the crap and simply ask:

When you say "based on an extensive number of 7-game matches where the dice rolls are generated by the bot", could you quantify "extensive"?

Sorry, but I can't understand how can GNU cheats if it is an open source. Everyone can check the code, no?

Please correct me if I'm missing something very basic here, but surely we expect to see bots/good players deliver significantly better 'luck' ratings over time anyway (at least as defined by Gnub)?

Bad players don't put themselves in positions where big swings in equity are as likely as good players - which is how Gnub defines 'luck'. Just as a hyper simple example, someone making schoolboy errors on bearoff, such as not bearing off down to four checkers when only a double can save them at the end, is effectively throwing away any luck rating they can have by not allowing a joker to be of any use.

Gnub would happily say you were lucky in the cases where you play well, bear off the right number of checkers and leave yourself an out, but keeps silent when you make a huge mistake that throws away your 'lucky' get out of jail free card, and you have no possible swing in equity available.

If you just make this same mistake over and over again, Gnub will always come out luckier than you - even though luck has nothing to do with it, as when the shoe is on the other foot it has a chance to have a lucky roll, and you have none.

The bot basically never makes these errors, as it just unrelentingly grabs every bit of equity possible, and never misses how to maximise its outs, which on a rating scale like Gnubs we should expect to see it score better than humans - a huge part of its edge is that it never misses a spot to maximise jokers/long shot rolls, something humans don't, and if we are only measuring the size of the equity swing from the average (which is how Gnub calculates it), it will do better than humans simply because the deviation from the average of zero equity is zero.

Great post. All the really good bots show better 'luck' ratings over time, but I never really thought about exactly why this should be. Your post nailed it. Good job.

Assuming it's all calculated the same way from the same position, your expected luck on a given roll has to be 0. If you can put yourself in a position to need 66 only to win or be stone dead (if you even get another turn), then your EV from the former when you roll is -34/36. 35/36 you'll just lose for -2/36 equity and 35/36*-2/36 = -70/36*36 luck, and when you hit you'll pick up 70/36 1/36th of the time to balance out to 0. If you're busted, then no matter what you do it's 0 luck all the time. Basically the bots give themselves extra chances to hit jokers, but that also means they have extra chances to MISS jokers.

It's conceivable that there's some kind of quirk in the luck numbers where everything isn't getting calculated to the same depth (like the original position's equity pre-roll is calculated one way, then you roll 31 and the final equity is calculated 3 ply after the move which is 4-ply from the original position, etc.. or even rounding issues from having +equity more than -equity, which could manifest cumulatively over bazillions of games, not to +25% in a match to 7) and something in the evaluation function combined with the type of crappy positions rigtard tends to get in bias the luck stat (not the actual luck)... or the OP is working off a microsample or just lying.

Yeah I'm thinking of the luck rating screen in Gnub which flags the rolls marked 'Very lucky' 'Lucky' 'Unlucky' etc etc in match analysis. In this screen you should expect to see the bots be 'luckier', simply because the small equity they are losing all the times they fail to hit when getting in better joker/anti joker spots and missing them is just ignored by the grading thresholds.

The bottom of that screen has the overall luck rating, which I assume does not ignore results of a small change of equity (I can't find anything during a brief search in the Gnub literature explaining exactly how it works). Here I am lost, as if it does count everything, it should all equal out over time as per your point.

They do say luck on default is calculated 0 ply, so if you are playing with a higher ply the point you make in the second paragraph might be worth exploring.

Supposedly snowie calculates consistently (1-ply preroll, then uses those values). Didn't find anything on gnu or xg, although the xg guy posts here and should be able to check that easily enough. I'm not currently bored enough to attempt to decipher gnu source code. If they all do it consistently, I don't know why a bot would have a positive luck rating per roll (your point about percentage of above-threshold rolls makes sense though) over time.