True, he might have taken a break from crushing high stakes NL to battle in the streets competing against the likes of Juergen Bachmann, Tsuneaki Takeda, Luca Moschitta, and of course Mike phan at getting the highest win rate vs a bot to potentially win $5k canadian dollars (3783 usd)


Also it looks like Gaia Freire took a break from his restaurants and hot tv shows to play the bot as well

When you're winning at that high of a winrate, it is highly highly significant.

I think the chart tells us the opposite.

The variances and confidence intervals are massive.

When a player's winrate can be given as 5 plus-or-minus 729 after 3000 hands, we know nothing about that person's true ability

I lol'd

Looks like Juan Manual Pastor is the one who threw the contest. Why does Juan hate humanity?

I meant more the overall, but if you think AIVAT is legit (which it probably is), then 10 of the 11 of the players who played 3k hands were losing with 95% confidence. If you don't think AIVAT is legit, the humans were losing between 27.2 and 71.2 bb/100. That's a large range, but they were definitely losing a lot.

Becuase we can't narrow down their true winrates to a very small range with a high level of certainty is not at all the same as saying we can't know anything.

For the best performer it may seem like a wild guess because he was close to break even and the confidence intervals are large relative to his performance.

But in the case of the biggest loser(s) who played 3k hands, muskan and pol, it suggests that their true loss rate is between 850 and 1150mbb/g or -85bb/100 and -115bb/100. In the case of these two they're actually losing more than if they were to just fold every hand pre.

You can't say with a high level of certainty that the best performer was worse than the AI.

You can with a very high level of certainty say that the biggest losers are significantly worse than the best performers, and are also significant losers vs the AI.



Not surprising that they chose the people that they did. Would you really want to choose the best players in the world who would likely make the program that you've been working years look bad given that it'll probably have implications on the level of funding you get in the future? Much better to pick self proclaimed pros who actually suck.

Have just had a quick chat with someone who took part in this and who thinks the bot is 'pretty terrible'. Hoping to speak to him about it again soon more in-depth.

One strange aspect which I hadn't realised from the paper which was released was that after every hand the stack sizes were reset, so normal heads-up stack strategies which the pros might have employed were out of the window.

From the cover letter sent to prospective players in the trial:
'To be eligible to participate in this study you must be:
– of the age of majority in your country of residence,
– a professional poker player,
– willing to be identified in an associated research article, and
– able to understand written English, including this information letter.'

OK, 'professional poker player' covers a multitude of sins but I know at least that the player I spoke with is a good pro with plenty of online heads-up experience.

For the record, there were 3 prizes on offer 1. $5000, 2. $2500, 3. $1250 for the best performers (excluding the bot!)

So, now all I have to do is decipher the initial win-rate/loss-rate charts :/ Still making no sense, and for anyone who has looked at the maths in the study paper...good luck with that! :O

The paper is very good. The real world testing not so much.

Beating professional golf long drivers at a 9 iron closest to the pin game is a lot different than beating the best short game professionals in the world.

This contest is basically akin to that.

That doesn't invalidate the contest and make it meaningless, a bot beating a range of competent poker professionals at a game most of them likely never play is still a feat, but it's not obvious to me that some of the low level HU bots that populate non-Stars sites in particular could not also do this too.

I get that they want their software to be successful, but this is pretty ridiculous. You reset the stacks after each hand (not how HU NL is played in reality) and you face pros largely of a different subset of poker. Huge stretch here, not to mention there are some very successful accounts that have been caught and banned for winning at HUNL poker with automated decision making (moreso at shorter stack HU, but also variable stack, not resetting to the same each hand).

Yeah this test is super flawed, if only top 3 winners vs the bot get prize money then people are obv gonna play super agro or just punt off stacks to try and run up a big enough win to finish in the top 3.

I guess the joke was too subtle. I was trying to suggest that the next level of bot will be even better than DeepStack, if that is indeed the best in the world right now. FWIW, I've been warning about the imminent #RobotTakeover for years.
I can't fault computer science researchers for doing their jobs (machine learning is a fascinating subject, and it's going to change the world massively), but it disturbs me that the AI experts at CMU and Alberta Uni are kind of embroiled in an arm's race to see which team can claim to have killed poker first.

Robot > ******

Hi guys, my name is Victor Santos from Brazil and I took part in this human v AI poker test. While I am not a HU specialist, I have played a reasonable amount in this format and consider myself a decent HU reg, as evidenced by my above average win rate on Pokerstars under the screen name OtB_RedBaron. I can confirm this bot is the real deal. In fact, it beat the crap out of me to the tune of those figures up there ^^^ (I don't understand them, do you?). Some of the moves were really sick, including a soul reading check back on the river with a str8 flush v my royal. I would urge you guys to steer clear of this and all bots.

gl

if you average the 10 players above the line, which i think is 10 of the 11 players who completed the challenge, it's 35 bb/100 loss.

BUT, 2 players account for most of that...... one of those two players is losing over 200bb/100. don/t have it in front of me anymore but it was either 214bb/100 or 244bb/100..

still think the bot is a huge winner overall.

would have to check if players who didn't do well just quit i.e. are the complete records more skewed to losses

yes, it looks like the incomplete players were highly skewed towards losses. so they were losing and quit

I'm confused. It looks like positive numbers are favorable to the bot, negative numbers are favorable to the player. So in terms of chips (not AIVAT) weren't you actually winning by a lot after the 58 hands you played?

People seem to think that it's trivial to win at over 40bb/100. But Let me remind you that even the best bumhunter in the world, ie. Kaintd aka Retej on Stars, can barely win at such a winrate against ******s. The bot wins at this rate against presumably non-******s without even trying to exploit anyone, which makes it incredibly impressive in my book.

The bot also doesn't have to pay rake.

It's actually not hard at all to have a massive winrate HU in a rakeless environment against bad players.

I mean sure it's definitely not something to be ignored, but I'll wait until I see it playing 50/100 regs and winning to say a bot beats pros.

Rake is like what? 10bb/100 at midstakes? Probably even less, so still an impressive winrate for the bot.

If the "pros" are only getting paid by beating the bot for the most $, they're going to have to gamble in -EV spots for it... or at least think they do, looking at the chart, maybe not heh.

https://www.technologyreview.com/s/6...s_1&set=603353

MIT's take on the AI

^ "Poker Is the Latest Game to Fold Against Artificial Intelligence".

Inb4 variations of that headline appear on the BBC and in the Daily Mail.

lol yeah this pretty much


A few questions

1) why did they "tune" the bot for 200bb instead of 100bb where 99% of action is happening and where all the top pros excel at?

2) why are they only 2-tabling since they know the bot is gonna take so much time on turns? The pace of the game could be better, for them and for the observers as well of they played more tables

3) do they share the hand histories with human players after each day to analyze since Libratus is obv adjusting/refining its game overnight?

4) although he played his solid game today after watching him for hours, why did jason les just decided to randomly punt off chips in last ~5-10 hands? Not that i am in a position to judge his play throughout these hands, i am 1000x worse overall, but this was just insane. I get that you are exhausted and just want it to be over with, but damn... you have 5 hands to finish this

5) the professor seems to have no clue how to play. I mean, i guess he does not have to have expert knowlege about a specific game he is developing a bot against, but you would expect him to at least beat a micro stakes reg to get an overall grasp on how the game works

Overall it just seems the whole competition format gives a much worse winrate to the Brains than they would have playing in their natural state/enviroment in online games

How does DeepStack deal with actions other than the ones it models - F,C,2P,3P,A?