https://arxiv.org/pdf/1701.01724.pdf

I recognized one guy, and it is Phil Laak.

so maybe someone explain AIVAT (mbb/g) and mbb/g in real terms for all us amateurs out here (99% of 2=2'ers)
Thanks

I can see they got some real HU crushers to play the bots for these 3k samples

Take Mike Phan out and results are way different.

would love to see the mike phan hands

Googled a random name there since Laak was the only name I recognized.

Got this: https://twitter.com/muskaansethi

Based on my relevant sample of 2 I'm going to hazard a guess that basically none of them could beat 50NL online. At least I learned Muskan is a female name.

FWIW, several of the players are MTT specialists, not HU cashgamers.

We'll see if the DeepStack bot is still around in five years.

The Irish duo of Fintan Gavin and Dara O'Kearney are both well-known in Europe and good players, with $1.5million and $900,000 respectively in live tournament winnings on HendonMob. Neither heads-up specialists afaik though.

Not sure what the sarcasm is about, although perhaps you don't understand the online poker landscape - bots will definitely be around in 5 years and they will be much stronger than they are today.

The incentive for players to play was 5k CAD for whoever performed best, $2.5k CAD for whoever performed second best and 1.25k CAD for whoever performed thirst best.

That's not the kind of incentive to bring on the best of the best.

the mbb/g is just mili BB per game. A game is 1 hand.

So to convert just divide by 10. So 486 mBB/g = 48.6 BB/100.

I'm not familiar with the technique applied evaluate the performance (AIVAT), so if someone better than me at statistics could please follow I would appreciate. Although it seems a little suspicious that they can achieve significant performance results with sample sizes as small as they did (3k hands per player at best).

EDIT: HH will be made available online. I'll try to find them or see when they are gonna be posted.

I've had a quick look online and skimmed through what I found.

- AIVAT = action-informed value assessment tool
AIVAT - purpose: to lower the variance, to achieve 'the same statistical significance with ten times less data'.
Because it is impractical to play the actual number of hands necessary for statistical significance with a human player.

- mbb/g = milli-big-blinds per game
'a milli-big-blind is one thousandth of the size of the big blind bet amount.'
I think this is their way of measuring the win rate. I'm guessing a 'game' is one hand.

link to the Artificial Intelligence paper: https://arxiv.org/pdf/1612.06915v1.pdf

lmao @ Mike Phan

Live tournament players are generally awful compared to any online cash game player that wins, so not sure what difference that makes.... (I said generally, there is obviously a few exceptions)

I'll take a 50NL Russian HU player (non bot, of course) VS this bot any day.

Y'all need to give Mike Phan a break, it says he only played 32 hands.

With a sample size that small the variance is astronomical. It says the 95% ci for his true winrate is -11223 +/- 18235, which means there is a 95% chance his true winrate vs this computer is somewhere between -2955 BB/100 and +700BB/100. Which is basically a meaningless statistic.

Also his 32 hands won't sway the overall results by much with other players playing multi thousands of hands.

Nice try Mike Phan

Wait we're looking at the results backwards. The numbers are the computer's wins, not the humans's. A positive # means the computer won and viceversa. So Phan pulled a hit and run winning session, maybe hoping everyone else would lose/ do worse.

Doesn't the negative value imply that the player won (meaning the lower the better they performanced)? The fact that the players who played 3k hands are ranked on the basis of how low their AIVAT number is suggests so.

Lol I don't know who he is, I'm just trying to clarify the stats. 32 hands is literally (literally) statistically meaningless

. And apparently Phan won at this game

Yeah + numbers are good for the bot; - good for the humans. Mike Phan = GOAT

Winning over 40k hands at 4.5bb/100 in an environment that controls for variance somewhat is not negligible, but again I am highly doubtful of the quality of opposition the bot faced.

You win this time Mike Phan, but this is far from over

I was just responding to the 'googled because I only recognised Phil Laak's name' comment - guess I should have quoted it for clarity.

Only recognise two players on the list and neither are HU specialists. Also seems like insignificant sample size.

Paging Doug Polk, can you beat up the bot for us please

They get the result from (AIVAT column * hands column) / total hands played.

21798823 / 44852 = ~ 486 mbb/g

Our friend Mike Phan needs to play more hands to make a difference.

If you remove just 4 of the weaker players in this experiment:
muskan sethi,
pol dmit,
youwei qin,
geidrius talacka

it brings down the number to:

11832392 / 35579 = ~ 332 mbb/g

Although there may be professional poker players in the list, I don't think they are all NLHU specialists.
It's possible the people in the list may be very good (or 'professionals' if you like) at their chosen format of poker and at the same time not good at NLHU. (So I'm saying nothing against them, I don't know them. I've heard of Phil Laak though, he's cool )
What I'm saying is the people doing this A.I. research project can easily skew their results any way they want by getting in the wrong type of people (and then claiming it beats professionals).

Also I don't believe their equations can possibly be accounting for the fact, that good players will adapt.
Anyway probably a good thing if they get their experiment wrong, we really don't need any A.I. bots destroying the game.
(can't they do something more useful with A.I. anyway, like solve unlimited renewable energy or find medical cures?)

(TL;DR - researchers can skew their own results & should really spend their time on more important projects)

OtB's name isn't public on the other hand, so he might have played and got crushed by this bot ... #thiscouldbetheend #