Two Plus Two Publishing LLC Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

News, Views, and Gossip For poker news, views and gossip.

Reply
 
Thread Tools Display Modes
Old 05-10-2015, 12:49 AM   #1451
polarizeddeck
newbie
 
Join Date: Apr 2015
Posts: 38
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ChicagoRy View Post
Based on the results, what is the % chance that the players are better than the bot?
This depends on your prior beliefs about what edge the players had on the bot before the competition (see Bayes's theorem). However, under a reasonable unbiased prior this would have to be 90+%.
polarizeddeck is offline   Reply With Quote
Old 05-10-2015, 12:55 AM   #1452
ChicagoRy
 
ChicagoRy's Avatar
 
Join Date: Jan 2007
Location: ColoradoRy
Posts: 20,494
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by polarizeddeck View Post
This depends on your prior beliefs about what edge the players had on the bot before the competition (see Bayes's theorem). However, under a reasonable unbiased prior this would have to be 90+%.
Wouldn't it just be "If you beat your opponent for x bb/100 over y sample, you are z% likely to be better than .1bb/100 winrate vs them?"

That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.
ChicagoRy is offline   Reply With Quote
Old 05-10-2015, 12:57 AM   #1453
Kirbynator
Carpal \'Tunnel
 
Kirbynator's Avatar
 
Join Date: Aug 2006
Posts: 52,005
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ike View Post
If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
Yeah seriously, this is the thing.

If the bot only lost 1.67 million dollahs they would have said the bot lost less than 1% of the money wagered!!!1 and it's insignificant!
Kirbynator is offline   Reply With Quote
Old 05-10-2015, 01:06 AM   #1454
.isolated
Carpal \'Tunnel
 
Join Date: May 2012
Posts: 21,341
Thumbs down Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ChicagoRy View Post
That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.
I see 92% and 95% thrown around but let's just say that it's 100% because that's what it is.
.isolated is offline   Reply With Quote
Old 05-10-2015, 01:07 AM   #1455
polarizeddeck
newbie
 
Join Date: Apr 2015
Posts: 38
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ChicagoRy View Post
Wouldn't it just be "If you beat your opponent for x bb/100 over y sample, you are z% likely to be better than .1bb/100 winrate vs them?"

That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.
This question actually can't be answered without a well defined prior distribution (although that prior can be "unbiased" giving no more weight to the bot or the humans).

The 95% confidence interval sometimes described that doesn't use "prior information" is not really the same thing. It's the realization of a random variable that contains the true win rate 95% of the time (this is the frequentist approach which is kinda unintuitive)
polarizeddeck is offline   Reply With Quote
Old 05-10-2015, 01:10 AM   #1456
tenderloinig
adept
 
Join Date: Dec 2013
Posts: 803
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ike View Post
If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
were you offered one of the 4 player spots in this challenge by any chance?
tenderloinig is offline   Reply With Quote
Old 05-10-2015, 01:20 AM   #1457
tenderloinig
adept
 
Join Date: Dec 2013
Posts: 803
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

watched doug's video. The bot's card removal issues are due to bucketing, no?
tenderloinig is offline   Reply With Quote
Old 05-10-2015, 01:32 AM   #1458
Frankie Fuzz
grinder
 
Join Date: Aug 2013
Posts: 534
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by polarizeddeck View Post
This question actually can't be answered without a well defined prior distribution (although that prior can be "unbiased" giving no more weight to the bot or the humans).

The 95% confidence interval sometimes described that doesn't use "prior information" is not really the same thing. It's the realization of a random variable that contains the true win rate 95% of the time (this is the frequentist approach which is kinda unintuitive)
To better illustrate this, assume that a group of universities are conducting studies to determine if psychics can predict whether a fair coin will be heads or tails. The first 9 studies show no significant ability. The 10th shows that the psychic had a statistically significant ability at the 95% confidence level. It would be incorrect to assume that the chance of the psychic being a real psychic is 95%. You must take into account priors (in this case previous experiments).
Frankie Fuzz is offline   Reply With Quote
Old 05-10-2015, 03:40 AM   #1459
psloth
newbie
 
Join Date: Jun 2011
Posts: 26
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by cheet View Post
Also I recorded this for 2p2 because I want to share what I thought was a great talk by Doug at the closing press conference of the challenge. He outlined everything quite well and I agree with everything he had to say.

https://www.youtube.com/watch?v=L10zaLtUabY
Thanks Jason.

I watched the full press conference yesterday, it can be found here: http://www.twitch.tv/claudico_extra/v/4813763. Starts at 2h11min.
psloth is offline   Reply With Quote
Old 05-10-2015, 05:15 AM   #1460
TimTamBiscuit
veteran
 
TimTamBiscuit's Avatar
 
Join Date: Oct 2007
Location: cRUSHed!!!!!!
Posts: 2,134
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by cheet View Post
I understand where the CMU public relations team is coming from when they call this a "statistical tie". However, I feel like the degree to which that (and additionally the "win vs money wagered" comparison used) has been pushed in the press release that was distributed to media outlets went too far and is being used to color this more favorably for their self-interest at the expense of the human's accomplishment here.

I also want to say that coming down the stretch in this challenge my personal results vs Claudico were often held as a trophy to the media as if it were evidence that Claudico was beating an individual, despite the format and spirit of this competition being a "team". I don't recall the term "statistical tie" ever being used then.
Jason
Indeed, statistically you may well have beaten the AI (that is your result is within the variance bounds of your true winrate being >0 vs the AI).

As the challenge continued Doug said he thought the Bot's adjustments were reducing the overall human winrate but for you the trend seemed the other way: you were winning more vs the machine as the challenge continued. You made a statement at one stage that you felt "you now had the AI figured out" or words to that effect.

What is your belief now? Were your adjustments increasing your edge vs the AI? Can you share with us what sort of mistakes the AI made that you were exploiting?
TimTamBiscuit is offline   Reply With Quote
Old 05-10-2015, 05:20 AM   #1461
TimTamBiscuit
veteran
 
TimTamBiscuit's Avatar
 
Join Date: Oct 2007
Location: cRUSHed!!!!!!
Posts: 2,134
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Given the "statistical tie" humans should form another Corporation (the team that played billionaire Andy Beal) and play CMU's Claudico HU4RollZ with all the assets of CMU on the line. I think the Corporation would bankrupt CMU. I think the Prof would be the first to decline because he knows in his heart he lost and lost badly.

Poker players don't have statistical ties they have HU4Rollz!
TimTamBiscuit is offline   Reply With Quote
Old 05-10-2015, 05:24 AM   #1462
Land O Lakes
Carpal \'Tunnel
 
Land O Lakes's Avatar
 
Join Date: Nov 2005
Posts: 6,373
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by TimTamBiscuit View Post
Given the "statistical tie" humans should form another Corporation (the team that played billionaire Andy Beal) and play CMU's Claudico HU4RollZ with all the assets of CMU on the line. I think the Corporation would bankrupt CMU. I think the Prof would be the first to decline because he knows in his heart he lost and lost badly.

Poker players don't have statistical ties they have HU4Rollz!
Have another drink.
Land O Lakes is offline   Reply With Quote
Old 05-10-2015, 05:27 AM   #1463
TimTamBiscuit
veteran
 
TimTamBiscuit's Avatar
 
Join Date: Oct 2007
Location: cRUSHed!!!!!!
Posts: 2,134
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Sam Ganzfried:
Quote:
"I think we can have a new program that can beat the humans in a year."
http://www.smh.com.au/world/man-best...09-ggxwuz.html
TimTamBiscuit is offline   Reply With Quote
Old 05-10-2015, 07:11 AM   #1464
27LB
Bigfoot
 
27LB's Avatar
 
Join Date: Jun 2009
Location: Split, Croatia
Posts: 714
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Durrrr doesn't owe any money to Jungleman.
Spoiler:
27LB is offline   Reply With Quote
Old 05-10-2015, 09:42 AM   #1465
Camus
centurion
 
Camus's Avatar
 
Join Date: Jan 2013
Location: USA
Posts: 151
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by ike View Post
If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
lol this.

also, one-table grinding for 8-10 hours/day over two weeks isn't exactly optimal conditions for human poker players. i doubt they saw much else other than the interior of the rivers casino. truncate the time table a bit, give claudico the CPU power to at least two-table (ideally four), and let the human side have HUD stats and let's see how much more "significant" that edge might have been.
Camus is offline   Reply With Quote
Old 05-10-2015, 11:55 AM   #1466
Kirbynator
Carpal \'Tunnel
 
Kirbynator's Avatar
 
Join Date: Aug 2006
Posts: 52,005
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

lot more likely humans would lose if they 4tabled the bot
Kirbynator is offline   Reply With Quote
Old 05-10-2015, 11:59 AM   #1467
Camus
centurion
 
Camus's Avatar
 
Join Date: Jan 2013
Location: USA
Posts: 151
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by Kirbynator View Post
lot more likely humans would lose if they 4tabled the bot
any more likely than if they had the mental and physical wear and tear associated with spending two weeks doing nothing but playing one table in public in a casino with bystanders sleeping in a hotel bed spending who knows how many hours outside of actual play revising strategy and parsing hand histories that were not readily provided to them at first?

not so sure about that
Camus is offline   Reply With Quote
Old 05-10-2015, 12:02 PM   #1468
DrRepper
adept
 
DrRepper's Avatar
 
Join Date: Jul 2010
Location: river
Posts: 756
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

it would be pretty satisfying if the human team spoke up to the media and called bs
DrRepper is offline   Reply With Quote
Old 05-10-2015, 12:10 PM   #1469
punter11235
Carpal \'Tunnel
 
Join Date: Mar 2005
Location: solving poker
Posts: 8,206
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
The first approach for computing exploitability in LHE was just developed a few years ago, and involved sophisticated techniques to be feasible, http://webdocs.cs.ualberta.ca/~bowli...ijcai-rgbr.pdf.
It's truly fascinating how you can come up with a game representation with 8 orders of magnitude more nodes to visit than needed. No wonder everything is untrackable in "state of the art" world

Last edited by punter11235; 05-10-2015 at 12:35 PM.
punter11235 is offline   Reply With Quote
Old 05-10-2015, 12:21 PM   #1470
Mecastyles
Pooh-Bah
 
Mecastyles's Avatar
 
Join Date: Oct 2007
Location: Hyper, Hyper
Posts: 4,660
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by Sam Ganzfried View Post
I did not make any comments regarding what confidence interval to use, whether it was a "tie," etc. If other people from CMU or the media want to claim it was a "statistical tie" that is their prerogative.
You can atleast say it is ****ing ridiculous statement.

used to look up to the beserious


Mecastyles is offline   Reply With Quote
Old 05-10-2015, 12:36 PM   #1471
lostinthesaus
old hand
 
lostinthesaus's Avatar
 
Join Date: Feb 2011
Posts: 1,520
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

A bot only losing 9 bb/100 to this team is very significant and should be taken very seriously. I think it will only get exponentially better from here on out both in speed of improvement and actual game improvement, especially with that Microsoft money backing it.

Would not be surprised to learn of many new poker think-tanks that have formed and existing ones newly inspired to start building/improving their own bots after this. Personally starting to feel the beginning of the true end of online poker. Win-rates sure to plummet as these bots improve. Legislation will likely be slowed for having to factor in bot use, penalties, TOC violations etc.

This was definitely a victory for the bot side of things. If not a victory for the bot, definitely a loss for the humans.

Live poker on the up-and-up. Who knew these live tourney donks had it right all along?

Very good article about AI I found extremely informative and appropriate for this thread:

http://waitbutwhy.com/2015/01/artifi...olution-1.html
http://waitbutwhy.com/2015/01/artifi...olution-2.html
lostinthesaus is offline   Reply With Quote
Old 05-10-2015, 12:38 PM   #1472
Kirbynator
Carpal \'Tunnel
 
Kirbynator's Avatar
 
Join Date: Aug 2006
Posts: 52,005
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by lostinthesaus View Post
Live poker on the up-and-up. Who knew these live tourney donks had it right all along?
wat
Kirbynator is offline   Reply With Quote
Old 05-10-2015, 01:29 PM   #1473
cheet
veteran
 
cheet's Avatar
 
Join Date: Jan 2006
Posts: 2,498
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
Originally Posted by TimTamBiscuit View Post
Jason
Indeed, statistically you may well have beaten the AI (that is your result is within the variance bounds of your true winrate being >0 vs the AI).

As the challenge continued Doug said he thought the Bot's adjustments were reducing the overall human winrate but for you the trend seemed the other way: you were winning more vs the machine as the challenge continued. You made a statement at one stage that you felt "you now had the AI figured out" or words to that effect.

What is your belief now? Were your adjustments increasing your edge vs the AI? Can you share with us what sort of mistakes the AI made that you were exploiting?
I agree with Doug that the Bot's adjustments were reducing the overall human winrate. My winrate down the last stretch of 10k hands or so was pretty good (Don't have it off hand) but obviously that is a very small sample that is not significant. This was me hitting the better side of variance and on a trajectory to realize my true winrate over a larger sample.

I believe at present I would have a good winrate over Claudico but I'd rather not specifically share exploits about the A.I. outside of what Doug said in his statement. I will say though that the biggest thing for me was realizing the A.I. didn't think like a human. It sounds very obvious but after so many hands of trying to perfect a skill of thinking how someone will react to my action in an exploitative way, it was hard to turn off that side of my brain.

It also started off very hard to win with sets
cheet is offline   Reply With Quote
Old 05-10-2015, 01:34 PM   #1474
ArtyMcFly
Carpal \'Tunnel
 
ArtyMcFly's Avatar
 
Join Date: Dec 2014
Location: Enchantment Under the Sea
Posts: 13,251
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot



More science...
Spoiler:
ArtyMcFly is offline   Reply With Quote
Old 05-10-2015, 01:41 PM   #1475
Andre_787
old hand
 
Andre_787's Avatar
 
Join Date: Oct 2010
Location: Croatia
Posts: 1,983
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

ahhahaha brilliant
Andre_787 is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 02:24 AM.


Powered by vBulletin®
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
Copyright © 2008-2020, Two Plus Two Interactive