Two Plus Two Poker Forums WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot
 Register FAQ Search Today's Posts Mark Forums Read TwoPlusTwo.com

 News, Views, and Gossip For poker news, views and gossip.

05-10-2015, 12:49 AM   #1451
polarizeddeck
newbie

Join Date: Apr 2015
Posts: 38
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ChicagoRy Based on the results, what is the % chance that the players are better than the bot?
This depends on your prior beliefs about what edge the players had on the bot before the competition (see Bayes's theorem). However, under a reasonable unbiased prior this would have to be 90+%.

05-10-2015, 12:55 AM   #1452
ChicagoRy

Join Date: Jan 2007
Posts: 20,494
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by polarizeddeck This depends on your prior beliefs about what edge the players had on the bot before the competition (see Bayes's theorem). However, under a reasonable unbiased prior this would have to be 90+%.
Wouldn't it just be "If you beat your opponent for x bb/100 over y sample, you are z% likely to be better than .1bb/100 winrate vs them?"

That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.

05-10-2015, 12:57 AM   #1453
Kirbynator
Carpal \'Tunnel

Join Date: Aug 2006
Posts: 52,005
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ike If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
Yeah seriously, this is the thing.

If the bot only lost 1.67 million dollahs they would have said the bot lost less than 1% of the money wagered!!!1 and it's insignificant!

05-10-2015, 01:06 AM   #1454
.isolated
Carpal \'Tunnel

Join Date: May 2012
Posts: 21,341
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ChicagoRy That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.
I see 92% and 95% thrown around but let's just say that it's 100% because that's what it is.

05-10-2015, 01:07 AM   #1455
polarizeddeck
newbie

Join Date: Apr 2015
Posts: 38
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ChicagoRy Wouldn't it just be "If you beat your opponent for x bb/100 over y sample, you are z% likely to be better than .1bb/100 winrate vs them?" That's what I'm asking. How likely is it that the players are better than the bot given the results of this challenge. I don't think prior information is relevant to that question.
This question actually can't be answered without a well defined prior distribution (although that prior can be "unbiased" giving no more weight to the bot or the humans).

The 95% confidence interval sometimes described that doesn't use "prior information" is not really the same thing. It's the realization of a random variable that contains the true win rate 95% of the time (this is the frequentist approach which is kinda unintuitive)

05-10-2015, 01:10 AM   #1456
tenderloinig

Join Date: Dec 2013
Posts: 803
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ike If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
were you offered one of the 4 player spots in this challenge by any chance?

 05-10-2015, 01:20 AM #1457 tenderloinig adept   Join Date: Dec 2013 Posts: 803 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot watched doug's video. The bot's card removal issues are due to bucketing, no?
05-10-2015, 01:32 AM   #1458
Frankie Fuzz
grinder

Join Date: Aug 2013
Posts: 534
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by polarizeddeck This question actually can't be answered without a well defined prior distribution (although that prior can be "unbiased" giving no more weight to the bot or the humans). The 95% confidence interval sometimes described that doesn't use "prior information" is not really the same thing. It's the realization of a random variable that contains the true win rate 95% of the time (this is the frequentist approach which is kinda unintuitive)
To better illustrate this, assume that a group of universities are conducting studies to determine if psychics can predict whether a fair coin will be heads or tails. The first 9 studies show no significant ability. The 10th shows that the psychic had a statistically significant ability at the 95% confidence level. It would be incorrect to assume that the chance of the psychic being a real psychic is 95%. You must take into account priors (in this case previous experiments).

05-10-2015, 03:40 AM   #1459
psloth
newbie

Join Date: Jun 2011
Posts: 26
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by cheet Also I recorded this for 2p2 because I want to share what I thought was a great talk by Doug at the closing press conference of the challenge. He outlined everything quite well and I agree with everything he had to say. https://www.youtube.com/watch?v=L10zaLtUabY
Thanks Jason.

I watched the full press conference yesterday, it can be found here: http://www.twitch.tv/claudico_extra/v/4813763. Starts at 2h11min.

05-10-2015, 05:15 AM   #1460
TimTamBiscuit
veteran

Join Date: Oct 2007
Location: cRUSHed!!!!!!
Posts: 2,134
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by cheet I understand where the CMU public relations team is coming from when they call this a "statistical tie". However, I feel like the degree to which that (and additionally the "win vs money wagered" comparison used) has been pushed in the press release that was distributed to media outlets went too far and is being used to color this more favorably for their self-interest at the expense of the human's accomplishment here. I also want to say that coming down the stretch in this challenge my personal results vs Claudico were often held as a trophy to the media as if it were evidence that Claudico was beating an individual, despite the format and spirit of this competition being a "team". I don't recall the term "statistical tie" ever being used then.
Jason
Indeed, statistically you may well have beaten the AI (that is your result is within the variance bounds of your true winrate being >0 vs the AI).

As the challenge continued Doug said he thought the Bot's adjustments were reducing the overall human winrate but for you the trend seemed the other way: you were winning more vs the machine as the challenge continued. You made a statement at one stage that you felt "you now had the AI figured out" or words to that effect.

What is your belief now? Were your adjustments increasing your edge vs the AI? Can you share with us what sort of mistakes the AI made that you were exploiting?

 05-10-2015, 05:20 AM #1461 TimTamBiscuit veteran     Join Date: Oct 2007 Location: cRUSHed!!!!!! Posts: 2,134 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot Given the "statistical tie" humans should form another Corporation (the team that played billionaire Andy Beal) and play CMU's Claudico HU4RollZ with all the assets of CMU on the line. I think the Corporation would bankrupt CMU. I think the Prof would be the first to decline because he knows in his heart he lost and lost badly. Poker players don't have statistical ties they have HU4Rollz!
05-10-2015, 05:24 AM   #1462
Land O Lakes
Carpal \'Tunnel

Join Date: Nov 2005
Posts: 6,373
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by TimTamBiscuit Given the "statistical tie" humans should form another Corporation (the team that played billionaire Andy Beal) and play CMU's Claudico HU4RollZ with all the assets of CMU on the line. I think the Corporation would bankrupt CMU. I think the Prof would be the first to decline because he knows in his heart he lost and lost badly. Poker players don't have statistical ties they have HU4Rollz!
Have another drink.

05-10-2015, 05:27 AM   #1463
TimTamBiscuit
veteran

Join Date: Oct 2007
Location: cRUSHed!!!!!!
Posts: 2,134
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Sam Ganzfried:
Quote:
 "I think we can have a new program that can beat the humans in a year."
http://www.smh.com.au/world/man-best...09-ggxwuz.html

 05-10-2015, 07:11 AM #1464 27LB Bigfoot     Join Date: Jun 2009 Location: Split, Croatia Posts: 714 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot Durrrr doesn't owe any money to Jungleman. Spoiler: IT'S A TIE
05-10-2015, 09:42 AM   #1465
Camus
centurion

Join Date: Jan 2013
Location: USA
Posts: 151
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by ike If this result doesn't qualify as statistically significant, how likely was this challenge to generate a statistically significant result?
lol this.

also, one-table grinding for 8-10 hours/day over two weeks isn't exactly optimal conditions for human poker players. i doubt they saw much else other than the interior of the rivers casino. truncate the time table a bit, give claudico the CPU power to at least two-table (ideally four), and let the human side have HUD stats and let's see how much more "significant" that edge might have been.

 05-10-2015, 11:55 AM #1466 Kirbynator Carpal \'Tunnel     Join Date: Aug 2006 Posts: 52,005 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot lot more likely humans would lose if they 4tabled the bot
05-10-2015, 11:59 AM   #1467
Camus
centurion

Join Date: Jan 2013
Location: USA
Posts: 151
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by Kirbynator lot more likely humans would lose if they 4tabled the bot
any more likely than if they had the mental and physical wear and tear associated with spending two weeks doing nothing but playing one table in public in a casino with bystanders sleeping in a hotel bed spending who knows how many hours outside of actual play revising strategy and parsing hand histories that were not readily provided to them at first?

 05-10-2015, 12:02 PM #1468 DrRepper adept     Join Date: Jul 2010 Location: river Posts: 756 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot it would be pretty satisfying if the human team spoke up to the media and called bs
05-10-2015, 12:10 PM   #1469
punter11235
Carpal \'Tunnel

Join Date: Mar 2005
Location: solving poker
Posts: 8,206
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 The first approach for computing exploitability in LHE was just developed a few years ago, and involved sophisticated techniques to be feasible, http://webdocs.cs.ualberta.ca/~bowli...ijcai-rgbr.pdf.
It's truly fascinating how you can come up with a game representation with 8 orders of magnitude more nodes to visit than needed. No wonder everything is untrackable in "state of the art" world

Last edited by punter11235; 05-10-2015 at 12:35 PM.

05-10-2015, 12:21 PM   #1470
Mecastyles
Pooh-Bah

Join Date: Oct 2007
Location: Hyper, Hyper
Posts: 4,660
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by Sam Ganzfried I did not make any comments regarding what confidence interval to use, whether it was a "tie," etc. If other people from CMU or the media want to claim it was a "statistical tie" that is their prerogative.
You can atleast say it is ****ing ridiculous statement.

used to look up to the beserious

 05-10-2015, 12:36 PM #1471 lostinthesaus old hand     Join Date: Feb 2011 Posts: 1,520 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot A bot only losing 9 bb/100 to this team is very significant and should be taken very seriously. I think it will only get exponentially better from here on out both in speed of improvement and actual game improvement, especially with that Microsoft money backing it. Would not be surprised to learn of many new poker think-tanks that have formed and existing ones newly inspired to start building/improving their own bots after this. Personally starting to feel the beginning of the true end of online poker. Win-rates sure to plummet as these bots improve. Legislation will likely be slowed for having to factor in bot use, penalties, TOC violations etc. This was definitely a victory for the bot side of things. If not a victory for the bot, definitely a loss for the humans. Live poker on the up-and-up. Who knew these live tourney donks had it right all along? Very good article about AI I found extremely informative and appropriate for this thread: http://waitbutwhy.com/2015/01/artifi...olution-1.html http://waitbutwhy.com/2015/01/artifi...olution-2.html
05-10-2015, 12:38 PM   #1472
Kirbynator
Carpal \'Tunnel

Join Date: Aug 2006
Posts: 52,005
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by lostinthesaus Live poker on the up-and-up. Who knew these live tourney donks had it right all along?
wat

05-10-2015, 01:29 PM   #1473
cheet
veteran

Join Date: Jan 2006
Posts: 2,498
Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot

Quote:
 Originally Posted by TimTamBiscuit Jason Indeed, statistically you may well have beaten the AI (that is your result is within the variance bounds of your true winrate being >0 vs the AI). As the challenge continued Doug said he thought the Bot's adjustments were reducing the overall human winrate but for you the trend seemed the other way: you were winning more vs the machine as the challenge continued. You made a statement at one stage that you felt "you now had the AI figured out" or words to that effect. What is your belief now? Were your adjustments increasing your edge vs the AI? Can you share with us what sort of mistakes the AI made that you were exploiting?
I agree with Doug that the Bot's adjustments were reducing the overall human winrate. My winrate down the last stretch of 10k hands or so was pretty good (Don't have it off hand) but obviously that is a very small sample that is not significant. This was me hitting the better side of variance and on a trajectory to realize my true winrate over a larger sample.

I believe at present I would have a good winrate over Claudico but I'd rather not specifically share exploits about the A.I. outside of what Doug said in his statement. I will say though that the biggest thing for me was realizing the A.I. didn't think like a human. It sounds very obvious but after so many hands of trying to perfect a skill of thinking how someone will react to my action in an exploitative way, it was hard to turn off that side of my brain.

It also started off very hard to win with sets

 05-10-2015, 01:34 PM #1474 ArtyMcFly Carpal \'Tunnel     Join Date: Dec 2014 Location: Enchantment Under the Sea Posts: 13,251 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot More science... Spoiler:
 05-10-2015, 01:41 PM #1475 Andre_787 old hand     Join Date: Oct 2010 Location: Croatia Posts: 1,983 Re: WCGRider, Dong Kim, Jason Les and Bjorn Li to play against a new HU bot ahhahaha brilliant

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home Poker News & Discussion     News, Views, and Gossip     Poker Blogs and Goals     Poker Beats, Brags, and Variance     YouTube Podcasts & Twitch Streams     General Poker Discussion Online Poker Sites & Marketplaces     Online Poker Sites         Discussion of Poker Sites         Global Poker         BetOnline.ag     Coaches & Schools         Seeking Coaching         Study Groups         General Coaching & Schools Discussion     Staking         Offering Stakes         Seeking Stakes         Selling Shares - Live         Selling Shares - Online         Staking Rails     Poker Software         General Software Discussion     General Marketplace     Transaction Feedback & Disputes Live Poker     Las Vegas Lifestyle     Venues & Communities     Tournament Events         WPT.com     Home Poker     Casino & Cardroom Poker Poker Strategy     Live No-Limit Hold’em Cash     Online No-Limit Hold’em Cash     No Limit Tournaments         Heads Up SNG and Spin and Gos     Mid-High Stakes MTT     Omaha         Omaha/8     Other Poker Games         Mid-High Stakes Limit         Micro-Small Stakes Limit         Stud     Psychology     Books and Publications     Poker Theory & GTO     Beginners and General Questions 2+2 Communities     Other Other Topics         OOTV     The Lounge: Discussion+Review     BBV4Life         omg omg omg     House of Blogs Sports and Games     Sporting Events         Single-Team Season Threads         Fantasy Sports     Sports Betting     Fantasy Sports         Sporting Events     Wrestling     Golf     Chess and Other Board Games         Backgammon Forum hosted by Bill Robertie.     Video Games         League of Legends         Hearthstone     Puzzles and Other Games Other Topics     Politics and Society     Business, Finance, and Investing     History     Health and Fitness     Travel     Science, Math, and Philosophy     Religion, God, and Theology     Laughs or Links!     Probability     Other Gambling Games     Computer and Technical Help Two Plus Two     About the Forums

All times are GMT -4. The time now is 02:24 AM.

 Contact Us - Two Plus Two Publishing LLC - Privacy Statement - Top