Two Plus Two Publishing LLC
Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

News, Views, and Gossip For poker news, views, and gossip

Reply
 
Thread Tools Display Modes
Old 12-02-2018, 12:43 PM   #1
robert_utk
Pooh-Bah
 
robert_utk's Avatar
 
Join Date: Jan 2005
Location: ValueTown
Posts: 4,129
So, when will Deepmind AlphaZero play poker versus Libratus?

News: Google AI called Deepmind with AlphaZero self-learning algorithm is the most powerful and highly rated entity to have ever played the game of chess, AINEC.

View: Google Deepmind, after only 12 hours of learning Holdem poker, would beat Libratus (the strongest poker engine) for >10bb/100.

Gossip: So why has this not already happened? Does Google simply not care enough about poker? Is there a potential weakness for Deepmind when the game features incomplete information (highly doubtful IMO)?

Or, does Google not desire to put an entire generation of Computer Scientists out of a job, given that poker is the most popular way for those scientists to study AI and earn a paycheck?
robert_utk is offline   Reply With Quote
Old 12-02-2018, 12:47 PM   #2
PokerPlayingGamble
old hand
 
Join Date: Mar 2018
Posts: 1,930
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

google has already implemented deepmind poker bots and is crushing all internet poker sites as we speak
PokerPlayingGamble is offline   Reply With Quote
Old 12-02-2018, 02:04 PM   #3
coon74
Carpal \'Tunnel
 
Join Date: Apr 2011
Posts: 10,685
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by robert_utk View Post
Or, does Google not desire to put an entire generation of Computer Scientists out of a job, given that poker is the most popular way for those scientists to study AI and earn a paycheck?
I still trust that Google is socially responsible.
coon74 is offline   Reply With Quote
Old 12-02-2018, 05:17 PM   #4
vanvliet
journeyman
 
Join Date: Aug 2006
Posts: 205
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Agree canít see why a game of incomplete info would be a problem for it
vanvliet is offline   Reply With Quote
Old 12-02-2018, 07:03 PM   #5
SolarAU
old hand
 
SolarAU's Avatar
 
Join Date: Apr 2015
Location: popping bottles w/ the boys
Posts: 1,762
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

I honestly think that if deepmind could be configured to learn poker it'd be a pretty long stretch to imagine it beating Libratus for over 10bb/100
SolarAU is offline   Reply With Quote
Old 12-02-2018, 08:08 PM   #6
DooDooPoker
Pooh-Bah
 
Join Date: Mar 2018
Posts: 4,378
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

We need Deep Mind to stream on twitch so us mere mortals can learn the ways of the robot.
DooDooPoker is offline   Reply With Quote
Old 12-03-2018, 02:53 AM   #7
valuecutting
journeyman
 
Join Date: May 2015
Posts: 226
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Hours is not a good metric for training time.

Neural networks currently struggle in games with incomplete info.
valuecutting is offline   Reply With Quote
Old 12-03-2018, 03:33 AM   #8
pucmo
adept
 
Join Date: Mar 2016
Location: Euro
Posts: 743
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

It would need to give us information that is more than we already get from books, GTO softwares and what we can buy. Did we get to see ten chess games or something like that? So, some hand histories perhaps, some percentage perhaps?
pucmo is offline   Reply With Quote
Old 12-03-2018, 04:14 AM   #9
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,192
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by SolarAU View Post
I honestly think that if deepmind could be configured to learn poker it'd be a pretty long stretch to imagine it beating Libratus for over 10bb/100
I believe that it is much easier to determine the max that your computer can be beaten for than to come close to GTO. Alberta did that although I am sure its a lot simpler to do that for limit vs no limit. Anyway would not it be quite possible that Libratus could not be beaten for 10bb with the perfect counterstrategy let alone with perfect GTO which would do worse.
David Sklansky is offline   Reply With Quote
Old 12-03-2018, 04:46 AM   #10
valuecutting
journeyman
 
Join Date: May 2015
Posts: 226
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Yeah in limit there are no abstractions so you can just calculate the ev of the best possible counterstrategy. It isn't possible measure that for something like Libratus because you have the option to not play in the abstraction. I'd guess it's exploitable to some extent in some bizarre lines. One of the key strengths of Libratus though was that it could train overnight to find weaknesses in the abstraction that the players were exploiting.
valuecutting is offline   Reply With Quote
Old 12-03-2018, 05:13 AM   #11
eaglesfaan
grinder
 
Join Date: Oct 2012
Location: Cali
Posts: 456
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Everything Libratus can do Deepmind would do better, easy victory.
eaglesfaan is offline   Reply With Quote
Old 12-03-2018, 11:48 AM   #12
zoogenhiem
old hand
 
Join Date: May 2016
Posts: 1,419
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Generally the top AI's "retire" once they achieve their task. More accurately they are then moved into real practice. For instance, IBM Watson hasn't played Jeopardy since it's big win but has been doing real work in medical records for instance. My understanding is that AlphaZero and AlphaGo are now retired from their competitive careers and are back to being secret, proprietary working programs.

There's nothing really in it for Google to play poker. It's a big deal to be the first program to beat a human in a given activity (see Watson, AlphaGo, Deep Blue, Libratus), not so big a deal to be second or tenth. It's surprising actually then how much press AlphaZero got, although AlphaGo got way more.
zoogenhiem is offline   Reply With Quote
Old 12-03-2018, 12:07 PM   #13
robert_utk
Pooh-Bah
 
robert_utk's Avatar
 
Join Date: Jan 2005
Location: ValueTown
Posts: 4,129
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by zoogenhiem View Post
Generally the top AI's "retire" once they achieve their task. More accurately they are then moved into real practice. For instance, IBM Watson hasn't played Jeopardy since it's big win but has been doing real work in medical records for instance. My understanding is that AlphaZero and AlphaGo are now retired from their competitive careers and are back to being secret, proprietary working programs.

There's nothing really in it for Google to play poker. It's a big deal to be the first program to beat a human in a given activity (see Watson, AlphaGo, Deep Blue, Libratus), not so big a deal to be second or tenth. It's surprising actually then how much press AlphaZero got, although AlphaGo got way more.


Interesting. Thank you. I am sure you already know everything I am saying below, this is just my discussion of the matter:


My interest in the AlphaZero algorithm is that it specifically beat its own predecessor, AlphaGo *because* it was not specifically trained to play the game of Go. It is only given the rules of the game, and learns the game from scratch.

This leads Deepmind AlphaZero to play games such as chess and go in a way that is more familiar to humans, but at a super-human level, that is best described as a highly evolved alien being might play.

So, DMAZ is not simply crunching numbers to evaluate a position. It has a PLAN for every position.

How well would this translate to poker?

That is debatable, since poker *strategy* may in fact be more about raw evaluation of odds, and merely limited by the ability of the human or computer to efficiently evaluate the portion of gametree for expectation.

So, for me to say that the DMAZ will beat Libratus for a meaningful margin, is to say that DMAZ will actually be playing poker, with a plan for every combo, right from the start. This will find weaknesses in the ability of Libratus to survey the tree, with powerful CFR. Further, the weaknesses will be numerous, yet tiny, and beyond the precision of Libratus to detect and correct.
robert_utk is offline   Reply With Quote
Old 12-03-2018, 06:12 PM   #14
David Sklansky
Administrator
 
David Sklansky's Avatar
 
Join Date: Aug 2002
Posts: 15,192
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by valuecutting View Post
Yeah in limit there are no abstractions so you can just calculate the ev of the best possible counterstrategy. It isn't possible measure that for something like Libratus because you have the option to not play in the abstraction.
I do not understand what your wrote.
David Sklansky is offline   Reply With Quote
Old 12-03-2018, 06:19 PM   #15
PeteBlow
Carpal \'Tunnel
 
Join Date: May 2012
Location: Guildford, UK
Posts: 9,359
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

I would be amazed if Deepmind haven't tried poker. The founder of Deepmind, Demis Hassabis, is not a stranger to the WSOP.

http://pokerdb.thehendonmob.com/player.php?a=r&n=42073
PeteBlow is offline   Reply With Quote
Old 12-03-2018, 08:02 PM   #16
Grothendieck
newbie
 
Join Date: Jan 2018
Posts: 38
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

The relative easiness of computing counter strategies comes from the fact that you need only consider pure strategies and that the EV of each individual hand can be maximized in isolation vs. villain's strategy. Even for no-limit not much abstraction would be needed. I think something like requiring bets/raises be a multiple of 0.5bb would make the computation feasible.
Grothendieck is offline   Reply With Quote
Old 12-03-2018, 08:28 PM   #17
Faustfan
grinder
 
Faustfan's Avatar
 
Join Date: Nov 2007
Location: in your head
Posts: 457
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by robert_utk View Post
News: Google AI called Deepmind with AlphaZero self-learning algorithm is the most powerful and highly rated entity to have ever played the game of chess, AINEC.
That statement is highly questionable. AlphaZero played a match against stockfish, which is the strongest chess program existing, but stockfish was severly handicapped during the match. they never played a match with stockfish at full strength, so we dont really have evidence as to what the result would be.
Faustfan is offline   Reply With Quote
Old 12-03-2018, 09:12 PM   #18
DooDooPoker
Pooh-Bah
 
Join Date: Mar 2018
Posts: 4,378
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Who wins OTB or Deep Mind?
DooDooPoker is offline   Reply With Quote
Old 12-03-2018, 11:36 PM   #19
ArtyMcFly
Carpal \'Tunnel
 
ArtyMcFly's Avatar
 
Join Date: Dec 2014
Location: Enchantment Under the Sea
Posts: 11,951
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Amusingly, the latest competitive AI from DeepMind is called AlphaFold, but it has nothing to do with poker. It's 'playing' a much more complicated game of predicting the 3D structures of proteins.
"Itís never been about cracking Go or Atari, itís about developing algorithms for problems exactly like protein folding," Hassabis said. The number of possible protein structures is around a googol cubed, or 1 followed by 300 zeroes. It's a much more complex 'game' than poker.
Quote:
On its first foray into the competition, AlphaFold topped a table of 98 entrants, predicting the most accurate structure for 25 out of 43 proteins, compared with three out of 43 for the second placed team in the same category.
Source: https://www.theguardian.com/science/...es-of-proteins
ArtyMcFly is offline   Reply With Quote
Old 12-04-2018, 05:30 AM   #20
jukofyork
Carpal \'Tunnel
 
jukofyork's Avatar
 
Join Date: Sep 2004
Posts: 11,364
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by PeteBlow View Post
I would be amazed if Deepmind haven't tried poker. The founder of Deepmind, Demis Hassabis, is not a stranger to the WSOP.

http://pokerdb.thehendonmob.com/player.php?a=r&n=42073
Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

Quote:
Johannes Heinrich, David Silver

Many real-world applications can be described as large-scale games of imperfect information. To deal with these challenging domains, prior work has focused on computing Nash equilibria in a handcrafted abstraction of the domain. In this paper we introduce the first scalable end-to-end approach to learning approximate Nash equilibria without prior domain knowledge. Our method combines fictitious self-play with deep reinforcement learning. When applied to Leduc poker, Neural Fictitious Self-Play (NFSP) approached a Nash equilibrium, whereas common reinforcement learning methods diverged. In Limit Texas Holdem, a poker game of real-world scale, NFSP learnt a strategy that approached the performance of state-of-the-art, superhuman algorithms based on significant domain expertise.
David Silver
Quote:
Professor David Silver (dob c.1976) leads the reinforcement learning research group at DeepMind and was lead researcher on AlphaGo.
Juk
jukofyork is offline   Reply With Quote
Old 12-04-2018, 11:29 AM   #21
robert_utk
Pooh-Bah
 
robert_utk's Avatar
 
Join Date: Jan 2005
Location: ValueTown
Posts: 4,129
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

I suppose I am jealous of the chess community, since Chess24 has post game analysis of the world championship match with input from DeepMind (free to view, highly recommended).

I wish us poker players could get to hear how DMAZ would play a certain hand, or a certain range of hands.
robert_utk is offline   Reply With Quote
Old 12-07-2018, 09:27 AM   #22
robert_utk
Pooh-Bah
 
robert_utk's Avatar
 
Join Date: Jan 2005
Location: ValueTown
Posts: 4,129
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by Faustfan View Post
That statement is highly questionable. AlphaZero played a match against stockfish, which is the strongest chess program existing, but stockfish was severly handicapped during the match. they never played a match with stockfish at full strength, so we dont really have evidence as to what the result would be.

My statement has just recently been confirmed. Deepmind has released the results of a few thousand more matches vs Stockfish in different configurations.

There is no doubt, Deepmind AlphaZero is the strongest chess playing entity ever.

So, with regards to poker, again I am puzzled that Deepmind is not entered into the competition for poker AIís.

Clearly self-learning is the future of AI. However, with poker, increasingly efficient CFR has been the key.

Can DMAZ teach itself enough CFR to win?

I think so.
robert_utk is offline   Reply With Quote
Old 12-07-2018, 05:51 PM   #23
pucmo
adept
 
Join Date: Mar 2016
Location: Euro
Posts: 743
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Quote:
Originally Posted by robert_utk View Post
Deepmind has released the results of a few thousand more matches vs Stockfish in different configurations.

There is no doubt, Deepmind AlphaZero is the strongest chess playing entity ever.
It seems so but it is a private lab test and some people are still skeptical.
pucmo is offline   Reply With Quote
Old 12-11-2018, 09:44 PM   #24
samooth
veteran
 
samooth's Avatar
 
Join Date: May 2009
Location: gutrotting
Posts: 3,239
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

http://science.sciencemag.org/content/362/6419/1118

Quote:
Chess, shogi, and Go are highly complex but have a number of characteristics that make them easier for AI systems. The game state is fully observable; all the information needed to make a move decision is visible to the players. Games with partial observability, such as poker, can be much more challenging, although there have been notable successes in games like heads-up no-limit poker (11, 12). Board games are also easy in other important dimensions. For example, they are two-player, zero-sum, deterministic, static, and discrete, all of which makes it easier to perfectly simulate the evolution of the game state through arbitrary sequences of moves. This ability to easily simulate future states makes MCTS, as used in AlphaZero, practical. Multiplayer video games such as StarCraft II (13) and Dota 2 (14) have been proposed as the next game-playing challenges as they are partially observable and have very large state spaces and action sets, creating problems for AlphaZero-like reinforcement learning approaches.
no real incentive for them to play poker, but it would be interesting to get a take on how good A0's appraoch would work for probabilistic imperfect information games.
samooth is offline   Reply With Quote
Old 12-12-2018, 03:10 AM   #25
chocLatee
grinder
 
chocLatee's Avatar
 
Join Date: Nov 2011
Location: on a plain.
Posts: 598
Re: So, when will Deepmind AlphaZero play poker versus Libratus?

Anyone remember when Facebook made a couple AI accounts/profiles and they created/were communicating via their own language and it could not be interpreted and they shut the entire thing down? That was creepy.
chocLatee is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 11:35 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright © 2008-2017, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online