Open Side Menu Go to the Top
Register
Heads Up Hold'em Solved? Heads Up Hold'em Solved?

10-09-2013 , 02:06 PM
Quote:
Originally Posted by NxtWrldChamp
How can you have 2 "different" GTO strategies?

If one is playing perfectly GTO and the other is playing a different strategy it would appear the 2nd is not playing GTO.

no

There is evidence that there are several mixed strategy GTO, even in the simplest game like khun poker, OneCard poker, etc. ....
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:10 PM
Quote:
Originally Posted by NxtWrldChamp
How can you have 2 "different" GTO strategies?

If one is playing perfectly GTO and the other is playing a different strategy it would appear the 2nd is not playing GTO.

Also just showing us the results of 2 bots vs each other does nothing to show how good they are.
I think there only has to be at least one GTO for a 2 player game, meaning there can be multiple.

That aside, showing a graph like that and saying its close to GTO is pretty lacking.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:12 PM
Quote:
Originally Posted by VictorChandler
That aside, showing a graph like that and saying its close to GTO is pretty lacking.
Those bots are designed to be close to the GTO

But maybe the Isaac Haxton 's brain was designed to be closer to the GTO

Last edited by ohsosick; 10-09-2013 at 02:19 PM.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:26 PM
Can't find any claim anywhere that limit hold'em is entirely solved and i don't think what you quoted means that it is solved. Results of the computer poker championship seem to say that first place beats second by 2bb/100 for HUNL as far as i can tell in 2013 and the blurb about how it works indicates to me that it is a long way from GTO and I'm pretty sure it wouldn't claim to be anywhere near close enough to GTO that this winrate is close to the maximum achievable vs either of the top 2 competitors.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:26 PM
Even though its becoming clear you have no idea what you are talking about...

Quote:
While heads-up limit is sufficiently small that the suboptimality of strategies
can now be evaluated conveniently [8] and close approximations to an optimal
strategy are becoming possible [6], the situation in the no-limit ACPC events
appears bleak. Even the smallest of the three no-limit variants is far larger than
heads-up limit. This is simply a reality of the domain: the game is intrinsically
far more complex, and presents additional challenges for state-space abstraction
research
Edit: Quote from one of the creators of the bot you are discussing, from this year.

200bb 1/2 nlhe is ~= 2.28e58 times as large as HULHE.

Last edited by mperich; 10-09-2013 at 02:38 PM.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:32 PM
Quote:
Originally Posted by Kanu
Can't find any claim anywhere that limit hold'em is entirely solved and i don't think what you quoted means that it is solved.
Quote:
We complete Fixed Limit Hold’em game tree search without approximation.
what don't you understand in this sentence ?

send an email to the alberta university if you don't believe me
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:39 PM
Quote:
Originally Posted by ohsosick
what don't you understand in this sentence ?

send an email to the alberta university if you don't believe me
No approximation in the game tree doesn't necessarily mean that the game tree is solved completely. I would guess that it means that they don't bucket hands?
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:42 PM
let it be known: if you are in a field that a computer will eventually be able to outperform your abilities, your career has an expiration date. poker could, although we have yet to see, be this way.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:45 PM
Quote:
Originally Posted by Kanu
No approximation in the game tree doesn't necessarily mean that the game tree is solved completely. I would guess that it means that they don't bucket hands?
no approximation = the whole game

all the possible (Hero/Vilain) preflop holdings X all the possible boards X all the possible betting sequence on all streets
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:47 PM
I believe they would say without abstraction in that case Kanu, not without approximation. I do think it was solved or extremely close to it.
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:48 PM
this discussion is giving me a headache
Heads Up Hold'em Solved? Quote
10-09-2013 , 02:58 PM
Quote:
Originally Posted by mperich
I believe they would say without abstraction in that case Kanu, not without approximation. I do think it was solved or extremely close to it.
I'm surprised that when I google "limit holdem solved" that it doesn't immediately come up with anything confirming that it is solved if it has indeed been solved. I expect that the sentence doesn't mean what you think it means. I'm not an expert on computer poker so I could be wrong but I don't think that it going down every path in the game tree = game solved. I think it was already pretty close to solved and I guess this is probably another step closer but yeah I'd just be really surprised if a google search didn't come up with the fact it had been solved if it had been. Think it'd be pretty huge news in the sector if it had happened given how long people have been trying. Everything still seems to say "practically solved", "effectively solved" or "almost solved".

Edit: Also just looked on the Neo Poker Lab website and don't see a claim on there that they solved limit Holdem and there would surely be one if they believed they had.
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:11 PM
http://neopokerlab.com/about-us

Quote:
Achievements

With several million hands played, Neo artificial intelligence agents have received positive feedback from professional poker players. These agents participated in 2012 and 2013 Annual Computer Poker Competitions and demonstrated solid results.

We have built several AIs which play different poker games using advanced algorithms and techniques. The quality of the poker AI is constantly improving. Our research also involves other less popular but mathematically complex poker games such as Omaha and Stud.
I think they'd mention if they'd solved limit hold'em.
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:14 PM
I've send an email to the neo poker bot conceptor.
I've asked him to post in this thread....
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:29 PM
Hard to argue with that logic. Also, their strategy only took up 3TB which seems too small for the full game so Id prob agree w/ Kanu and PGUK here.
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:32 PM
Quote:
Originally Posted by mperich
Hard to argue with that logic. Also, their strategy only took up 3TB which seems too small for the full game so Id prob agree w/ Kanu and PGUK here.
3TB with a LZW compression and they remove the isomorphisms
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:52 PM
Not sure where you are seeing it was 3TB compressed. Non-compressed strategy with isomorphs removed would take up 33TB according to UofA paper. Anyways, Im done w/ the derail. Even if you're right about LHE being solved (def could be based on the wording they used, altho unlikely now that Ive looked/though about it further)...you are very wrong about NLHE which was where the derail came from in the first place.
Heads Up Hold'em Solved? Quote
10-09-2013 , 03:56 PM
Hey - my name is Mike Johanson, I'm one of the PhD students at the University of Alberta that developed Hyperborean and Polaris. I was emailed and asked to drop into the thread to say whether heads-up limit Texas hold'em has been solved yet. I haven't read the rest of the thread, so my apologies if I cover something that's already been discussed.

Heads-up limit isn't solved yet. A couple of years ago, I posted in the "Math is apparently not important" thread (link) thread to discuss a technique we had for measuring how close a strategy was to a Nash equilibrium by computing how much it can lose against a perfect adversary. A Nash equilibrium would lose $0 / game, and the lower your exploitability is, the closer you are to Nash.

In that 2011 thread, I mentioned that the strategies we had for the 2008 Man-vs-Machine match were beatable for 235 milli-big-blinds / game, or 11.75 BB/100 (just divide by 20 to convert). The best strategies we had in 2011 were quite a bit better, being exploitable for 104.41 mbb/g, or about 5 BB/100.

We've made a lot of progress since then. Our most recent game solving algorithm, called CFR-BR (link) lets us use abstraction techniques but get as close as possible to a Nash equilibrium within an abstraction. In that paper we had strategies as low as 41.199 mbb/g (2 BB/100), and we've made more progress since then. We didn't use this strategy in the Annual Computer Poker Competition this year; our heads-up limit entry in 2012 and 2013 was the same 2011 entry I mentioned in that previous post that's beatable for 104 mbb/g. So at least for the University of Alberta - we're getting close to equilibrium, but we aren't there yet. We haven't solved the game.

Neo Poker Bot won the Annual Computer Poker Competition heads-up limit event this year. They did great - I'd be curious to know more about how their system works. In their description of their agent on the ACPC website (link), they said that they did game tree search without approximation. This doesn't mean that they've solved the game; game tree search is a different problem than game solving, and search is way easier than solving. For example, you could use a Monte-Carlo Tree Search technique to search the game without using abstraction, and it wouldn't necessarily even take much memory, but it wouldn't converge to a Nash equilibrium (ie, solve the game) even in the limit. So their bot is clearly good, but as I understand it I don't think they're claiming to have solved the game.

I'd need to see some pretty serious proof before I'd believe anyone's claim of having solving heads-up limit. To actually brute-force your way to an exploitability of very nearly 0 in heads-up limit without using approximation techniques, the state-of-the-art CFR algorithm would need 523 terabytes of memory (probably RAM, but possibly disk) and our rough estimate is 100,000 core-years of computation. We think it's way more likely to be done first by using abstraction techniques that are good enough to get you to a tiny exploitability of under, say, 1 mbb/g. Once you're that close, the game is solved for all practical purposes; someone that could win that 1 mbb/g would have to play 100 million games to prove it with statistical confidence.
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:04 PM
^^ excellent post, sir. thanks for your time.
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:05 PM
^ Thanks for clearing that up.
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:06 PM
Thank you for having responded to my e-mail and answering in this thread
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:07 PM
How do you work out much it would lose against a perfect adversary without knowing what a perfect strategy is?
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:11 PM
I just looked at the last couple of pages. One-on-one performance doesn't indicate at all how close to Nash equilibrium you are. For years, the top couple of bots in the ACPC have only had tiny edges of 5-10 mbb/g (0.25 - 0.5 BB/100) against each other. But in 2010 and 2012, we did big experiments where we worked with the creators of the entries to measure how exploitable they were. In 2010, for example, the first, second and third place bots were exploitable for 300, 237 and 135 mbb/g respectively. That's interesting for two reasons. First, the third place entry (Hyperborean, our bot) was exploitable for about half of the first place bot Rockhopper, although it lost a few milli-big-blinds per game against it. Second, Always-Fold would lose for 750 mbb/g. So even though Rockhopper won, it was beatable for 40% of what always folding would lose. Rockhopper was good, but that's a long way from optimal play.

The only useful measure of how close you are to a Nash equilibrium is to measure how much you lose against a perfectly exploitive adversary. Measuring how much you lose to another guy that's just trying to not lose doesn't tell you much.

Last edited by FullyCompletely; 10-09-2013 at 04:34 PM.
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:13 PM
Sick post, also made me feel better about the "bot threat" , most posts regarding the matter gives me tummy hurt
Heads Up Hold'em Solved? Quote
10-09-2013 , 04:13 PM
Quote:
Originally Posted by PartyGirlUK
How do you work out much it would lose against a perfect adversary without knowing what a perfect strategy is?
nm. found it.
Heads Up Hold'em Solved? Quote

      
m