Open Side Menu Go to the Top
Register
Heads Up Hold'em Solved? Heads Up Hold'em Solved?

10-16-2013 , 05:52 PM
In real life if they represent strategies with say 3 decimal digits (should fit into 4 bytes) or more likely ~6 decimal digits (8 bytes) it's more than good enough.
What counts is how far away they are from Nash. I would say that if they got 0.01bb/100 they have "we solved poker" bragging rights.

Quote:
and the irrational numbers ?
imagine that you have to 4bet pi % w/ A2s
u can't stock in memory the exact value
+1
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:02 PM
Quote:
Originally Posted by NPLSupport
Hello everybody,

I represent Neo Poker Lab and I’m going to answer to some of your questions. Below you will find an answer to the question we’ve received from one of 2+2 member, specifically “We want to know if you have solve the HU limit holdem or not”

The game is solved numerically. We get to the level of precision high enough to consider FLHU solved. There is no problem for us to increase level of precision say by one order. However it’s not evident if this is practically reasonable because it takes some time plus there are some more interesting problems we have that can be solved by algorythmes used to solve FLHU, including ones not related to poker.

No approximation or Monte Carlo simulation, as well as CFR was employed during calculation. The algorithm searched every possible board cards combination, pocket cards and found optimal action for every possible game history. For this strategy we didn’t use a supercomputer or anything similar.

Also I would like to thank FullyCompletely for his activity and help with explaining current state of research in poker AI field. Also we really appreciate Alberta CPRG contribution to Game Theory and team poker related publications.

Unfortunately Neo is a commercial project so we couldn’t go into detailed discussions about our approach and prefer to keep this confidential as much as we can to protect our know-hows.
Thanks for taking the time to post here. However, I`m sure you must be aware that your post is confusing people after the information that was provided by FullyCompletely regarding state-of-the-art research in "solving" HU limit poker, especially when you yourself consider it the (public) "current state of research in poker AI field".

I understand that you want to protect your know-how, but with such heavy evidence ITT it seems like a hard proposition to buy that you have solved the game "numerically" and without approximation or a supercomputer, when that seems sort of unfeasible right now. Making such a claim without any form of proof is neither scientifically correct nor a good marketing strategy for a commercial business IMO.

I would higly appreciate if you could provide a bit more insight. Cheers

Last edited by samooth; 10-16-2013 at 06:04 PM. Reason: hope he is legit
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:13 PM
Quote:
Originally Posted by ohsosick
and the irrational numbers ?
imagine that you have to 4bet pi % w/ A2s
u can't stock in memory the exact value
No you cannot store the exact value of pi, but pi % is an exact solution.
Quote:
Originally Posted by Kanu
why does the number have to fit into a fraction like 1/7?
Not saying it has to be a fraction, just saying that a solution is exact if you know the term representing it.

If NLP knew all the exact terms, than they had a solution. Remember that the game is solved if a nemesis is ev-neutral against the solution, and you could still prove that if you knew the exact terms.

Also, last time I looked (no pun intended, last time I worked with c was in university ten years ago) the computer epsilon for the programming language c was 2^-64, (and better precisions are easily possible res. already implemented in scientific programming systems) so that there is no way that effect accounts for the 0.01 or whatever bb they are away from the solution.

Last edited by franxic; 10-16-2013 at 06:26 PM.
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:18 PM
Quote:
I understand that you want to protect your know-how, but with such heavy evidence ITT it seems like a hard proposition to buy that you have solved the game "numerically" and without approximation or a supercomputer, when that seems sort of unfeasible right now. Making such a claim without any form of proof is neither scientifically correct nor a good marketing strategy for a commercial business IMO.
Academics are always behind commercial teams. It was the case in chess where incentives for commercial programmers are much smaller and naturally it's the case in poker. What kind of "evidence" you are talking about ?
Those university teams don't have time to program full time and have 101 different obligations like writing papers, teaching students and making stuff at least a bit general (and not only applicable to poker). It's natural that they are behind.

That being said:
Quote:
I would higly appreciate if you could provide a bit more insight.
Me too!
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:20 PM
Quote:
Originally Posted by franxic
No you cannot store the exact value of pi, but pi % is an exact solution..
pi is just his name.You can't give a name to all irrational numbers.

you can't find all the decimal of pi , or of the kanu number 22.47598938820934829047236473693488...
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:26 PM
Quote:
Originally Posted by punter11235
Academics are always behind commercial teams. It was the case in chess where incentives for commercial programmers are much smaller and naturally it's the case in poker. What kind of "evidence" you are talking about ?
Those university teams don't have time to program full time and have 101 different obligations like writing papers, teaching students and making stuff at least a bit general (and not only applicable to poker). It's natural that they are behind.
Please re-read the last long post from FullyCompletely and focus on the part where he tries to estimate the time as well as the memory needed in order to brute force HU limit. It seems that the bolded part in your post is not the limiting factor here.

Last edited by samooth; 10-16-2013 at 06:48 PM.
Heads Up Hold'em Solved? Quote
10-16-2013 , 06:29 PM
Quote:
Originally Posted by ohsosick
pi is just his name.You can't give a name to all irrational numbers.

you can't find all the decimal of pi , or of the kanu number 22.47598938820934829047236473693488...
You still aren't guessing but calculating that number, and need a term representing it to do so.

1/7 or whatever term also does not have a name, but expresses an exact value. We do not need to know digit #1,000,000 of the decimal to know that 7*(1/7) is exactly 1.

Last edited by franxic; 10-16-2013 at 06:54 PM.
Heads Up Hold'em Solved? Quote
10-16-2013 , 07:21 PM
Quote:
Originally Posted by franxic
You still aren't guessing but calculating that number, and need a term representing it to do so.

1/7 or whatever term also does not have a name, but expresses an exact value. We do not need to know digit #1,000,000 of the decimal to know that 7*(1/7) is exactly 1.
If the way I was thinking about it was correct then you would be being pretty dumb here. Since your posts are rarely dumb, I am going to assume I was not thinking about it correctly I assumed we got a solution, it sees that KJo has marginally higher EV to 3bet than to call, therefore it 3bets slightly more often in the next iteration and converges on a solution that way, meaning that there is no real term that represents how often to 3bet KJo that we can know without knowing all the decimal places and it is just a case of making finer and finer adjustments to the model as we 3bet KJo closer and closer to the correct % but if the correct % were an irrational number then we could keep adjusting forever without reaching it (extremely simplified explanation of how it would work obv). I guess you are saying that the "solution" is in fact a ton of equations and it is just a case of getting those equations correct? If they are all correct then it would be the equivalent of knowing the formula for pi and you could then say you know what pi is despite not being able to quote all the decimal places?
Heads Up Hold'em Solved? Quote
10-16-2013 , 07:43 PM
Quote:
Please re-read the last long post from FullyCompletely and focus on the part where he tries to estimate the time as well as the memory needed in order to brute force HU limit. It seems that the bolded part in your post is not the limiting factor here.
I already said I think he is off several orders of magnitude. Neo poker guys seem to agree and they are the ones delivering the strongest bot.
I've noticed people believe what they want to believe in this thread so they will pick the most comforting version (bazillion Pluto years of super computer work or w/e) but there is nothing I can do about it.

Quote:
If the way I was thinking about it was correct then you would be being pretty dumb here. Since your posts are rarely dumb
He is right in a sense that it's possible to describe all the numbers involved (even if by infinite formulas) and that "real" solution involves only such well described numbers.
On the other hand requiring standards like that would be unheard of for something like solving limit holdem. We don't express distance to the Sun in such a way for example nor gravity constant or about anything else much less so in field of solving things with computers.
Heads Up Hold'em Solved? Quote
10-16-2013 , 07:56 PM
Kanu, you clearly have a point since they claim to have a numerical solution, but that is just a vehicle to reduce computing effort. NLP claims not to use any approximations though (which kind of contradicts that), so i assume that is not the reason they cannot achieve a higher precision (if the claim they did solve it is true at all, which I still doubt; guess we will see at the next botting championship), and good numerical methods converge towards a very high precision very quickly, so that doesn't make too much sense to me.

I am very positive that it has nothing to do with storage precision issues.

I strongly disagree that my posts are rarely dumb (but anyway, thank you Sir), I am brainfarting a lot (i.e. post quicker than i manage to think, look at the % of edits at my posts), and I cannot totally discard the possibility that I return tomorrow and cry "mea culpa" after thinking it through completely. Sometimes I am struck by lightning and feel like i better make a new account to avoid the embarressment.

But w/e, at least we had an interesting discussion then. Have a good night.
Heads Up Hold'em Solved? Quote
10-16-2013 , 08:31 PM
Quote:
Originally Posted by ohsosick
and the irrational numbers ?
imagine that you have to 4bet pi % w/ A2s
u can't stock in memory the exact value
Quote:
Originally Posted by Kanu
why does the number have to fit into a fraction like 1/7?
Quote:
Originally Posted by punter11235
+1
I think you can quite canonically write poker as an LP with rational entries so there will not be irrational solutions, only fractions, on mixed strategies.
Heads Up Hold'em Solved? Quote
10-16-2013 , 08:42 PM
Quote:
Originally Posted by Kanu
why does the number have to fit into a fraction like 1/7?
If you use some sort of neural network algorithm it doesn't have to be an exact anything. If you solve it 'with maths' it will be.
Heads Up Hold'em Solved? Quote
10-16-2013 , 09:37 PM
Quote:
Originally Posted by franxic
A solution should contain the exact term, e.g. while you cannot express 1/7 (0.14285714..) as a decimal with a finite number of digits, it wouldn't be necessary at all to claim an exact solution, 1/7 would be fine.
Another way to look at this:

1/7 can't be expressed in a finite number of digits in base 10. But, for example, in base 7, 1/7 would be equal to 0.1, and in base 14 1/7 = 0.2

Something like pi can't be expressed in a finite number of digits in any base number.
Heads Up Hold'em Solved? Quote
10-16-2013 , 09:42 PM
Quote:
Originally Posted by Klairic
Something like pi can't be expressed in a finite number of digits in any base number.
Base pi!

Juk
Heads Up Hold'em Solved? Quote
10-16-2013 , 09:54 PM
I'm pretty confident that the Nash Equilibrium strategies for HU Hold'em have very little mixing and the Nash Equilibrium strategies for HU Omaha and O/8 are either pure or veeeeeeeery close pure (i.e the best non-mixed strategy for HU Omaha or O/8 is beatable by < .05 bb/100).
Heads Up Hold'em Solved? Quote
10-16-2013 , 10:08 PM
Quote:
Originally Posted by PartyGirlUK
I'm pretty confident that the Nash Equilibrium strategies for HU Hold'em have very little mixing and the Nash Equilibrium strategies for HU Omaha and O/8 are either pure or veeeeeeeery close pure (i.e the best non-mixed strategy for HU Omaha or O/8 is beatable by < .05 bb/100).
Curious about why you say this. I'm inclined to agree with you on the nature of the Omaha variants, though.
Heads Up Hold'em Solved? Quote
10-16-2013 , 10:12 PM
Can it be proven/disproven that some solutions even have irrational numbers? We know that solutions for some games do not.
Heads Up Hold'em Solved? Quote
10-16-2013 , 10:25 PM
Here is a video of 100 hands where hyperborean_iro is using minbets http://www.youtube.com/watch?v=PlfEfKu5tZA.
Someone said that if we could look at a bot playing a GTO strategy it might look like a huge fish because many plays would seem counter-intuitive. Attempting to make sense of a single hand or even a 100 hands might be a pointless exercise. We are watching the result of millions of iterations of an equilibrium finding algorithm and the result might not be intuitive at all.
Heads Up Hold'em Solved? Quote
10-16-2013 , 10:26 PM
Quote:
Originally Posted by Rei Ayanami
Curious about why you say this. I'm inclined to agree with you on the nature of the Omaha variants, though.
+1
Heads Up Hold'em Solved? Quote
10-16-2013 , 10:39 PM
Quote:
Originally Posted by franxic
.
No, unfortunately everything you posted there is well reasoned but ultimately incorrect.

GTO means unexploitable - it has nothing to do with how much you exploit something else for. GTO and "perfect" play are not the same. It's also entirely possible there are many, perhaps infinite, GTO solutions to poker. What this means is that a GTO solution can play against any other strategy, even a different GTO solution, and it will never lose money. If two GTO solutions play forever - they'd break even forever.

Hopefully that's clear. Getting to the next point. Let's consider two GTO solutions. GTO 1 and GTO 2. It's entirely possible that against any given non-GTO strategy that GTO 2 would have a higher expectation than GTO 1. GTO 2 would be a "better" GTO solution than GTO 1.

Reaching GTO is enough to call poker solved, but you could potentially refine GTO solutions indefinitely. Given the search space of poker perfect play will not be mapped out for the foreseeable future. Where you're getting confused is that this does not mean that unbeatable strategies will not be created or have not already been created. Unbeatable and perfect are two very different beasts. The checkers bot chinook is unbeatable, but far from perfect. It doesn't change the fact that checkers is solved.
Heads Up Hold'em Solved? Quote
10-16-2013 , 11:38 PM
Quote:
Originally Posted by Rei Ayanami
Curious about why you say this. I'm inclined to agree with you on the nature of the Omaha variants, though.
I think there's enough hand combinations that you'll normally be able to get very close to the requisite ratios by playing a pure strategy - e.g when it comes to bluff check raising the JJ2r flop there are reasons why it's better to do it with 86s than 87s, or there are reasons to do it with 87o when both of your suits are on the board but not when only one of them is. IMO enough of this minute variation that you won't need to mix very often, certainly from the flop onwards, and that when you do mix postflop, it's going to be with one or two specific hands (e.g the only hand you should mix with on JJ2r is 76 with a backdoor in the suit of the 2), and that playing pure with these would be essentially unexceptionable.

I haven't followed the progressions of high stakes top level online HU NL for a while, so perhaps Kanu would chime in here, but my impression is that most top players would mix up flatting or 4 betting certain very strong hands such as AA and it's considered 'standard expert play' to have aces in both your flatting and 4 betting ranges against another excellent player?

Well, I do think the most likely street where you'd 'need' to mix is preflop. But I'm not convinced you should mix up flatting with aces. People flat with aces so that they can credibly represent a strong hand on certain flops, or can trap aggro opponents right? But you can just raise for value or call down lighter on these flops too. So you don't need to flat aces! But maybe you should always flat aces? People 4 bet aces at least some of the time so that they don't get 5 bet bluffed, or to a build a pot etc. But what if you should always flat AA? You'll still have strong hands in your 4 bet range, and if you're scared about him '5 bet bluffing too much', you can just 6 bet more for value and as a semi bluff. Maybe it turns out that preflop you should always flat AA and QQ and always 4 bet KK, or vice versa ... there's no particular reason to think that playing a pure strategy preflop is particularly exploitable.

Omaha is just an extension of this. When you have 4 cards, there are just some many possible combinations and minute little differences that it's almost certain you can get to your longed for x to 1 value to bluff ratio by selecting certain specific hands.

Permitting me to misuse some terminology: as you add more and more hole cards, poker exponentially converges from a discrete game to a continuous game, and as it tends to a continuous game, the Nash equilibrium asymptotically converges from a mixed strategy to a pure strategy.

I'm pretty sure that the version of holdem I played where we could only look at one of our cards has plenty of mixing (although on a maximum of one specific hole card per information set), and if the best pure strategy in six card omaha is beatable for more than .01 bb/100 I'm going to ask for some pretty good evidence. HU Hold'em is imo discrete enough that the Nash equilibrium definitely includes quite a few mixed strategy points, but continuous enough one could play a pure strategy such that the best response strategy would not beat the rake.

Last edited by PartyGirlUK; 10-16-2013 at 11:49 PM.
Heads Up Hold'em Solved? Quote
10-17-2013 , 01:39 AM
Good reply.

Quote:
Originally Posted by PartyGirlUK
HU Hold'em is imo discrete enough that the Nash equilibrium definitely includes quite a few mixed strategy points, but continuous enough one could play a pure strategy such that the best response strategy would not beat the rake.
For information hiding–related reasons, mixed strategies seem like a necessity in all spots that call for multiple bet sizes, which could be most spots. This seems especially true when those sizes 'fork' off of each other (e.g., n1 c-bet sizes, each of which are matched with n2 double-barrel sizes, each of those being further matched with n3 triple-barrel sizes).

(eta: Assuming you're not referring to µNL/SSNL rake.)

Last edited by Rei Ayanami; 10-17-2013 at 01:46 AM.
Heads Up Hold'em Solved? Quote
10-17-2013 , 01:53 AM
I was referring to the rake at say 10/20 NL or 100/200 LHE. Intuitively I doubt there are more than a few potential bet sizes in any given spot, perhaps something like {All in (Full house, quads, bottom 5%)},{2/3 pot (trips, straight, 85% to 95% percentile)} why do you think you'd need to go much more complicated than that?.
Heads Up Hold'em Solved? Quote
10-17-2013 , 04:39 AM
Quote:
I think you can quite canonically write poker as an LP with rational entries so there will not be irrational solutions, only fractions, on mixed strategies.
Right. Thanks for pointing this out.

Quote:
Can it be proven/disproven that some solutions even have irrational numbers? We know that solutions for some games do not.
I think quoted above is enough (although I don't really know much about LP it just seems likely that there is some theorem which says that LP with only rational coefficients have rational solutions or something along the lines).

Quote:
I'm pretty confident that the Nash Equilibrium strategies for HU Hold'em have very little mixing
On this one I can tell you that you are wrong.
There is huge amount of mixing on the river. This is because when you bluff you can't have given card too often in bluffing range (because then if opponent has that card he will not call). The same goes for bluff catching.
Imagine for example: flop K9732 no flushes, lcheck-call, check-call, check-allin line in 100bb NLHE. Say player in position has to call it sometimes with top pair (as all the evidence suggest from solving games with 3/4 pot size bets with wide enough ranges).
He can't only call with AK because then the opponent will have easy bluffs with A4/A5, he can't only call with KQ because then playing QJ/QT like that would be very profitable so it's better to bluff catch with say 0.1 of your AK/KQ/KJ and include all of A5/A4/QJ/QT/JT/ in bluffing range.
You have a lot of mixing in those river spots to make sure ranges are "card removal" balanced and in games with big bets available on the river the effect is very significant.
Fortunately solving rivers to say 0.00001bb/hand is easy, it's very close to the real thing and there is still a ton of mixing confirming the above (in fact I know it because I saw enough solved rivers to observe the pattern)

Quote:
I doubt there are more than a few potential bet sizes in any given spot
Here my evidence is weak but I am pretty sure you are right. I am also pretty sure min bet is not one of them (or in worse case that you can remove it along with all small bets and not sacrifice any significant exploitability).

Quote:
For information hiding–related reasons, mixed strategies seem like a necessity in all spots that call for multiple bet sizes,
Reverse is true.
The more bet sizes the more information you give away and the more difficult is to balance them.
Remember, that in optimal play environment you have to hand the opponent your ranges for every action

Quote:
Omaha is just an extension of this
Yeah, I would be shocked if pure strategy in Omaha is significantly worse than GTO.

Last edited by punter11235; 10-17-2013 at 04:53 AM.
Heads Up Hold'em Solved? Quote
10-17-2013 , 12:45 PM
Quote:
Originally Posted by PartyGirlUK
... why do you think you'd need to go much more complicated than that?.
I was planning to use an example where n = 2 or 3. The actual number could be higher than that, but this is relevant even if it's not.

Quote:
Originally Posted by punter11235
Reverse is true.
The more bet sizes the more information you give away and the more difficult is to balance them.
Remember, that in optimal play environment you have to hand the opponent your ranges for every action
I never forgot.

Could you elaborate more on why the reverse is true, though?

Let's simplify the math by a lot and call to mind a spot where the SB's range on the flop corresponds to ~1000 possible river combos. The SB c-bets & double barrels & triple barrels 50% each and has two sizings on each street, for a total of eight possible sizing paths. Therefore, the SB's entire triple-barreling range is ~125 combos, which is further divided into those eight paths. It's clear at this point that many of these sub-ranges will contain less than a few dozen combos. (Not to mention all of our other ranges.)

I was thinking that building such small ranges with a pure strategy would create compositional imbalances that--precisely because we are handing our strategies to our opponent--would be ripe for card removal–based exploitation (in a way comparable to that of your example).

Is this right / wrong / not as important as it might seem / etc.?

Last edited by Rei Ayanami; 10-17-2013 at 12:56 PM.
Heads Up Hold'em Solved? Quote

      
m