Two Plus Two Publishing LLC Two Plus Two Publishing LLC
 

Go Back   Two Plus Two Poker Forums > >

Notices

Poker Theory General poker theory

Reply
 
Thread Tools Display Modes
Old 11-16-2013, 08:56 AM   #126
knircky
veteran
 
Join Date: Jan 2010
Posts: 2,097
Re: Pokersnowie question

Quote:
Originally Posted by Qlka View Post
If it cannot determine correct ranges for 4bb 3-handed poker how can I believe it will provide for 100bb 9-handed? What is your point with tourneys if I look at chip ev mode?
If I understand correctly, snowie does not calculate but "remember" EV. So it approximates and thus can make mistakes. You should check the example on their website.

So when snowie starts learning AA has the same EV as 72 since snowie has no model. Once it plays AA a few times it learns that AA has more EV and adjust. So it will play AA a million times and 72 a million times and then learn the different EV. It will also look at different moves and learn which ones are more profitable etc.

Obv it's gonna be much more correct in 100bb spots vs short stack situation.

Last edited by knircky; 11-16-2013 at 09:04 AM.
knircky is offline   Reply With Quote
Old 11-16-2013, 08:58 AM   #127
knircky
veteran
 
Join Date: Jan 2010
Posts: 2,097
Re: Pokersnowie question

Quote:
Originally Posted by Didace View Post
I don't think anyone has said it is bad, just that it is not GTO.
Sure it's not. Can't be since it's limited. Yet it's the closest GTO I have. Since I am playing poker and not writing a GTO thesis snowie = GTO for me.
knircky is offline   Reply With Quote
Old 11-16-2013, 10:53 AM   #128
skario
enthusiast
 
Join Date: Jan 2007
Posts: 84
Re: Pokersnowie question

Quote:
Originally Posted by Didace View Post
I don't think anyone has said it is bad, just that it is not GTO.
I say folding queens is bad.
skario is offline   Reply With Quote
Old 11-16-2013, 01:42 PM   #129
Power On/Off
banned
 
Join Date: Nov 2013
Posts: 11
Re: Pokersnowie question

Quote:
Originally Posted by skario View Post
I say folding queens is bad.
in addition you're playing zoom 1k like the Montenegro guys. right?
Power On/Off is offline   Reply With Quote
Old 11-16-2013, 02:01 PM   #130
masque de Z
Carpal \'Tunnel
 
masque de Z's Avatar
 
Join Date: Aug 2009
Location: Stanford, CA USA
Posts: 8,576
Re: Pokersnowie question

I am prepared to say that i agree that folding QQ is correct to the shock of everyone here even if i wouldnt do it live myself.

But not because GTO or near GTO convergence implies that snowie must check all his top range here as preflop raiser.

No my reasoning would be in fact this;

GTO or near GTO likely suggests that player 1 in example post 108 has an 8% preflop naturally tight raising range to 3bb in a 9 handed table. And he has to cbet now this flop with ;

99,TT,JJ,QQ,KK,AA,AcKc,AcQc,AcJc,QcJc,KcJc,ATs,KcQ c or something like that.

Now QQ cannot improve all that easily in this board, It has another player left to act that can easily now have a small but significant fraction of time either a set or a big draw himself taking out with a potential push or a reraise or call a lot of the QQ equity.

As it is vs the above range in that board QQ has only 28.5% with the other guy left to act ( i used all his=3rd guy, range preflop to get the 28.5%, in fact if he folds some and participates with the better part of that range he is a worse problem as with him out QQ has still only 37% or so anyway, nothing great).

So yes it is very possible that due to the natural tight range of the preflop opener that he cbets this flop over 50% of the time with a range that is in fact the above or a bit wider, against which QQ doesnt exceed 30%, that QQ is indeed in trouble. Because it is a ~30% if only it were all in and in reality we have future bets and QQ can only improve here with a set and even then it may improve while someone gets a straight, it may indeed be perfectly rational that QQ is a fold here when taking into account all that can happen down the road.

Shocker but possible.

Only however if we give snowie cbet range the above hands. To claim that GTO or near GTO is to check the board with such hands is insane.I think near GTO is to in fact bet all these hands and maybe some more and that the flow of the game if you take into account all possible turn cards is such that QQ is in deep trouble here vs 2, moreover the fact that at surface ignoring the details it looks safe.

I again insist someone needs to verify what snowie sees as proper cbet range at this flop. Only by seeing that, we can understand the QQ fold. I am prepared to say that snowie is correct but only if he cbets the above hands at the very least.

Last edited by masque de Z; 11-16-2013 at 02:08 PM.
masque de Z is offline   Reply With Quote
Old 11-16-2013, 07:48 PM   #131
tultfill
veteran
 
tultfill's Avatar
 
Join Date: Jan 2009
Location: internet
Posts: 2,928
Re: Pokersnowie question

Quote:
Originally Posted by Power On/Off View Post
in addition you're playing zoom 1k like the Montenegro guys. right?
Because testimonials is a trustworthy source of information.
tultfill is offline   Reply With Quote
Old 11-16-2013, 07:57 PM   #132
ExaMeter
Pooh-Bah
 
ExaMeter's Avatar
 
Join Date: Jul 2006
Posts: 3,986
Quote:
Originally Posted by tultfill View Post
Because testimonials is a trustworthy source of information.
There are definitely players from montenegro playing in the 500 and 1k zoom games and its pretty much certain that they are from the snowie staple.
I heard rumors that they are not losing money after rakeback. But i know nothing definite about their profitability.

sent from phone, lol auto correct
ExaMeter is offline   Reply With Quote
Old 11-16-2013, 08:29 PM   #133
Evenflow99
grinder
 
Join Date: Jul 2012
Location: Bommeskonten
Posts: 558
Re: Pokersnowie question

Quote:
Originally Posted by knircky View Post
Guys yelling how stupid snowie is just don't understand GTO and how poker works.
That's also my opinion, but i'm not gonna say more otherwise Rusty boils over again!
Evenflow99 is offline   Reply With Quote
Old 11-17-2013, 09:02 AM   #134
Cangurino
Carpal \'Tunnel
 
Cangurino's Avatar
 
Join Date: Apr 2008
Location: The Circus
Posts: 13,478
Re: Pokersnowie question

Quote:
Originally Posted by PokerRon247 View Post
- Maybe a perfect game can't be produced, but how perfect does it need to be until it is effectively unbeatable? I'm pulling numbers out of my ass here and not sure this even makes sense, but say it progressed to a point where it had the game 20% solved, could it approximate the other 80% to a satisfactory point? Would it be possible to solve one or two streets (ie preflop then flop) and then turn the decisions over to the player?
I'm pretty sure it is impossible to solve early streets independently later streets. To evaluate your preflop game you need to know the value of seeing a certain flop with a certain range, which depends on your postflop game. You can solve later streets if you make assumptions as to the ranges that you get there with.

Quote:
Originally Posted by Piconzaz View Post
Thanks knircky. "Trial and error" can be a method where you indeed don't need to explicitly build a model. However, without any assumption about the underlying function you are implicitly optimizing, you cannot have any guarantee at all that your algorithm will converge anywhere. And if it's just randomly trying strategies and picking the best among them, it's probably the least effective method conceivable. The space of all the potential strategies has a crazy high dimension. Exploring it randomly would require an insanely high number of trials. And even if that was possible, you still would not be able to say anything but 'this is the best I've tried so far'. It's nothing more than trying to play the lottery (and a lottery with a LOT of numbers to be guessed) until you actually win. Except that you can't even really know when you win.

I assume that it's a little more tricky than this. And I'm very inclined to believe that their strategy isn't too bad. But I don't see by which black magic they could come up with any kind of argument making it possible to compare it to GTO.

I really could be missing something here. If so, I'll be very glad if anyone can point it out.
"Trial and error" here is misleading since it implies generating random strategies and selecting the best, which is utterly inefficient. A neural network is a function that depends not only on the input but on tons of hidden parameters. The goal is to "learn" the best set of parameters. What's usually done is some sort of evolutionary algorithm where parameter sets of several successful individuals (networks) are combined to form "offspring" which are then subjected to selection. The general topology or shape of the network is more or less known, the parameters are allowed to evolve. This procedure is quite efficient for finding approximate solutions to a wide range of optimization problems.

It will therefore be futile to ask anybody about the overall strategy; all they could (but won't) do is describe the general shape of the network. The way the parameters work together can't be understood by humans.
Cangurino is offline   Reply With Quote
Old 11-17-2013, 10:22 AM   #135
zachvac
Carpal \'Tunnel
 
zachvac's Avatar
 
Join Date: Apr 2008
Posts: 13,724
If they have done enough work to get close to gto, generating a nemesis strategy is not hard and is actually fairly trivial. They can then calculate exactly how far from gto they actually are.

Or maybe they're geniuses who get as many people to use snowie as possible then they employ the nemesis strategy as a bot who will crush snowie users.
zachvac is offline   Reply With Quote
Old 11-17-2013, 10:23 AM   #136
zachvac
Carpal \'Tunnel
 
zachvac's Avatar
 
Join Date: Apr 2008
Posts: 13,724
Also read what cangurino said he knows what he's talking about.
zachvac is offline   Reply With Quote
Old 11-17-2013, 11:35 AM   #137
jamakine
adept
 
Join Date: Feb 2012
Location: Comfort Zone
Posts: 709
Re: Pokersnowie question

Quote:
Originally Posted by zachvac View Post
If they have done enough work to get close to gto, generating a nemesis strategy is not hard and is actually fairly trivial. They can then calculate exactly how far from gto they actually are.

Or maybe they're geniuses who get as many people to use snowie as possible then they employ the nemesis strategy as a bot who will crush snowie users.
I agree that looking how much you lose to a nemesis strategy is an intuitive way to look how far away is gto but I haven't seen any formal work on this.
jamakine is offline   Reply With Quote
Old 11-17-2013, 02:09 PM   #138
Piconzaz
stranger
 
Join Date: Nov 2013
Posts: 10
Re: Pokersnowie question

Quote:
Originally Posted by Cangurino View Post
A neural network is a function that depends not only on the input but on tons of hidden parameters. The goal is to "learn" the best set of parameters.
You make a good point but I'm still very confused about all that.
In fact, even though I'm not that knowledgeable about neural networks, I'm actually working in the field of machine learning, so I'm a little aware about them.
They can be seen as optimization algorithms. You choose a loss function you want to optimize and then, you can use, for instance, a neural network to find your optimum.
I still have two concerns though. Knircky was saying they were not relying explicitly or implicitly on any model of poker. Then I'm very curious how they actually choose their cost function.
Depending on the properties of the function they actually are trying to optimize and on the structure or topology of the network, you may not be able to prove anything about the solution that your neural network will achieve, or even what it would achieve in an infinite amount of time. In fact, it may not converge at all, it may converge to some spurious local extrema. So without any further information, there is absolutely nothing we can say about their solution, nor do we have the guarantee it actually is going in "the right direction".
Piconzaz is offline   Reply With Quote
Old 11-17-2013, 02:12 PM   #139
ZenFish
journeyman
 
ZenFish's Avatar
 
Join Date: Dec 2012
Location: GTOpia
Posts: 275
Re: Pokersnowie question

Quote:
Originally Posted by jamakine View Post
I agree that looking how much you lose to a nemesis strategy is an intuitive way to look how far away is gto but I haven't seen any formal work on this.
http://forumserver.twoplustwo.com/sh...5&postcount=62
ZenFish is offline   Reply With Quote
Old 11-17-2013, 02:55 PM   #140
Cangurino
Carpal \'Tunnel
 
Cangurino's Avatar
 
Join Date: Apr 2008
Location: The Circus
Posts: 13,478
Re: Pokersnowie question

Quote:
Originally Posted by Piconzaz View Post
You make a good point but I'm still very confused about all that.
In fact, even though I'm not that knowledgeable about neural networks, I'm actually working in the field of machine learning, so I'm a little aware about them.
They can be seen as optimization algorithms. You choose a loss function you want to optimize and then, you can use, for instance, a neural network to find your optimum.
I still have two concerns though. Knircky was saying they were not relying explicitly or implicitly on any model of poker. Then I'm very curious how they actually choose their cost function.
Depending on the properties of the function they actually are trying to optimize and on the structure or topology of the network, you may not be able to prove anything about the solution that your neural network will achieve, or even what it would achieve in an infinite amount of time. In fact, it may not converge at all, it may converge to some spurious local extrema. So without any further information, there is absolutely nothing we can say about their solution, nor do we have the guarantee it actually is going in "the right direction".
The thing with evolutionary algorithms is that you don't need an explicit cost function, you just need to be able to compare candidate solutions. Since they claim they don't use any a priori knowledge my guess would be that they compare candidates by simply letting them play against each other and see who profits after some sample of hands.

You are right, without further information we can't judge the quality of their solution, although it has been pointed out above that it is possible to find a maximally exploitive counterstrategy and see how it fares.
Cangurino is offline   Reply With Quote
Old 11-17-2013, 05:52 PM   #141
RustyBrooks
Carpal \'Tunnel
 
RustyBrooks's Avatar
 
Join Date: Feb 2006
Location: Austin, TX
Posts: 22,915
Re: Pokersnowie question

You always need a cost function, or some kind of score. In your example above the profit after some sample of hands IS your cost function. It doesn't have to be easily calculatable.

How do you find the nemesis of a solution like this (a black box)? I don't know of an approach off-hand except for the same kind of approach you used to find the original strategy, i.e. fictitious play I guess it's called.
RustyBrooks is offline   Reply With Quote
Old 11-17-2013, 06:44 PM   #142
Piconzaz
stranger
 
Join Date: Nov 2013
Posts: 10
Re: Pokersnowie question

I guess we all agree that the approach pokersnowie has developped "makes sense" and seems to give results that are not ridiculously ba. But it's not really principled or theoretically sound by any mean. This is not necessarily a bad thing, but it *should* prevent them from making claims like "pokersnowie plays GTO".
Piconzaz is offline   Reply With Quote
Old 11-17-2013, 09:46 PM   #143
zachvac
Carpal \'Tunnel
 
zachvac's Avatar
 
Join Date: Apr 2008
Posts: 13,724
Quote:
Originally Posted by RustyBrooks View Post
You always need a cost function, or some kind of score. In your example above the profit after some sample of hands IS your cost function. It doesn't have to be easily calculatable.

How do you find the nemesis of a solution like this (a black box)? I don't know of an approach off-hand except for the same kind of approach you used to find the original strategy, i.e. fictitious play I guess it's called.
No, you can easily find an exact nemesis given a strategy. Construct the game tree for this game with the given strategy. Now examine each leaf. It will either have an obvious payout or a showdown. At showdowns you can determine how to play every hand, determine the payout based on the range villain arrives there with, and go up the tree choosing the max at every decision.
zachvac is offline   Reply With Quote
Old 11-17-2013, 10:43 PM   #144
RustyBrooks
Carpal \'Tunnel
 
RustyBrooks's Avatar
 
Join Date: Feb 2006
Location: Austin, TX
Posts: 22,915
Re: Pokersnowie question

Quote:
Originally Posted by zachvac View Post
No, you can easily find an exact nemesis given a strategy. Construct the game tree for this game with the given strategy. Now examine each leaf. It will either have an obvious payout or a showdown. At showdowns you can determine how to play every hand, determine the payout based on the range villain arrives there with, and go up the tree choosing the max at every decision.
I was actually referring to the initial construction of the neural network - the training algorithm, whatever it is, HAS to have some function to evaluate the fitness of the net. Maybe I misunderstood Cangurino though and he was actually talking about finding the nemesis.

In which case, I agree with you. But also... the decision tree is very large. There are about 134 million board + hole card combos (just your hole cards and the board, i.e. C(52,7). For each potential board run out there are a LOT of potential betting patterns. To make a rough approximation, pretend that we don't care about bet size and there's never more than 4 bets per round. I counted 17 betting patterns per street but I may have miscounted, I just enumerated them by hand real quick. For 4 streets of betting this means 17^4 or 83,521.

So the total tree should be around 17^4 * C(52,7) or about 11 trillion. This is for HU, for 3 players it would be much much worse. If we represented the tree using a 4 byte float per node, that would be around 44.7 terrabytes. A 8 byte double would obviously be twice that.

I mean it seems vaguely doable with enough firepower but it's not simple either.

And if you allow different bet sizes and uncapped raises it's much much much worse.
RustyBrooks is offline   Reply With Quote
Old 11-17-2013, 10:45 PM   #145
RustyBrooks
Carpal \'Tunnel
 
RustyBrooks's Avatar
 
Join Date: Feb 2006
Location: Austin, TX
Posts: 22,915
Re: Pokersnowie question

I suppose you never have to actually have all of the tree at once though, you can break it up into chunks. It's probably not so bad, I've never tried doing it for anything but simple limit games like limit 5 card draw HU.
RustyBrooks is offline   Reply With Quote
Old 11-17-2013, 10:53 PM   #146
zachvac
Carpal \'Tunnel
 
zachvac's Avatar
 
Join Date: Apr 2008
Posts: 13,724
But if they can calculate an entire game tree they definitely have enough computing power to calculate a nemesis. In terms of space/time required a nemesis calculation takes a fraction that coming anywhere close to gto does.
zachvac is offline   Reply With Quote
Old 11-17-2013, 10:58 PM   #147
RustyBrooks
Carpal \'Tunnel
 
RustyBrooks's Avatar
 
Join Date: Feb 2006
Location: Austin, TX
Posts: 22,915
Re: Pokersnowie question

Quote:
Originally Posted by zachvac View Post
But if they can calculate an entire game tree they definitely have enough computing power to calculate a nemesis. In terms of space/time required a nemesis calculation takes a fraction that coming anywhere close to gto does.
It's not really a given that they're evaling the whole tree now though. They could simply be choosing, say, a random .1% of the game tree and evaluating that. Honestly I have no idea.
RustyBrooks is offline   Reply With Quote
Old 11-17-2013, 11:13 PM   #148
zachvac
Carpal \'Tunnel
 
zachvac's Avatar
 
Join Date: Apr 2008
Posts: 13,724
Quote:
Originally Posted by RustyBrooks View Post
It's not really a given that they're evaling the whole tree now though. They could simply be choosing, say, a random .1% of the game tree and evaluating that. Honestly I have no idea.

Lolwut? How do they get close to gto consider ING .1% of the game tree?
zachvac is offline   Reply With Quote
Old 11-17-2013, 11:28 PM   #149
RustyBrooks
Carpal \'Tunnel
 
RustyBrooks's Avatar
 
Join Date: Feb 2006
Location: Austin, TX
Posts: 22,915
Re: Pokersnowie question

Quote:
Originally Posted by zachvac View Post
Lolwut? How do they get close to gto consider ING .1% of the game tree?
I never said they were close to GTO. They say that.

But don't let me put up a straw man here - I don't really know what they're doing. I'd be pretty surprised if they're doing full tree evaluations for every candidate solution they have, though. Of course, you often don't have to evaluate every node in the tree - there are pruning algorithms.

I'm kind of curious now how fast I could make something that would evaluate an entire (simplified) tree. I don't really have time to work on it though.

If you allowed just, say, 1/2 pot and full pot bet sizes, it would be way worse than the numbers I had above, because you'd have to consider a lot of potential stack sizes too. Even making more approximations (like say, just using stack sizes of 10, 20, 30, 40 etc up to like 300 or something) it would probably make the problem something like a billion to a trilion times worse. That's something I should do (calculate how large the full tree would actually be)
RustyBrooks is offline   Reply With Quote
Old 11-18-2013, 04:52 AM   #150
Hans Gusen
centurion
 
Join Date: Nov 2013
Posts: 131
Re: Pokersnowie question

"Solving [$1/$2 Heads Up NL Holdem (200bb deep)] using a standard CFR implementation (2 double-precision floats per canonical infoset-action) would require 2 383 484 794 528 738 021 376 773 (2.383 × 1024 ) yottabytes of RAM." [1]

[1] http://poker.cs.ualberta.ca/publicat...rt-nl-size.pdf

Last edited by Hans Gusen; 11-18-2013 at 05:03 AM.
Hans Gusen is offline   Reply With Quote

Reply
      

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Forum Jump


All times are GMT -4. The time now is 12:44 PM.


Powered by vBulletin®
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © 2008-2010, Two Plus Two Interactive
 
 
Poker Players - Streaming Live Online