Open Side Menu Go to the Top
Register
Holy Grail of Poker Holy Grail of Poker

10-19-2010 , 05:45 PM
pberhman,

TRUE OR FALSE?

you used earlier versions of this bot to illegally play microlimits on FTP AAAND hired other players to run this bot until you were caught and all your accounts frozen! You then tried on other sites unsuccessfully..

Last edited by spino1i; 10-19-2010 at 05:52 PM.
Holy Grail of Poker Quote
10-19-2010 , 08:21 PM
Quote:
Originally Posted by pbehrman

I think the advice should be to bet most of the time here; however, the program doesn't do things like "if c bet opportunity and very strong hand, and one opponent, then c bet x %". It simulates the hand out many times and makes the recommendation based on that simulation result. When I see moves that I would not make it's usually one of two things:
(a) a mistake in the program's assumptions or
(b) a very interesting learning opportunity.
So for each decision is a simulation done? That is, you have some model for your opponent, and then you try out different lines against that model and see which one does best? Over many runs, obviously.
Holy Grail of Poker Quote
10-19-2010 , 08:48 PM
Question about stats. I assume the program using PT/HEM type stats to help do opponent modeling. Can you explicitly plug it into these programs so it has the benefit of our existing database? Or does it only use hands that it has imported to do oppoenent modelling?
Holy Grail of Poker Quote
10-19-2010 , 09:58 PM
Quote:
Originally Posted by TJD
I re-imported as you said and over another 10 occasions it thought flop bet was better 9 times. On some occasions this difference was very large.

It "accepted" my turn raise 3 times out of 10 but each time it still had the EV of a call as slightly higher.

This raises 2 questions:-

1) How far out does a deviation have to be before it decides an action is "wrong"

2) Does it consider bet sizing as a "tell". Clearly the program is saying my turn raise is wrong (and it may well be, especially seeing the probable width of his range) but part of my "read" was his small bet which although it is sometimes a trap is much more often a weak hand. Does the AI consider that?

I prefer not to post a HH on here to protect the innocent. However, if I could send it to you directly for investigation and you undertook not to use/release it for any other prupose, I am fine with that.

Cheers

T
1) For a deviation to be flagged "wrong" (which is not always wrong if you are mixing up your play), it needs to both be negative EV (vs. the alternative) by 5% of the pot or more and not a recommended action 30% of the time or more.

2) Yes, it makes bet size reads, and this is in the process of being enhanced. However, off the top of my head I'm not sure it's making the same read as you. PM me the hand history -- and I won't share it. But I can analyze it, and I can see what our AI is thinking about that bet size in that situation.

Thank you for the terrific questions / advice.
Holy Grail of Poker Quote
10-19-2010 , 10:00 PM
Quote:
Originally Posted by gaming_mouse
Question about stats. I assume the program using PT/HEM type stats to help do opponent modeling. Can you explicitly plug it into these programs so it has the benefit of our existing database? Or does it only use hands that it has imported to do oppoenent modelling?
Yes.

Today it only uses hands imported for opponent modelling; however, we plan to make it work well w/ PT and HEM so that (a) it always reads those hand histories for profiling purposes whether you've imported them or not and (b) simplify things for you so that it's easier to analyze all the hands you've already imported into the trackers.
Holy Grail of Poker Quote
10-19-2010 , 10:03 PM
Quote:
Originally Posted by gaming_mouse
So for each decision is a simulation done? That is, you have some model for your opponent, and then you try out different lines against that model and see which one does best? Over many runs, obviously.
Yes. That's exactly how it works. Here's how we describe it.
Holy Grail of Poker Quote
10-19-2010 , 10:12 PM
Quote:
Originally Posted by gball
Would be cool if there was a way to set for example your hole cards, and flop cards, and then play the hand over and over multiple times.
I got the thumbs up from Poker Stars to do this!

You'll see it soon!

Thank you very much for the terrific suggestion.
Holy Grail of Poker Quote
10-19-2010 , 10:24 PM
Quote:
Originally Posted by pbehrman
I got the thumbs up from Poker Stars to do this!

You'll see it soon!

Thank you very much for the terrific suggestion.
Won't this essentially allow you to create a HGP bot for PF and flop? That seems bad.... why would they allow that?

EDIT: btw paul, thanks for your prompt replies
Holy Grail of Poker Quote
10-19-2010 , 10:51 PM
answer my question, were you botting on fulltilt or not?
Holy Grail of Poker Quote
10-20-2010 , 01:22 AM
Quote:
Originally Posted by gaming_mouse
Won't this essentially allow you to create a HGP bot for PF and flop? That seems bad.... why would they allow that?

EDIT: btw paul, thanks for your prompt replies
no. We'll only be able to do this when offline (i.e., if the poker site is running then this part of our application won't function).
Holy Grail of Poker Quote
10-20-2010 , 01:23 AM
Quote:
Originally Posted by spino1i
answer my question, were you botting on fulltilt or not?
No, we didn't.
Holy Grail of Poker Quote
10-20-2010 , 01:29 AM
Quote:
Originally Posted by CmonSon
any1 gave it a shot yet? If u can get it for merge ill def test it out
We'll have it for iPoker and Party pretty soon.

Not sure when we'll get to Merge. But hopefully soon!
Holy Grail of Poker Quote
10-20-2010 , 01:55 AM
Quote:
Originally Posted by GalacticRewind
When playing against simulated game against the AI, we have three import options: 1 all hands, 2 no hands, and 3 only when playing with advice hidden. I wanted to play using Hero while I import all hands (or only when playing with advice hidden), and using Hero2 while I import no hands. I think I figured out how to do it, but it was a bit confusing (I had to right-mouseclick when I was playing at the table and create a new hero that way).

The goal is to be able to play using one of them while I look at the advice, and to play using the other one while I do not look at the advice. I will look at the advice when I play using Hero2; but I will not look at it when I play using Hero and import at the same time. When I play using Hero, and do not look at the advice, and import, that gives HGP the ability to adjust to my play. But if I start looking at the advice while playing and importing with Hero, then HGP will not adjust correctly to my play, so when I want to look at the advice I switch over to Hero2.

As far as the GUI is concerned, here is how I think it would work well:


That drop-down box that shows Hero playing is disabled, but it would be better if it was enabled and allowed the user to choose Hero, Hero2, Hero3, etc. (There also needs to be a screen/window for creating those other heros too.)

By the way, here is how I ended up creating (and switching to) Hero2, but I do not think it is good to have to do it like this:
Thank you for the terrific illustration! We'll try to execute your suggestion.

Thanks!
Holy Grail of Poker Quote
10-20-2010 , 04:32 AM
I do not know how you can get round this potential problem but I throw it out there to see what you and others think.

You say that the program is intended as a training tool as if we have a coach sitting looking over our shoulder. However, due to processing time pressure it runs the simulation a relatively small number of times compare to infinity That can create, as it did in the case of the J9 hand I highlighted, a situation where it gives advice that HGP itself thought was wrong the next 9 times it tried.

Because it looked "odd" I questioned it and you explained the method of how to get the prog to re-import to give it another chance to consider the hand.

However, I am a player with many years of experience. What if I was a relatively new player? I would have no experience base on which to "question" HGP and would now be left thinking that "slow playing" top and bottom pair on a 2 flush 3 str8 board is the correct play.

(I can just about see an argument - but still disagree - for not betting on the basis that my equity is not brilliant when called by worse and awful when called/raised by better. If that equity balance was such that I had a very small edge when he did not fold, I may indeed make more money on the hand by inducing bluffs or hero calls later but I would not call this a slow play; it is simply a pot control type line looking to maximise EV by inducing because I make litttle "profit" from a value bet)

Would I notice, as a novice, that in a few thousand hands time, HGP then advised betting when a similar situation re-appeared?

The problem of different assumptions will always remain but the randomness caused by small sample runs could be confusing to relative beginners.

Cheers

T
Holy Grail of Poker Quote
10-20-2010 , 04:39 AM
Quote:
Originally Posted by pbehrman
...
Today, if I see a hand that's played poorly by our AI here's what I do. I turn on some logging that isn't currently part of the software. The software then writes to an html file all its assumptions about the hand including
  • the probability that your opponent will take the actions he took with each of 1326 hands (or 169 preflop)
  • the probability he would think you would take your actions with every possible card
  • and the results of the simulation

From this I can see... Oh it assumed he folded all over cards. Okay, why did it assume that. Okay maybe I need to fix some element of the Opponent Model.

We could share this with customers. It's not currently very user friendly."

At some point, I'd like to provide our data about a hand to an EV calculator. If you are familiar with these (e.g., Stox / Card Player have one) they allow you to analyze a hand, but they are exceedingly complex and time consuming. We could populate that tool with all of our assumptions in seconds. Then you could go into that tool and change assumptions and recalculate.
These ideas sound great. To be honest, I'm not that interested in whether a small stakes player (i..e, the program) thinks I should bet / call / raise or whatever. I am interested in something which thinks about poker in a different way (even if it's weaker than me overall, it might be stronger in certain spots) and that makes solving complex poker problems simpler. I have StoxEV and had the same thought - your program could basically provide the starting point for how the villain would act. However, I'm not sure how it would work on, for example, flop situations - the tree is very large as the are thousands of turn and river card combinations.

A couple of thoughts on frequencies. First, I don't really understand how they're derived. From what you've said, the program doesn't run through every potential holding hero might have so it's not clear to me how it balances its frequencies.

More fundamentally, I don't understand the need for the frequencies in most cases anyway. In hold 'em we effectively have a continuum of hands - it's much like Chen's [0,1] game. This means that our hand can act as a randomiser. For example, bluffing with a very weak hand in position on the river dominates bluffing with a fairly weak hand in position (because the fairly weak hand has showdown value). The program might say we should bluff 80% with the very weak hand and 20% with the fairly weak hand. Why not just bluff 100% with the very weak hand and 0% with the fairly weak hand - this is a better strategy and does not affect our balance (assuming for the sake of argument that we are equally likely to hold the fairly weak and very weak hands). From my understanding of the poker game theory literature, we only need to play a mixed strategy for the one hand at certain inflexion points (e.g., bluff / give up; value bet / take the showdown). (Ignoring the blocker effect.)
Holy Grail of Poker Quote
10-20-2010 , 05:26 AM
PS After I wrote my previous post HGP came up with another odd result. I had misplayed it preflop (so true!) but on the flop it suggested raising. My thought was WTF? (or simlar). I re-ran it and suddenly it said call was "correct" by 25% of the pot. A couple of other re-runs gave similar big +EV results for just calling. Then it said I should raise again - back to call etc. etc.

What I would like to be able to do is to get longer runs for POST play analysis without needing to do this manually.

I understand this AI has a lot of processing to do, so, if it is supposed to be an immediate post hand critic, it has to reduce the number of trials to a relatively small number.

However, in the calm of the following day when a more experienced player may be interested in whether his game has a few leaks he should be thinking about, the output is sometimes incorrect based on the small sample run. Not only does it throw up "mistakes/deviations" that are not present over longer runs but the short run may also fail to find problems that would exist over a longer one so we never even know it was there.

It would be useful to be able to ask HGP to re-run ALL of our hands that are filtered for analysis but to allow it to run many more cycles (choice made by the user depending on the time available to THEM) so that its results are based more on a long term basis and are more reliable.

Alternatively if time was a little more pressing for the user, we could ask HGP NOT to re-run hands that were so "certain" over the initial smaller run performed on import that a re-run would be most unlikely to change its opinion.

These choices of a) number of cycles b) probability of initial run being correct would allow users to only see results that we were pretty sure were likely to be useful rather than have a look and say WTF? and then need to re-run the hand several times manually to see what an "average" result looks like.

I cannot imagine implementing this would be difficult for post play analysis; but what do I know?

T
Holy Grail of Poker Quote
10-20-2010 , 06:05 AM
Quote:
Originally Posted by TJD
I do not know how you can get round this potential problem but I throw it out there to see what you and others think.

You say that the program is intended as a training tool as if we have a coach sitting looking over our shoulder. However, due to processing time pressure it runs the simulation a relatively small number of times compare to infinity That can create, as it did in the case of the J9 hand I highlighted, a situation where it gives advice that HGP itself thought was wrong the next 9 times it tried.

Because it looked "odd" I questioned it and you explained the method of how to get the prog to re-import to give it another chance to consider the hand.

However, I am a player with many years of experience. What if I was a relatively new player? I would have no experience base on which to "question" HGP and would now be left thinking that "slow playing" top and bottom pair on a 2 flush 3 str8 board is the correct play.

(I can just about see an argument - but still disagree - for not betting on the basis that my equity is not brilliant when called by worse and awful when called/raised by better. If that equity balance was such that I had a very small edge when he did not fold, I may indeed make more money on the hand by inducing bluffs or hero calls later but I would not call this a slow play; it is simply a pot control type line looking to maximise EV by inducing because I make litttle "profit" from a value bet)

Would I notice, as a novice, that in a few thousand hands time, HGP then advised betting when a similar situation re-appeared?

The problem of different assumptions will always remain but the randomness caused by small sample runs could be confusing to relative beginners.

Cheers

T
Good point. We need to give people the option to re-run the simulation more times.
Holy Grail of Poker Quote
10-20-2010 , 06:07 AM
Quote:
Originally Posted by TJD
PS After I wrote my previous post HGP came up with another odd result. I had misplayed it preflop (so true!) but on the flop it suggested raising. My thought was WTF? (or simlar). I re-ran it and suddenly it said call was "correct" by 25% of the pot. A couple of other re-runs gave similar big +EV results for just calling. Then it said I should raise again - back to call etc. etc.

What I would like to be able to do is to get longer runs for POST play analysis without needing to do this manually.

I understand this AI has a lot of processing to do, so, if it is supposed to be an immediate post hand critic, it has to reduce the number of trials to a relatively small number.

However, in the calm of the following day when a more experienced player may be interested in whether his game has a few leaks he should be thinking about, the output is sometimes incorrect based on the small sample run. Not only does it throw up "mistakes/deviations" that are not present over longer runs but the short run may also fail to find problems that would exist over a longer one so we never even know it was there.

It would be useful to be able to ask HGP to re-run ALL of our hands that are filtered for analysis but to allow it to run many more cycles (choice made by the user depending on the time available to THEM) so that its results are based more on a long term basis and are more reliable.

Alternatively if time was a little more pressing for the user, we could ask HGP NOT to re-run hands that were so "certain" over the initial smaller run performed on import that a re-run would be most unlikely to change its opinion.

These choices of a) number of cycles b) probability of initial run being correct would allow users to only see results that we were pretty sure were likely to be useful rather than have a look and say WTF? and then need to re-run the hand several times manually to see what an "average" result looks like.

I cannot imagine implementing this would be difficult for post play analysis; but what do I know?

T
We'll add this to the list. It's not hard.
Holy Grail of Poker Quote
10-20-2010 , 06:10 AM
Quote:
Originally Posted by pbehrman
No, we didn't.
Really?

Be careful, because I can prove you're lying.

So is your plan now to lie in this thread, and get people to help you make a new bot so you can then rip off legit online poker players once again?
Holy Grail of Poker Quote
10-20-2010 , 06:16 AM
Oooh, this just became very much more interesting...
Holy Grail of Poker Quote
10-20-2010 , 06:27 AM
Quote:
Originally Posted by spino1i
So is your plan now to lie in this thread, and get people to help you make a new bot so you can then rip off legit online poker players once again?
If we wanted to bot, making this software available would not be smart. The way bots are detected is usually by seeing a number of bots playing the same strategy. Since everyone has this AI's strategy it would be quite easy for people to detect it.

The way we improve our AI is to replay a million hands played by two of the world's best poker players and see "their mistakes" which are more often "our mistakes". This and/or hiring such pros would be a lot more efficient at finding improvement opportunities than spending a year building training software.

We are trying to do something helpful for the industry -- yes with technology that we could use to profit off the industry at its expense (if we wished). If so, sharing that AI with the world would make it so easy to identify our AI.
Holy Grail of Poker Quote
10-20-2010 , 06:39 AM
Quote:
Originally Posted by TJD
I do not know how you can get round this potential problem but I throw it out there to see what you and others think.

You say that the program is intended as a training tool as if we have a coach sitting looking over our shoulder. However, due to processing time pressure it runs the simulation a relatively small number of times compare to infinity That can create, as it did in the case of the J9 hand I highlighted, a situation where it gives advice that HGP itself thought was wrong the next 9 times it tried.

Because it looked "odd" I questioned it and you explained the method of how to get the prog to re-import to give it another chance to consider the hand.

However, I am a player with many years of experience. What if I was a relatively new player? I would have no experience base on which to "question" HGP and would now be left thinking that "slow playing" top and bottom pair on a 2 flush 3 str8 board is the correct play.

(I can just about see an argument - but still disagree - for not betting on the basis that my equity is not brilliant when called by worse and awful when called/raised by better. If that equity balance was such that I had a very small edge when he did not fold, I may indeed make more money on the hand by inducing bluffs or hero calls later but I would not call this a slow play; it is simply a pot control type line looking to maximise EV by inducing because I make litttle "profit" from a value bet)

Would I notice, as a novice, that in a few thousand hands time, HGP then advised betting when a similar situation re-appeared?

The problem of different assumptions will always remain but the randomness caused by small sample runs could be confusing to relative beginners.

Cheers

T

I think we'll add the ability to change # of simulations to reduce this issue, and have the default be longer/more simulations.

Thank you for your insight.
Holy Grail of Poker Quote
10-20-2010 , 10:08 AM
Quote:
Originally Posted by pbehrman
no. We'll only be able to do this when offline (i.e., if the poker site is running then this part of our application won't function).
Paul,

You should know that this provides approximately 0 protection. It would be super ez to run HGP in a Virtual Machine and the poker client on the main PC (or in a 2nd VM), and write a program to automatically enter the actions happening on the poker client into HGP, getting real time feedback. This would enable anyone to create a bot or advisor. I am shocked Pokerstars with their supposedly crack security team would not realize this. In any case, I strongly urge you to reconsider implementing that function. You need to be very careful with this baby you've created. It's already dangerous enough that the AI is out there on individuals PCs and could in theory be stolen by a dedicated hacker.
Holy Grail of Poker Quote
10-20-2010 , 10:40 AM
Quote:
Originally Posted by gaming_mouse
Paul,

You should know that this provides approximately 0 protection. It would be super ez to run HGP in a Virtual Machine and the poker client on the main PC (or in a 2nd VM), and write a program to automatically enter the actions happening on the poker client into HGP, getting real time feedback. This would enable anyone to create a bot or advisor. I am shocked Pokerstars with their supposedly crack security team would not realize this. In any case, I strongly urge you to reconsider implementing that function. You need to be very careful with this baby you've created. It's already dangerous enough that the AI is out there on individuals PCs and could in theory be stolen by a dedicated hacker.
hmmm Good point....

Maybe there's something we can do to solve that issue.
Holy Grail of Poker Quote
10-20-2010 , 10:57 AM
Quote:
Originally Posted by gaming_mouse
Paul,

You should know that this provides approximately 0 protection. It would be super ez to run HGP in a Virtual Machine and the poker client on the main PC (or in a 2nd VM), and write a program to automatically enter the actions happening on the poker client into HGP, getting real time feedback. This would enable anyone to create a bot or advisor. I am shocked Pokerstars with their supposedly crack security team would not realize this. In any case, I strongly urge you to reconsider implementing that function. You need to be very careful with this baby you've created. It's already dangerous enough that the AI is out there on individuals PCs and could in theory be stolen by a dedicated hacker.
Quote:
Originally Posted by pbehrman
hmmm Good point....

Maybe there's something we can do to solve that issue.
For now, we've decided not to add that. Maybe we can come up w/ something smart in the future...

Again, thank you
Holy Grail of Poker Quote

      
m