09-12-2011 , 03:12 PM
Hi DoctorBaboon,

When we were making the 2011 IRO strategy, we were able to measure its exploitability at three points: after 62 billion iterations, 100 billion, and 130 billion. Each "iteration" is similar to one game of poker that the program plays against itself while learning its strategy. The 130 billion strategy finished just after the competition deadline, so we submitted the 100 billion strategy instead. In terms of time, this was the result of running CFR for about 6, 9, and 11.5 days respectively.

Here's the exploitability for each of them:
62b: 102.034
100b: 104.41
130b: 106.035

For the competition, we usually do a postprocessing step to make sure that the strategy doesn't do anything embarrassingly terrible, like fold the nuts. This is left over from our very earliest equilibrium approximations, like 2003's PsOpti4 (aka Sparbot, from Poker Academy), where the abstractions were so coarse that the programs could actually make mistakes like that. Some of the corrections are more nuanced, like "don't check on the river if your hand is above a certain strength", and aren't necessarily improvements to a good strategy. We call this postprocessing an "overlord". During a match, the overlord looks at the strategy we get from CFR, and can override its decision in a few of these specific cases. With the overlord in place, the strategy that competed in this year's competition is exploitable for 125.674. So, we now know that we've gotten to the point that the strategies we get from CFR are strong enough that they don't need to be postprocessed, as the overlord is hurting instead of helping.

Back to your original question, I'm guessing the lowest point was much earlier than 62b iterations. It may have dipped under 100 mb/g briefly, which would have been a nice milestone to hit. But when we're working with large abstractions for the competition, it's kind of a hassle to measure the exploitability at more than a handful of points, so I'm not sure how low it got or where. It takes about one day to run the ABR analysis, but doing that ties up 72 processors that we could otherwise be using to make strategies for the competition.

This year's TBR strategy was adaptive, so we can't measure it's worst case performance. But last year's TBR strategy (and all the years before it) were single static strategies that we could evaluate easily.
09-12-2011 , 08:27 PM
Quote:
Originally Posted by FullyCompletely
Hi everyone,

I forwarded Herald's script on to the competition organizer. He's going to have a look, and will probably wind up hosting it on the competition website. When it's available, I'll post a link to it. It'll probably be a few more days, though - sorry for the delay.
Sorry for the delay. I've posted the scripts in this thread on the computer poker competition web site. Please let me know in this thread or in that one if you need any help with the script.

http://www.computerpokercompetition.....php?f=5&t=874

Cheers,
Herald
09-12-2011 , 10:55 PM
Quote:
Originally Posted by Herald
Sorry for the delay. I've posted the scripts in this thread on the computer poker competition web site. Please let me know in this thread or in that one if you need any help with the script.

http://www.computerpokercompetition.....php?f=5&t=874

Cheers,
Herald
Thanks! For those of us who know nothing about scripts, how do we use that file?

Last edited by jujujaja34; 09-12-2011 at 10:55 PM. Reason: spelling
09-12-2011 , 11:02 PM
Quote:
Originally Posted by Herald
Sorry for the delay. I've posted the scripts in this thread on the computer poker competition web site. Please let me know in this thread or in that one if you need any help with the script.

http://www.computerpokercompetition.....php?f=5&t=874

Cheers,
Herald
Sigh, I only have a very basic knowledge about programming and Python from 1st year college CS class, so I'll need a bit of help here.
Now I got Python installed on Windows, but don't have a clue how to run the script.
Could you please show us how to run this thing in Windows environment?
Thanks a lot.
09-13-2011 , 03:35 AM
Quote:
Originally Posted by BetaPro
Sigh, I only have a very basic knowledge about programming and Python from 1st year college CS class, so I'll need a bit of help here.
Now I got Python installed on Windows, but don't have a clue how to run the script.
Could you please show us how to run this thing in Windows environment?
Thanks a lot.
No trouble, we'll try and get you up and running.

There's a FAQ on this topic on the python website here for more information:
http://docs.python.org/faq/windows.html

Cliffs though:
1. Install python
2. Edit your path to include the python directory. This involves right clicking "my computer", clicking the "advanced" tab, and clicking properties. Then adding the directory that the python executable is in.
3. Launch a command shell window. You can do this by clicking start->run and then typing "cmd".
4. Navigate to the directory the script is in using the command "cd". For example, if your logs is currently on the c: drive you might type:
"cd c:\acpclogs", without the quotes.
5. Run the program by typing in:
"python acpcLogToFTLog.py <pathToLogFile>".

Hope that helps? These instructions are for winXP - I'm not as familiar with win7, but there should be an equivalent way to run it there.

Herald
09-13-2011 , 05:30 AM
Quote:
Originally Posted by FullyCompletely
Back to your original question, I'm guessing the lowest point was much earlier than 62b iterations. It may have dipped under 100 mb/g briefly, which would have been a nice milestone to hit. But when we're working with large abstractions for the competition, it's kind of a hassle to measure the exploitability at more than a handful of points, so I'm not sure how low it got or where. It takes about one day to run the ABR analysis, but doing that ties up 72 processors that we could otherwise be using to make strategies for the competition.
By comparing with the charts in the ABR paper it looks like it should've been below 100 mb/g by some margin, unless the much larger abstraction behaves differently.

Does the CPRG have access to all that processing power continuously or do you have to pay a fee every time you use it? It sounds like solving very large abstractions with 100+ billion iterations and running ABR against them could become very expensive in dollar terms.

Quote:
Originally Posted by Herald
Sorry for the delay. I've posted the scripts in this thread on the computer poker competition web site. Please let me know in this thread or in that one if you need any help with the script.

http://www.computerpokercompetition.....php?f=5&t=874
Finally something gives me an excuse to install python. I'm doing all my programming in C# and C++, but I've heard that python is good for this kind of stuff. Thanks for the detailed description! I will try this when I get home later tonight.
09-13-2011 , 08:56 AM
Quote:
Originally Posted by Herald
No trouble, we'll try and get you up and running.

There's a FAQ on this topic on the python website here for more information:
http://docs.python.org/faq/windows.html

Cliffs though:
1. Install python
2. Edit your path to include the python directory. This involves right clicking "my computer", clicking the "advanced" tab, and clicking properties. Then adding the directory that the python executable is in.
3. Launch a command shell window. You can do this by clicking start->run and then typing "cmd".
4. Navigate to the directory the script is in using the command "cd". For example, if your logs is currently on the c: drive you might type:
"cd c:\acpclogs", without the quotes.
5. Run the program by typing in:
"python acpcLogToFTLog.py <pathToLogFile>".

Hope that helps? These instructions are for winXP - I'm not as familiar with win7, but there should be an equivalent way to run it there.

Herald
Thanks for the help, after an hour of trying and googling, I finally went through all these steps. I'm proud of myself hehe. But looks like the program only prints all the converted logs in the command shell, so how do you output those logs as a text file in the system?
09-13-2011 , 10:03 AM
Quote:
Originally Posted by BetaPro
Thanks for the help, after an hour of trying and googling, I finally went through all these steps. I'm proud of myself hehe. But looks like the program only prints all the converted logs in the command shell, so how do you output those logs as a text file in the system?
Nice! Yeah, for those not used to the steps, it takes a bit of mucking around to get used to it.

To redirect the output to a text file, append a ">" symbol followed by a filename. Example:

python acpcLogToFTLog.py exampleLogFile.log > myFTStyleHH.txt

Herald
09-13-2011 , 01:04 PM
Quote:
Originally Posted by Herald
Nice! Yeah, for those not used to the steps, it takes a bit of mucking around to get used to it.

To redirect the output to a text file, append a ">" symbol followed by a filename. Example:

python acpcLogToFTLog.py exampleLogFile.log > myFTStyleHH.txt

Herald
Worked like magic!! makes me want to learn CS again. One problem though, seems like in the hand histories file it converted, all hole cards are shown at the end; but in HEM, it only shows hole cards from hands that go to showdown, could you maybe do something about it?
Thanks again.

Last edited by BetaPro; 09-13-2011 at 01:14 PM.
09-13-2011 , 01:25 PM
FullyCompletely,

Just curious, I was reading a thread in which 5-card stud was mentioned:
http://www.pokersyte.com/five_card_stud.htm

and it looks to me like a HU version of this game is small enough to solve using CFRM in the unabstracted game. I wasn't exactly sure how you'd calculate the size of the tree, though, since there are obviously some isomorphisms you'd want to exploit and I'm not sure how to determine them all. What do you think?
09-13-2011 , 02:02 PM
Quote:
Originally Posted by BetaPro
Worked like magic!! makes me want to learn CS again. One problem though, seems like in the hand histories file it converted, all hole cards are shown at the end; but in HEM, it only shows hole cards from hands that go to showdown, could you maybe do something about it?
Thanks again.
I noticed the same.
Also I tried to convert all the log files to txt, but it's a lot of work to do it 1 by 1 so I put them in 1 big log file and then converted it. But when I tried to import it it only imported the first 3000 hands, and the other 1.4 million were import errors Does anybody have a good idea how to import more of them at the same time?

I really like all the answers on this forum that you guys give, makes the game a lot more interesting again
09-13-2011 , 02:45 PM
Quote:
Originally Posted by Jeltsin
I noticed the same.
Also I tried to convert all the log files to txt, but it's a lot of work to do it 1 by 1 so I put them in 1 big log file and then converted it. But when I tried to import it it only imported the first 3000 hands, and the other 1.4 million were import errors Does anybody have a good idea how to import more of them at the same time?

I really like all the answers on this forum that you guys give, makes the game a lot more interesting again
I'm guessing there's a problem with hand numbers ... I'll work on a quick fix here.

For the showdown issue: I don't have an active copy of HEM installed so I can't easily test this (I'm on a mac, and there isn't a version of HEM out for mac yet ...). Can someone try adding the lines

Dealt to player1 [As Ac]
Dealt to player2 [Ts Td]

after the hole cards line in one of the hand histories that doesn't go to showdown and try importing that into HEM to see if that fixes the issue? If that works, I'll put that in the next fix as well.

Herald
09-13-2011 , 03:19 PM
Yes, it worked, add those lines after HOLE CARDS. and yes, hand history numbers makes it only possible to import 3000 hands
09-13-2011 , 03:29 PM
I got it working. The code looks kinda similar to what I would have written in C#.

This may not count as an actual bug, but I think that instead of listing every hand as having taken place on March 29, 2007, I think the script should read the time stamp of the log file and use that.
09-13-2011 , 04:19 PM
Quote:
Originally Posted by BetaPro
Yes, it worked, add those lines after HOLE CARDS. and yes, hand history numbers makes it only possible to import 3000 hands
Quote:
Originally Posted by AmyIsNo1
I got it working. The code looks kinda similar to what I would have written in C#.

This may not count as an actual bug, but I think that instead of listing every hand as having taken place on March 29, 2007, I think the script should read the time stamp of the log file and use that.
Thanks for the suggestions and looking into this. I've posted a 0.2 version that addresses these issues:

Version 0.2
- Fixed up hand history indexes to start at either a starting index passed in on the command line, or zero.
- Added 'Dealt to playerX' lines in the hand history to resolve the cards not showing up properly in non-showdown hands in HEM.
- The timestamp for each hand is not based on the timestamp of the log file being read in. Each hand takes exactly one minute (since this information is not available in the log file).

The file has been replaced in the same place as earlier. (http://www.computerpokercompetition....4&p=1231#p1231) Keep the comments coming

Herald
09-13-2011 , 05:12 PM
umm this time it shows as incomplete hands when importing into HEM, I'll check the hands to see what's wrong.
09-13-2011 , 05:16 PM
got it!!! you missed the "ET" behind each timestamp lol

and I fixed it myself!! OMG I'm awesome.

Now it works, just imported 6000 hands.

Last edited by BetaPro; 09-13-2011 at 05:32 PM.
09-13-2011 , 05:26 PM
Quote:
Originally Posted by BetaPro
got it!!! you missde the "ET" behind each timestamp lol

and I fixed it myself!! OMG I'm awesome.
Nice catch! I'll upload a quick fix here with that for the other folks.
09-13-2011 , 05:50 PM
One major problem here, I think you might have the positions reversed. In my HEM, there are pots that are shipped to the losing hands and bots folding super strong hands repeatedly.
Here:
Hyperborean-2011-2p-limit-iro|Slumbot
STATE:2515:rc/cc/crf:4h9h|8c5c/3c8hKs/5d:-20|20
in converted hand history it's Hyperborean holding 4h9h, and slumbot with 8c5c, Hyperborean checking back the flop and betting turn, slumbot folded. I think you misinterpreted the log hands; it should be Slumbot in the small blind with 85 and Hyperborean in the big blind instead.
And here:
Hyperborean-2011-2p-limit-iro|Slumbot
in HEM, it showed, Hyperborean checking back flop, betting turn, call raise, and calling river with JT, which is the losing hand, yet the pot shipped to Hyperborean. It should be Slumbot in the SB with Q9 checking back flop and calling turn checkraise, calling down Hyperborean bluff to win 80 imo.
09-13-2011 , 06:22 PM
Yeah, I also just noticed this. In the log file the players appear to be listed with the big blind first and small blind/button second. This is most easily seen in the hands where the big blinds gets a walk (just "f"); the first number is then positive.
09-13-2011 , 06:37 PM
Quote:
Originally Posted by BetaPro
One major problem here, I think you might have the positions reversed. In my HEM, there are pots that are shipped to the losing hands and bots folding super strong hands repeatedly.
Here:
Hyperborean-2011-2p-limit-iro|Slumbot
STATE:2515:rc/cc/crf:4h9h|8c5c/3c8hKs/5d:-20|20
in converted hand history it's Hyperborean holding 4h9h, and slumbot with 8c5c, Hyperborean checking back the flop and betting turn, slumbot folded. I think you misinterpreted the log hands; it should be Slumbot in the small blind with 85 and Hyperborean in the big blind instead.
And here:
Hyperborean-2011-2p-limit-iro|Slumbot
in HEM, it showed, Hyperborean checking back flop, betting turn, call raise, and calling river with JT, which is the losing hand, yet the pot shipped to Hyperborean. It should be Slumbot in the SB with Q9 checking back flop and calling turn checkraise, calling down Hyperborean bluff to win 80 imo.
Hmm. I should've spot checked more hands. I think I've found the problem and will post a fix when I have it. Sorry about that, guess I should've tested this a bit more!
09-13-2011 , 07:01 PM
Okay, that bug should be fixed. Sorry about that. The cause of that was a subtle difference in the way the acpc logs switch the button around compared to the other log files that this code used to use.

I imported logs now generated by this into PT3, and it seems to work okay, although there's some weirdness in PT3 splitting the log into two different table sessions. Not sure what that's about, but I'll take a look later on.
09-13-2011 , 07:23 PM
Hmm, still have bugs with some hands it seems like,

Slumbot|Hyperborean-2011-2p-limit-iro
STATE:794:rc/cc/crf:2cTh|3h7h/9hKh6d/4d:-20|20

Here Slumbot should be BB and Hyperborean the SB with 3h7h, while the converted HH has Slumbot in the SB with 3h7h.

The first name in the log is always BB, right?

And a minor bug with the "ET" which should be behind the timestamp and not the date.

Thanks for all the efforts.
09-13-2011 , 07:43 PM
Quote:
Originally Posted by BetaPro
Hmm, still have bugs with some hands it seems like,

Slumbot|Hyperborean-2011-2p-limit-iro
STATE:794:rc/cc/crf:2cTh|3h7h/9hKh6d/4d:-20|20

Here Slumbot should be BB and Hyperborean the SB with 3h7h, while the converted HH has Slumbot in the SB with 3h7h.

The first name in the log is always BB, right?

And a minor bug with the "ET" which should be behind the timestamp and not the date.

Thanks for all the efforts.
Hmm - I'm a little confused. What you've copied there has the names of the players first, followed by the STATE: keyword. But as far as I can tell, each hand history starts with the STATE: keyword. So that line actually reads:

STATE:794:rc/cc/crf:2cTh|3h7h/9hKh6d/4d:-20|20:Hyperborean-2011-2p-limit-iro|Slumbot

Which has Hyperborean with 2cTh and Slumbot with 3h7h. Does that not seem correct to you? The way I interpret that line is Hyperborean is in the BB. S raises, H calls preflop. Then on the flop H checks, S checks. Then the turn H checks, S bets and H folds.

Herald

Edit: I'm gonna head off since it's like 1am here. Me being a little tired probably has a lot to do with that ET bug not being fixed properly -- that and PT3 doesn't seem to care about it as much as HEM does. I'll upload a fix for the ET issue tomorrow along with any other fixes.

Last edited by Herald; 09-13-2011 at 07:52 PM.
09-13-2011 , 08:02 PM
so a log starts with the hand number then the actions, followed with cards, and player names are at the end?

EDIT: you're right, that's indeed how a log file should be read.

Last edited by BetaPro; 09-13-2011 at 08:14 PM.

m