Open Side Menu Go to the Top
Register
Help verifying a frequency-based range modeller Help verifying a frequency-based range modeller

03-12-2019 , 11:50 PM
Hello Poker Theorists,

My friend suggested that I post a message here and ask for your help verifying a frequency-based range modeller I’ve been working on. I’m hoping for input from people who are better-versed than average in poker theory to help confirm the ranges my app creates.

The tool is based off Janda’s book “Applications of No-Limit Hold ‘em” and translates a preflop range, a flop, as well as value and draw continuation percentages into a continuation range. The resulting model is fed into the next street, where you can repeat the process. The problem is I’ve been fiddling with the algorithm in isolation for too long, and I could really use some feedback from theory-savvy poker players.

At present, and on the flop and turn, the model ranks drawing hands according to their potential future value and blocker strength. On the river, bluffing hands are ranked according to their valuelessness and blocker strength.

The resulting range models look good to me but I would really appreciate some feedback on the output. If you’re interested, please check it out at https://app.rangevisor.com. I would welcome your thoughts!


Cheers,
Dave
Help verifying a frequency-based range modeller Quote
03-13-2019 , 04:16 PM
Seems to work OK, but I only spent a few minutes playing with it.
Help verifying a frequency-based range modeller Quote
03-13-2019 , 11:47 PM
Thanks for taking a look! If you have a chance to put it through it's paces, I'd appreciate any observations you might have on the output.

My goal here is to build a tool that can accurately suggest a continuation range to a novice and let them explore the hands contained therein. The underlying algorithm recommends semi-bluffs based on drawability but shades the recommendation toward blockers a bit as well.

Hoping that the mix of those two factors is about right and that the street-to-street models are reasonably accurate.
Help verifying a frequency-based range modeller Quote
03-15-2019 , 04:12 PM
Would be nice to help you verify without having to sign up...
Help verifying a frequency-based range modeller Quote
03-15-2019 , 10:28 PM
+1
Help verifying a frequency-based range modeller Quote
03-16-2019 , 03:20 AM
Quote:
Originally Posted by just_grindin
Would be nice to help you verify without having to sign up...
I definitely understand your reluctance to sign up and I've felt the same way myself. I know that signing up looks like an excuse to spam you later.

The problem is that authentication is the only way to prevent my servers and data from being abused. The app is free and I very much want your insight and input. And if I leave it unprotected, the system will be hacked, tampered with, and otherwise mistreated. So I'm sorry but authentication prevents bad things from happening.

Do you have a junk email address and can you use that to login? If not, using one is my preferred method for dealing with spam (if that's the concern).

Hope you understand and are still willing to help out.
Help verifying a frequency-based range modeller Quote
03-16-2019 , 08:57 AM
Yea I think I might have a spam email. I don't use it very often I just ignore most things that I don't think bring me value. No offense, I am sure your site is interesting.

The other thing is your site looks like a phishing attempt, which was what first raised my warning flags.

Can you just produce some dummy testing accounts? You don't have to answer here but PM if you want to provide one.

Congratulations on your project though. It's always good to get something completed like that!
Help verifying a frequency-based range modeller Quote
03-16-2019 , 11:26 AM
Quote:
Originally Posted by just_grindin
looks like a phishing attempt, which was what first raised my warning flags.
Gah, I didn't even think about that.

FWIW, the best way to avoid phishing is to use a different password, one to get into your actual email account and a separate one for sites that use that account as a login.

Authentication is hard so RangeVisor uses a third party service (auth0.com) for for that. I don't have visibility into the password you're using on the site and neither do they.

Quote:
Originally Posted by just_grindin
Congratulations on your project though. It's always good to get something completed like that!
Thanks very much! It's been a lot of work, mostly in isolation (never been a forum guy), hence my validation request here. The hard part was creating an algorithm that could replace the inherent value of small pairs with semi-bluffs that had potential value later on. You can do that easily with an equity calculation, but equity requires another hand or range to compare against, and I didn't want the tool to work that way.

So I literally spent years getting a non-equity based approach to work and in the end built a 56 billion record data set (I'm a database architect by day) to solve the problem statistically. The app runs off of aggregates built from that raw data, which is why it's so fast.

The ranges that technique produced always looked a little off to me. The drawing ranges it recommended had too many baby pairs in them until I factored in blockers. When I calculated a blocker metric and blended that statistic with drawability, those baby pairs fell away and were replaced with hands that had the potential to make strong pairs that also blocked over- and top pair hands. Those blockers (and blockees) are browsable within the tool by the way.

Full disclosure: this is the precursor to a product I hope to charge for some day. Along with the ranges it produces now, I have other features in mind that will allow users to compare different range models and tweak the internal algorithm to their preference. Getting there depends on whether there's interest in what I've done so far.

Anyway, I'm sure all that is more information than you required! No phishing. Just computational poker analytics. If you like (or don't like) the ranges the tool produces, please let me know!


Cheers,
Dave
Help verifying a frequency-based range modeller Quote
03-19-2019 , 05:15 PM
Ok, the people have spoken and signing up is pretty unpopular. I'm working on removing the login/signup screen and will post again once that's out.

Please standby, and thanks for your patience while I get this dialed in.
Help verifying a frequency-based range modeller Quote
03-21-2019 , 11:47 PM
Hi All,

I've gone ahead and removed authentication. If you were reluctant to try RangeVisor before, please consider giving it another shot now that you don't have to sign up. Thanks for bearing with me while I got that straightened out.

And if you're interested in trying RangeVisor out, here's a few ways you can use the tool:

1) The default continuation frequencies built into RangeVisor follow Ed Miller's suggestion from "Poker's 1%" that GTO continuation percentages approximate 70% with a 1:2, 1:1, and 2:1 value:bluff ratio on the flop, turn, and river respectively. Those values won't always be right, but they're a good place to start if you're trying to approximate what GTO continuation ranges look like.

RangeVisor lets you ask, what would those ranges look like if I actually used those frequencies? If 70% strikes you as too high, what do the results look like at 60%? What about 50%? Do different preflop ranges affect how appealing those continuation rates seem? What about different flops? How do different turn cards change the hands I'd see a river with? How do different rivers change the hands I'd bluff with?

2) You can also use RangeVisor to profile typical opponents' ranges. If nits start off with narrow preflop ranges and only continue to the turn with strong value and very strong draws, you can model what hands their ranges might contain by adjusting the inputs to the model accordingly. Then you can ask, what hands could a nit hold given a particular turn card? If an Ace peels off, what value hands could they still be holding?

The same goes for the archetypal maniac, who opens a wide range of hands and continues with numerous weak value hands and draws. Need some confidence to play back at them? Model and review all the junk they're taking to the turn.

I don't play online so I don't have any HUD data to draw on. But if you do, you could use a particular opponent's PFR to build their preflop range and then use their CBET frequency as RangeVisor's Continuation Percentage to estimate the range they're holding.

I'm sure there's numerous other ways to apply RangeVisor models as well. If you try any of the above, I would love to hear your thoughts on the results. If you have use cases of your own, I'd love to hear about those too.


Cheers,
Dave
Help verifying a frequency-based range modeller Quote
03-22-2019 , 06:55 AM
I am hitting an error page "Unable to configure page" with a login button shaded in red. Hitting Login returns me to the same page.

Maybe the browser on my phone but thought I'd share to let you know.
Help verifying a frequency-based range modeller Quote
03-22-2019 , 09:58 AM
Quote:
Originally Posted by just_grindin
I am hitting an error page "Unable to configure page"
Thanks for looking and for passing along the error. Does it help if you go to https://app.rangevisor.com/logout and then either click "Login" or try going to https://app.rangevisor.com?

If not, curious if it works from a different browser. If you've hit the login page before, I suspect the app may be finding confusing cookies now that it's logging in with default credentials.
Help verifying a frequency-based range modeller Quote
03-22-2019 , 10:08 AM
Having the same issue from either link and I never actually logged in, just hit the login page before you opend it up. Like I said I'm on my mobile device so it could be just the browser I use which is different.
Help verifying a frequency-based range modeller Quote
03-22-2019 , 10:48 AM
Really weird. Could very well be the browser if it's an exotic variety. I've tried Chrome and Safari from my phone and it was always able to get straightened out using those.

Maybe clear the cookies if that's not too painful?

All that said, the app isn't really optimized for mobile devices at the moment, recommend a desktop or laptop for the best experience.
Help verifying a frequency-based range modeller Quote
03-22-2019 , 11:00 AM
Confirmed it was the browser. Went to the default browser and could log in. Stuff didn't work as expected but again I am sure it's the browser.

Figured desktop would be next but I am hardly on one anymore so Idk if I'll get to it haha.
Help verifying a frequency-based range modeller Quote
03-27-2019 , 11:40 PM
I’m surprised this thread isn’t stimulating more discussion. Dave contacted me a few months ago, and I met with him personally (on a recent trip to Denver) and can vouch for the amount of work he’s put into this model (like 2-3 years). And there isn't any other tool I know of that generates these "Janda optimal" betting/calling ranges (for a lack of a better term).

Dave, maybe you can explain how you do this so fast? I'm not a database architect but it would seem like the amount of number crunching required (for each analysis) would take a long time.
Help verifying a frequency-based range modeller Quote
03-28-2019 , 02:07 PM
I didn't spend much time on this, but I had a few quick observations/suggestions.

Need a clear all/ reset. If there, could not find it
Need a restore to default ratios 1:2 1:1 2:1
Need an example or two in the instructions
Need guidance on setting ratios other than default

Program is interesting. I'll try to spend some more time with it.

I wrote a simple Excel/VBA program on value to bluff bet freqs for indifference, with value bets < 100% and bluff bet > 0%. Can/does your program handle that?
Help verifying a frequency-based range modeller Quote
03-30-2019 , 01:16 AM
Quote:
Originally Posted by Steve
Dave, maybe you can explain how you do this so fast? I'm not a database architect but it would seem like the amount of number crunching required (for each analysis) would take a long time.
Hey Steve! While there's a lot of data wrangling that happens at run-time, by the time you build your model, most of the work's already been done.

I mentioned before that I calculated statistics for every possible hand on every possible flop, turn, and river (roughly 56B different runouts). All that data gets rolled up into pre-calculated aggregates that RangeVisor uses to rapidly construct its models. The data is all compressed in the cloud, and is pulled into the app when a model creation request is received. It's essentially an exercise in real time computational poker data mining.



Quote:
Originally Posted by statmanhal
Need a clear all/ reset. If there, could not find it
Need a restore to default ratios 1:2 1:1 2:1
Need an example or two in the instructions
Need guidance on setting ratios other than default
Thanks for digging into the models, statmanhal. These are all great suggestions and I will get to work on them.

Quote:
Originally Posted by statmanhal
I wrote a simple Excel/VBA program on value to bluff bet freqs for indifference, with value bets < 100% and bluff bet > 0%. Can/does your program handle that?
RangeVisor doesn't do those calculations (yet anyway), but I think they'd make excellent inputs into the model as Value and Draw Continuation percentages. The defaults that are in there now, are meant to create indifferent river opponents, but they are approximations. I've fiddled with those calculations myself and the best I came up with was this:

Code:
  bet  caller_break_even
-----  -----------------  
0.250              0.167              
0.333              0.200              
0.500              0.250              
0.667              0.286              
0.750              0.300              
1.000              0.333
So for a pot sized river bet (1.000 above), a caller needs to be correct 1/3 of the time to break even. The caller would be indifferent if your betting range matched a 2:1 ratio of value hands to bluffs. A 2/3 pot river bet (0.667 above) means a caller would be indifferent if your mix of river bets matched a 5:2 ratio of value to bluffs, right?

What I haven't been able to figure out, is how to carry those values backward from the river. If we need that 5:2 mix on the river, and assuming we're firing a 2/3 pot bet on every street, what should our mix on the turn be? What about the flop?

Is it 2:5, 1:1, and 5:2? Or is that too simple?
Help verifying a frequency-based range modeller Quote
03-30-2019 , 08:59 AM
Quote:
Originally Posted by RangeVisor
Hey Steve! While there's a lot of data wrangling that happens at run-time, by the time you build your model, most of the work's already been done.



I mentioned before that I calculated statistics for every possible hand on every possible flop, turn, and river (roughly 56B different runouts). All that data gets rolled up into pre-calculated aggregates that RangeVisor uses to rapidly construct its models. The data is all compressed in the cloud, and is pulled into the app when a model creation request is received. It's essentially an exercise in real time computational poker data mining.











Thanks for digging into the models, statmanhal. These are all great suggestions and I will get to work on them.







RangeVisor doesn't do those calculations (yet anyway), but I think they'd make excellent inputs into the model as Value and Draw Continuation percentages. The defaults that are in there now, are meant to create indifferent river opponents, but they are approximations. I've fiddled with those calculations myself and the best I came up with was this:



Code:
  bet  caller_break_even

-----  -----------------  

0.250              0.167              

0.333              0.200              

0.500              0.250              

0.667              0.286              

0.750              0.300              

1.000              0.333


So for a pot sized river bet (1.000 above), a caller needs to be correct 1/3 of the time to break even. The caller would be indifferent if your betting range matched a 2:1 ratio of value hands to bluffs. A 2/3 pot river bet (0.667 above) means a caller would be indifferent if your mix of river bets matched a 5:2 ratio of value to bluffs, right?



What I haven't been able to figure out, is how to carry those values backward from the river. If we need that 5:2 mix on the river, and assuming we're firing a 2/3 pot bet on every street, what should our mix on the turn be? What about the flop?



Is it 2:5, 1:1, and 5:2? Or is that too simple?
Mixing like that might be an alright approximation but definitely too simple when we have simulator software like pio, monker, GTO+, etc. that all look at EV directly.
Help verifying a frequency-based range modeller Quote
03-30-2019 , 05:12 PM
Quote:
Originally Posted by RangeVisor
(roughly 56B different runouts).
Are you getting that number from 1326 starting hands x approx 20,000 flops x 47 turn cards x 46 river cards (or something like that)?

That does seem to come out to about 56 billion.

What do you do about hands that have a mixture of both drawing potential and value on the flop?
Help verifying a frequency-based range modeller Quote
03-30-2019 , 06:11 PM
Quote:
Originally Posted by RangeVisor
1) The default continuation frequencies built into RangeVisor follow Ed Miller's suggestion from "Poker's 1%" that GTO continuation percentages approximate 70% with a 1:2, 1:1, and 2:1 value:bluff ratio on the flop, turn, and river respectively. Those values won't always be right, but they're a good place to start if you're trying to approximate what GTO continuation ranges look like.
The problem with ^this^ is that Miller and Janda were wrong. Betting and barreling 70% is not very close to what optimal play really looks like. (Betting frequencies are usually much lower than 70% on the turn and river).
Help verifying a frequency-based range modeller Quote
03-30-2019 , 08:44 PM
Quote:
Originally Posted by Steve
Are you getting that number from 1326 starting hands x approx 20,000 flops x 47 turn cards x 46 river cards (or something like that)?
Yes, and to geek out a bit here, there's only 1176 hands left in a deck after a flop has been dealt. 22100 flops * 1176 hands * 47 turns * 46 rivers = 56,189,515,200. Alternately, there's only 19600 flops possible after you've been dealt two cards: 1326 hands * 19600 flops * 47 turns * 46 rivers is also 56,189,515,200.

Anyway, using that dataset I statistically identify the likelihood of each hand having value by the river, aggregate those results, and then pick the best draws from those aggregates, which is much faster than grinding through the whole dataset every time RangeVisor builds a model.


Quote:
Originally Posted by Steve
What do you do about hands that have a mixture of both drawing potential and value on the flop?
Great question, thanks! RangeVisor assigns value hands first and then picks its draws from whatever is left of your preflop range (or your previous street's continuation range). So if it's a monochrome flop and you're holding AA with the A of flush, RangeVisor still assigns that hand to the Value Range and tries to build a Drawing Range of the requested size.

Those draws will tend to include some second- and/or middle-pair hands, especially if they have straight- or flush-draw potential. So if you tell it you want to build a Flop continuation range with 20% value hands and 40% draws, the remaining 40% of your preflop range gets assigned to a checking range (presumably the check-folding range).

Building a model kicks off these steps:
  1. Discard the dead hands from your preflop range that are blocked by the board
  2. Rank value hands according to their current value. The top 20% (from the 20% Value Continuation Percent we requested) are assigned to the value range, the rest are considered to be drawing candidates.
  3. Draw candidates then get scored for their potential value by the river. This is where the aggregates I mentioned above come in and RangeVisor uses that data to pick the best draws. It also factors blocker strength in here so draws that also block value hands get ranked higher than the same draw without the blocker potential. Based off that drawing index and the 40% Draw Continuation Percent we asked for, the best 40% get assigned to the drawing range.
  4. Those Value and Drawing ranges get combined into the combined Continuation Range.
  5. The remainder gets allocated to the checking range

All of this repeats from street to street where the flop continuation range becomes the previous range for the turn, etc. The only difference is that on the river, we're assigning bluffs (not draws); RangeVisor considers the hands with the least actual value and the best blocker strength to be the best bluffs. I'd argue in this case the checking range ceases to be a check-folding range and some of the hands in RangeVisor's river checking ranges should be treated as check-calls instead.

See below for an illustration of the process:
Help verifying a frequency-based range modeller Quote
03-30-2019 , 10:10 PM
Quote:
Originally Posted by ArtyMcFly
The problem with ^this^ is that Miller and Janda were wrong. Betting and barreling 70% is not very close to what optimal play really looks like. (Betting frequencies are usually much lower than 70% on the turn and river).
Thanks, Arty: this is a great and very important point. RangeVisor uses 70% continuation with 1:2, 1:1:, and 2:1 value:bluff ratios as a default and I hope that isn't taken as a recommendation.

Recognizing that 70% is too high, I needed to set the defaults somewhere so I took them from the literature. And I'm happy to adjust those defaults downward if there's a better, well-founded recommendation.

RangeVisor is meant as a range modeling and exploration tool. You can set the frequencies you like and see what the results are immediately. If you're trying to create indifferent opponents on the river, you can adjust your ratios accordingly and study the results.

IMHO, it's this process that's important and fiddling with RangeVisor can show a variety of interesting things. I've had a number of insights using the tool, from "Oh yeah, I can't have KhQh here because I folded that on the flop" to "Wow, rainbow flops block more two-pair combinations than monochrome ones."

What's more, you can model sub-optimal ranges as well and get an estimate of what an archetypal opponent might be holding in a particular spot. And as I mentioned earlier, I think HUD stats would make really interesting and compelling inputs for RangeVisor models. Definitely not my area of expertise, so if anyone has a good HUD database I'd love to hear if those stats produce a useful estimate of an opponent's range.

Anyway, I digress. Bottom line, RangeVisor's defaults are not recommendations. If anyone has a default they'd recommend, I'd love to hear it as well as hear how the resulting ranges look when those recommendations are used.
Help verifying a frequency-based range modeller Quote
03-31-2019 , 07:55 AM
Your default is probably as good as any since the default continuation ranges will be dependent on the flop, turn, and river.
Help verifying a frequency-based range modeller Quote
04-01-2019 , 12:14 AM
Quote:
Originally Posted by just_grindin
Your default is probably as good as any
Thanks, just_grindin: will leave it as is for the time being.

Quote:
Originally Posted by just_grindin
default continuation ranges will be dependent on the flop, turn, and river.
Good point and I'd add it depends on the preflop range as well. I've been doing a bit of work on this by trying to rank flops and adjust continuation frequencies by how favorable the flop is for one's preflop range. Still a work in progress, but if I get that sorted, it would be cool to work those metrics into the model.
Help verifying a frequency-based range modeller Quote

      
m