So it can be added to the digest of commonly posted about topics- now there will be an authoritative discussion on the matter that will serve as a great point to direct those that start threads like:


http://forumserver.twoplustwo.com/78...ng-bad-816120/

http://forumserver.twoplustwo.com/78...justed-808637/

http://forumserver.twoplustwo.com/78...all-ev-782335/

http://forumserver.twoplustwo.com/78...all-ev-778644/

http://forumserver.twoplustwo.com/78...survey-739720/

Agreed. Thanks Funky! Not an easy topic, and a very nicely constructed OP.

"It's just a stat, it is what it is. Sure I look to see if I'm above or below my expected value but I don't consider this one stat by any stretch of the imagination to be an indicator of a person's poker success or abilities. They have nothing to do with one another. I think the EV graph was added so math nerds could have something else to contemplate and analyze. No offense to math nerds."

Also, gives something for people to whine about while being completely blameless.

You are correct. I should have said binomial rather than normal. That said, this mistake does not have a material impact on the claim being made. binomial distributions still have variance and standard deviation and you can run above or below expectation in any particular category that is a binary event (e.g. hero was dealt AA, did a villain get dealt KK? Hero had KK and a villain had AA, was villain a loose player or not?)



Here you caught me exaggerating when I said "for every situation there is a normal (binomial) distribution".

What I really meant to say was that for every binary category there is a binomial distribution. While every distribution in poker that can be conceived of is not a binomial distribution it is true that binomial distributions turn up very frequently when we go looking at all the places where luck plays a factor.


I stand by my wording corrected claim that the "binomial distribution turns up with a fractal like regularity when we look at how luck affects our results in poker" and I acknowledge your point that there are stats (e.g. pot size) that have a different distribution.

I'm always amazed at how well some of you guys are able to dig up previous topics.

This is definitely a hot topic and a ton of members, as you illustrated, have questions about their EV. The difference I see between this topic and previous CoTW topics is that in the past they have been related directly to improving one's game citing how to handle certain situations, theories, and strategies. Like I mentioned before, EV is what it is. There's no way to change the outcome of your EV when it's measuring the results of AI situations prior to the final card(s) being delt.

Anyways, it was a random thought not meant to detract from the solid discussion it has sparked or the efforts of the OP which are/were top notch!

Why would the mean (i.e. EV) of our lifetime AIEV be expected to be 0? Perhaps you mean that the distance of our actual results from our EV is expected to be 0?


First, as was pointed out by another poster, I should have said binomial distribution rather than normal distribution. Since, for large sample sizes, the normal approximates the binomial fairly well this is not a big mistake IMO.

Let us get more precise with an example:

Consider my sample of hands discussed in the original post. There we 389 AIEV events (situations) in this sample. For each AIEV event, there is a pot size and a hero equity. In the C programming language we might notate this thus:

Code:

typedef struct {
      double  pot_size;
      double  hero_equity;
      double  hero_bet;
} aiev_event_t;

// start  C array index at 1 rather 
// than 0 for the laity
aiev_event_t    aiev[389+1];

And we might describe the 3rd AIEV in the program thus:

aiev[3].pot_size = 97.22;
aiev[3].hero_equity = 0.8213333;
aiev[3].hero_bet = 22.0;

Now, to calculate the EV for this sample of 389 AIEV events, we simply:

Code:

total_ev = 0;
for(i=1; i <= 389; i++) {
   total_ev += aiev[i].hero_equity * aiev[i].pot_size
        - aiev[i].hero_bet;
}

Let us define the following terms:

• AIEV profile: the array that describes a series of AIEV events (such as my aiev[389+1] array above).
• AIEV trial: simulating each AIEV event in an AIEV profile once and recording the result.
• AIEV trial total: the sum of the results from each simulated hand in an AIEV trial.

given an AIEV profile and a computer it is a trivial matter to write a program that will simulate as many AIEV trials as we like.

Now, my actual real life result in the 389 AIEV hands of my sample can be viewed as a single AIEV trial of my aiev[389+1] profile, as defined above. Given computer technology, it is a simple matter to simulate as many AIEV trials as we want for a particular AIEV profile.

Given any particular AIEV profile <P> and a large sample of AIEV trial totals for profile <P>, is there any doubt as to the shape of the resulting histogram of AIEV trial totals? Do you doubt that we can determine the standard deviation of profile <P>? Please explicitly comment on this point as it is key.

I really don't feel like writing the program to do this AIEV profile simulation myself but perhaps I can convince the PT3 team or author of HoldemLuck to do this as they are already 90% of the way there. I could, in short order, write a toy program that given a user created AIEV profile, would run a number of AIEV trials against the profile and generate a histogram. It is the SQL programming and glue code for getting actual data out of PT3 or HEM's database that is more work than I want to do.

For argument sake, assume every NLHE poker hand you have ever played is in your PT3 (or HEM) database. When you die, we can create an AIEV profile for every AIEV situation you ever played. In HEM and PT3 we can view both your AIEV (i.e. the EV line) and your actual results (the money won/lost line) and measure the difference between these two.

Given the AIEV profile for your life we can also calculate the standard deviation for that AIEV profile and determine how far your actual results deviated (in units of standard deviations) from EV.

I hope that is less vague. As a professional computer programmer who spends most of my days talking with other computer programmers I often erroneously assume that everyone can imagine the above description if I merely hint at it. (I suspect this happens to most specialists who spend most of their time dealing with folks with self similar skill sets)

the probability of being > 1 standard deviation from the mean does not change with sample size. The size of a standard deviation does change with a sample size.

Consider a bet where we are getting the best of it: a coin flip (50% heads, 50% tails) with a payoff matrix of:

• heads: we win $1.5 (bet $1, win $2.5 for a net of +$1.5)
• tails: we lose $1 (bet $1, lose $1 for a net of -$1)

for each flip:

• EV = + $0.25
• stddev = 1.25

Statistics tells us that the standard deviation for <N> trials of an event is:

stddev(N) = sqrt(N) * stddev

while

EV(N) = N * EV

so the standard deviation for 100 coinflips is $12.5 and the standard deviation for 10,000 coinflips is $125.

It is true that the signal to noise ratio of our EV to stddev increases as the sample size gets bigger. I.e. our win rate dominates more and more as the sample size gets bigger but that does not mean that, in absolute dollar terms, our standard deviation gets smaller with a larger sample size.

Putting this in practical poker terms:

Assume:

• we play 100NL for the rest of our life
• our win rate remains constant at 4bb/100 (we are balla)
• our standard deviation is 70bb/100
• we play 10,000,000 hands of 100NL before we die.

In absolute dollar terms we expect the difference between our all-in-EV and our all-in results to increase over time.

Here is an interesting question for a poker player: consider a single all-in preflop situation with effective stacks of 100bb. Which has a higher standard deviation (or variance):

• 22 vs AKo (i.e. a coin flip)
• AA vs KK (serious domination)

Assume you are getting the best of it in each of those situations, what is the signal to noise ratio for the situation?

The second statement is also false. The average absolute deviation from the mean tends to get larger over time, not smaller. Only the proportion relative to the mean tends to get smaller.

However, I think the first statement is actually closer to the truth. The average all-in equity of all players combined is always exactly 50%, and many players will cluster around a 50% average equity for their all-ins. Good tournament players seem to have a slight tendency to have an average all-in equity of a bit under 50%, because they understand chip value and ICM. But I would guess cash players tend to average very close to 50% equity for all-ins. If all bets were equal then 50% equity is equal to 0EV, but of course all bets aren't equal, and there is no reason to assume all-in bet sizes average out. In samples I've looked at, they don't.

yes, it is false. I was limiting myself to trying to guess his original intended meaning.

In my post above I go through an example where I show that the absolute (dollar) value of standard deviation goes up as the sample size increases.

This ignores the rake, which is significant in the micros. Therefore, it is not 50%, but can be as bad as 47.5% overall depending on the stakes. A good cash player should be significantly positive in AI equity. I'm certainly not good, but I'm running over 60% equity when AI, even though I'm 5% below my expected AIEV for the last 18 months. I'm beginning to wonder if the RNG sees 2+2 activity and decides to punish it.

Wait, why would this hold for AIEV since you can have positive and negative deviations?

It is possible to have an actual result that is identical to the expected result but the standard deviation grows as the sample size gets larger so in terms of dollars, your expected result is likely to be farther and farther from EV as the sample size gets larger. play around with uDevil's Poker Result calculator to see this. In particular, click on the total winnings button.

If instead, you compare how far from EV you are to your total EV:

absolute_value(actual_result - EV) / EV

this fraction is likely to shrink. This is what folks are refering to when they say "standard deviation gets smaller with a larger sample size" -- they mean that relative to the total expected value, standard deviation gets smaller.

The default graph is for a win rate of 2, stddev of 16 and a confidence interval of 95% (approximately +-2 stddevs) and 200 periods.

for any win rate and standard deviation the upper and lower red lines get farther and farther apart as we move to the right along the periods axis.

If you want to see a confidence interval that represents a single standard deviation then put "68.26" into the confidence box of the graph tool.

For this tool, the units for WR, stddev and period can be what ever you want as long as they are consistent. Here at 2+2 we generally use "100 hands" as the base unit.

If you click repeatedly on the "chart it" button of the tool it will redraw the black line which represents a single simulation of a player's performance given the win rate and standard deviation. Sometimes the black line runs bad, sometimes it runs good. On rare occassions it will reach the right side of the graph near the green EV line.

In NLHE, I think a standard deviation of 50bb/100 is considered quite low and a standard deviation of 90bb/100 hands is quite high. My standard deviation/100 has stayed around 60bb for quite some time now and I believe this figure is fairly consistent if your playing style does not change.

Look at graphs with periods of 200, 2000 and 20000.

Some links about that?

No. Equity calculations are not affected by rake. You either win the hand or you don't. On the other hand, when doing EV vs. results (which is $, not hands) then the comparision is always done before rake. You must be thinking of winrates and the effect of rake.

That's fair.

+1 to zachvac and +1 to mpethybridge!

Time to shime in...

Tiny background: I've written a $AIEV engine that can import (from scratch) and perform $AIEV computation as fast as 17 000+ hands per second (it's not a typo). One beta-tester managed to reach such speeds on an octo-core (granted, on a normal machine you'll 'only' be running at 2000 or 3000 hands per second). So using that software I wrote, you can import 2 million hands in less than two minutes and instantly plot a shiny real results + $AIEV graph

First, the title of this CoTW is a strawman right? It's a logical fallacy, but I'm not sure of it's name (I'm not a native english speaker).

It says "Why all-in-EV is a horrible measure of overall luck" but nobody ever said that. Never ever. It has already been pointed in this thread but it bears repeating. This entire CoTW is a strawman.

Neither the HEM nor the PT3 definition are saying it's a measure of overall luck.

zachvac is right on spot with this: if you think that money won is a better indication than the all-in ev line is of expected winnings you have a super flawed understanding of what ($AIEV) luck means and what all-in ev is.

The $AIEV is computed when perfect (spacebidder would say near perfect) information is available: an $AIEV computation for a particular has no "past memory" (no betting history, no nothing). The only thing an $AIEV does is computing the indisputable mathematical expectation value once perfect information is available (what happened previously in the deal has no effect and no player action can happen anymore).

What that value represent is very simple (yet very often not understood at all, including by people with 4-digits+ posts).

The $AIEV represents your actual results, from which one luck factor, namely the variance associated to pre-river all-in luck, has been removed. If you believe this is pointless, then you should never ever look at your real results, because they're even more pointless

The only thing that is right is that there's a TINY card removal effect. The one spacebidder is talking about, altough he's IMHO unnecessarily dramatizing the effect of card removal in that case. It is tiny. It has nothing to do with the bias in the street-by-street-EV-mega-fiasco towards hands that hit the board/are willing to go to showdown.

From spacebidder's very interesting research, you can see that an ace is actually 2.5% less likely then what "normal" AIEV computation consider and a deuce 2.5% more likely or so.

Let us now be serious: an all-in occuring at flop means there are, typically, C(45,2) possibilities to run to do a full exhaust. How much do you think having a deuce 2.5% more likely and an ace 2.5% more likely will change the mathematical expectation of the $AIEV? It is going to be tiny. Negligible. I'm willing to change my $AIEV engine for research purpose to take spacebidder's numbers/card removal effect into account but I'm not holding my breath: it is going to be a tiny insignificant number.

Even by taking this tiny card removal effect into account, the $AIEV still gives a number that is going to be much closer to your theoretical results than your real result.

In addition to that, there's something else very important to point out: if you sum the "$ - $AIEV" of all the players in your DB, then it will add up to exactly... zero! (or your $AIEV engine is severely broken).

So, ok, there's a tiny card removal effect, but for each deal, either the fact that an Ace is 2.5% less likely will either disadvantage you or advantage you OR neither advantage nor disadvantage you (because both you and your opponents are affected in the same way by the tiny card removal effect). And unless you have a very weird playing style, the tiny card removal effect will tend to always cancel out (for all the allins it is going to advantage you, you'll have about the same number of allins where it is going to disadvantage you).

So IMHO even if we consider the tiny negligible card removal effect that spacebidder is taking into account, it will totally cancel out.

On the subject of rake: my tracker performs all the $AIEV computation, for every pot, with the pot amount after the rake has been deduced. There's zero rake issue in that computation, it's a non-existing problem.

A CoTW entitled "What is really the $AIEV"" would have been much more constructive. But here I'm tempted to call this whole CoTW strawman and FUD because once again: nobody ever said that the $AIEV was a measure of overall luck

A thread wouldn't be complete without a car analogy right?

"CoTW: Why a Formula 1 is a horrible car to win Pike's Peak"?

A Formula 1 is a horrible car on gravel, hence it is useless.

Erf, what!? It is useless to win Pike's Peak, sure. Just as the $AIEV is useless to measure overall luck. That's not what a Formula 1 does. That's not what the $AIEV does. It makes neither of them useless.

And most of all, it makes absolutely zero sense to start a discussion around the "concept" of "Why a Formula 1 is a horrible car to win Pike's Peak"...

Once again, zachvac is right on spot: if you think that money won is a better indication than the all-in ev line is of expected winnings you have a super flawed understanding of what ($AIEV) luck means and what all-in ev is.

Now I'm a stupid person: I just managed to connect two neurons to write a software able to import at speed up to 17 000+ hands/second (and which also can do things like 3-ways preflop ranged "22+ vs AJs+ vs 88+" full enum 40 times faster than PokerStove) and maybe I'm all wrong on this

But still, I stand with zachvac.

The $AIEV problem is people thinking that it's a measure of overall luck and people saying that the $AIEV is useless because it's not a measure of overall luck.

Now I'm willing to modify my $AIEV engine for research purpose if someone is willing to send me 50 000 or so deals with an all-in postflop (and pre-river) [that should amount to 8 MB zipped or so] and put the numbers spacebidder came up with and see what gives. I shall show a tiny insignificant card removal effect that tends to always cancel out...

IOW the $AIEV adjusted using spacebidder's numbers will be very close (nearly identical) to the plain $AIEV and it will still be a much closer representation of your theoretical winrate than what your actual results are.

I'm all for a "CoTW: Really understanding what the $AIEV is", made using math and logical argumentation, not logical fallacies

You shouldn't try to put a bias correction in the calculation. As you point out, preflop it is small. Post-flop the variability of the effects is going to be bigger and it isn't quantified and perhaps not even quantifiable (without a large sample containing all players' hole cards). The removal effect is just something to be aware of, and know that AIEV is not 100% accurate, but when calculated on preflop all-ins with a single caller it is accurate enough.

As to your discussion about the point of this thread, you obviously haven't seen the hundreds of threads where users equate their AIEV deviation to "running bad" in general, and they refer to it as just being under EV, not being under only on all-ins.

This.

------------

TANGENT: TacticalCoder, if you run 10,000 simulated trials against a single AIEV profile (e.g. the AIs from your last 100k hands) what is the shape of the resulting histogram? A binomial distribution? Are you able to give us the standard deviation of this distribution?

EDIT: I'm not saying you dispute this point but it might help others if you could post a graph of this.

Please, do write this.

You seem to have the skills and energy to address the question "what portion of all the luck in poker (NLHE) is AIEV luck" so I look forward to seeing this treated in your CoTW.

EDIT:
In terms of my poorly written CoTW this would be:

luck-a / (luck-a + luck-b)

I did... But I find it sad that instead of explaining what the $AIEV is this turns out into a discussion needlessly bashing the $AIEV by attacking it as something it is not, and then there are people in this thread commenting:

that's why $AIEV is useless

or:

that's why I never don't use AIEV

or "+1" without knowing what's going on

or needlessly bashing the two major Windows tracker as if they were deceitfully misleading their users.

Which IMHO is a major fail if the point was to educate micro-limit players as to what the $AIEV is.

The fact that I've seen all this posts is precisely why I wrote that I'm waiting for a more constructive CoTW called something like "What does the $AIEV really compute".

But "Why all-in-EV is a horrible measure of overall luck" simply makes no sense, it is a strawman.

Make it "Why all-in-EV is not a measure of overall luck" or [i]"Why all-in-EV is only a measure of all-in luck"[i] (shockers )

I think the more pertinent question is, what portion of variance from expected results is due to all-in variance from mean equity. My guess is that in tournament play it's probably around half. In cash play, not so much.

But I see no way the question can ever be answered accurately either.

we'd need to check with someone who actually knows math but I think the method has already been mentioned. PT3 (alledgedly) does a good job at estimating variance (stddev) of our overall game given our HH DB. If we run the same stddev calculation on the HH DB but with every AIEV situation replaced by the EV calculation rather than the actual result, have we not factored out AIEV luck? Is not the resulting standard deviation luck-b?

But then we're talking about something entirely different...

And it's yet another logical fallacy:

Me: "Langur monkeys can glide from tree to tree"
You: "Tell me how close to flying the Langur monkeys' gliding abilities is" (implying that it is very small for they can't fly like most birds and that hence their gliding ability is useless).

This simply makes no sense at all. Non-sequitur.

Even if it a small part of luck (even if Lankur monkeys can't really fly) it is still quantifiable and it still gives a number closer to your real results than what your actual results are.

My point is: your whole CoTW is a strawman for nobody ever stated that AIEV was a measure of overall luck. If people wrongly think that, then educate them by honestly showing them what the $AIEV is, instead of telling them "all-in luck is just one luck factor amongst many, hence $AIEV is useless".

This thread is a useless bash of $AIEV and keeps ignoring people explaining what the $AIEV is. So obviously the goal is not to explain to user what it is, but to keep on spreading fear, uncertainty and doubt

Btw, people will have to wait for me to reach pooh-bah before I can do a CoTW, I was just hinting that I'd like another pooh-bah to write something about it

You always dodged the real point made to you in answer to your strawman: by both me and zachvac (and acknowledged by others): the $AIEV is closer to your theoretical results than your actual results are.

What is your point of view on that?

It's not about "+1" and "this" and "A Formula 1 sucks on gravel so it is useless" and "Lankur monkeys not being able to do vertical-take-off/vertical-landing" and SDs, it's about honestly explaining to user what the $AIEV is instead of explaining what it is not. I don't care about what it is not. I care about what it is.

And the $AIEV is closer to your theoretical results than your actual results are.

And that is a fact, but I don't expect much more than non-sequiturs (like your tangent) seen the strawman that this entire CoTW is, with obviously the sole purpose of needlessly portraying the $AIEV in a negative light instead of really explaining what it is

To straighten things out, before starting again with moon phases and other whatnots:

"The $AIEV is closer to your theoretical results than your actual results are."

True or false? (shouldn't take the skills and energy you're attributing me to write the essay you'd wish I would write to answer my question by "true" or "false")

If you answer "true but irrelevant", then what about a micro-limits player (which is what this forum is about) playing with an aggressive bankroll strategy that is running 15 buy-ins above or below EV on small samples? Is looking at the $AIEV really that irrelevant "because there are many other luck factors" and should he keep looking instead at his actual results?

P.S: Langur monkyes are very cool to watch by the way, YouTube probably has videos of them gliding and it's impressive

wtf, this thread needs a warning - Only for smart people, I do agree w/ zachvac tho cause if all luck is randomized then measuring even 1% of that random luck and drawing a conclusion from it (aiev graph) will draw a more accurate conclusion on overall skill that 0%, therefore when comparing the skill of 2 players the ev line is more accurate than profit line. Oh by the way i'm 3k over ev this month, holllllllllllllllllllllllllld.

Also funkyj, do you know or have an idea of what percent of overall "luck" ev is?

I'm pretty darn sure (99.99%) that you can write the essay "AIEV explained", post it here in the uNLFR forum and the mods will not delete it.

Mpethy, split, Kurt:

1. can you confirm that you would not delete an essay on EV by TacticalCoder?
2. For an essay to be labelled a CoTW does it have to be authored by a poohbah?

I sincerely believe you (TacticalCoder) have the intellectual ability to write a good essay explaining AIEV. I'm less confident that you have the strength of character to do more than carp and epeen wave. Now you might think "that funkyj is merely insulting me and he secretly hopes I do not write an AIEV essay that is better than his since my doing nothing would make him look better". Nothing could be further from the truth. I would love to see you write an awesome essay that I could read and learn from. Are you up to the task? I would think that someone who has written better software than pokerstove would be.

Since I doubt the quality of your motives in this discussion (they seem to me to be destructive rather than constructive) this will likely be my last post responding to you. You, of course, are welcome to drone on and on.

I will summarize some key points of our discussion so far.

(1) The main thesis of my CoTW is that many people who post in this forum mistakenly think that if they are running well below AIEV in their actual results then their overall poker luck has been very bad during this stretch.

I stand by this thesis. You claim that this viewpoint is non-existent. I think we must agree to disagree on this point.

As part of (1) I attempt to show, qualitatively, that AIEV luck is a small fraction of overall poker luck. In the ensuing discussion there have been ideas about how this might be quantified. If, say, AIEV is 10% of all poker luck, then running 1 stddev below AIEV and 1 stddev above in all other luck (AKA luck-b above) could very well give the player a much better result than the opposite.

I'm not sure whether you dispute this point or not and I don't feel it is worth my time to poor over your previous posts to find out.

(2) I did say that comparing both results and AIEV adjusted results graphs was pointless. Cangurino accused me of being vague in many places and rightly so. This was one of the many statements I made that was exceedingly vague. All vague statements are chock full of assumptions and this vague statement on my part assumed that we would be comparing graphs (EV adjusted or not) of players who were close in skill.

Mpethy's main comment on this point was "given a sample size, two player's win rates and standard deviations we can quantify the likelyhood that player A is better than player b" and I agreed. This is a much better response than my original vague, chock full-o-assumptions, statement. Even better would be to create an example with numbers and work though the example. E.g.

sample size for both players is 100k hands. player A has an actual win rate of 3bb/100 and player B has a win rate of 2bb/100. Both of their standard deviations are 65bb/100. What is the likelyhood that player B's true win rate is higher than player A's true win rate?

For the above example we could work it with AIEV adjusted results but we would then need an AIEV adjusted standard deviation (PT3 and HEM do not yet supply this number). An alternate question might be "given the actual WR and standard deviations above, how many hands would we need in the sample before we were 70% confident that player A was a better player than player B".

With these sort of questions we can answer precisely rather than make vague statements about "useless" or "bad measure of".

(3) you ask me to respond "yes or no: are AIEV adjusted results closer to a players true win rate than actual results"?

I answered this question before your first post in the thread (grunch).

(4) I definitely agree that many people do not misuse AIEV. For them this CoTW is of little or no value.

I am sorry that when you opened this thread you did not find the essay you had hoped for. I think that in your frustration you misinterpret much of what I say.

I will, one last time, urge you to give something to this community by writing your own AIEV essay rather than wasting more time by arguing with me.

There was never any question which (actual results or AIEV adjusted results) is likely to be closer to your true win rate.

The real question is (as stated in a post above) given:

1. win rates for two player over a sample size
2. standard deviations for the players over the same samples
3. what is our confidence that the player with the higher win rate is the better player

PT3 and HEM will give us both actual and EV adusted win rates but they only give us actual standard deviations (no EV adjusted standard deviation). It seems likely that EV adjusted standard deviation is the same or lower than the actual standard deviation. (I would be shocked to see any player for whom, over a sample of 100k hands, had a higher EV adjusted standard deviation for their over all play).

Because I am lazy, I simply said "it is useless to compare graphs unless you have a very large sample" but all the numbers and math are available for us to be able to quantify the probability that the player with the better results over the sample is actually the better player (i.e. has a higher true win rate).


nice


As mentioned in the discussion above, we are not sure what this is but we have some guesses as to how we might find out.

I personally find this question interesting (for nerd reasons, not practical reasons) so I might bother to investigate this subject in the future.