I’ve often thought it would be interesting to aggregate the responses to LLSNL hand history posts to look for patterns in the responses. How often do the responses agree on what action OP should take, how often do the responses disagree, and (in cases where OP provided results) how often did we collectively make the correct decision?

So I spent an afternoon logging the responses to a total of 100 LLSNL posts, capturing a total of 535 responses from 199 different users.

I decided to focus specifically on posts describing river decisions, where the original poster is facing a bet or raise and is deciding between calling and folding. I’m especially interested in these spots because, as I’ve described before, I don’t think that many of us are good at deciding whether calling or folding is more likely to be profitable in these spots. End-of-action river decisions were also a good choice because they make the analysis simple: since there are only really two options available (call or fold), analyzing the data is more straightforward than on earlier streets where we have to think about multiple bet sizes, cards to come, and so on. I excluded posts where OP was considering actions other than calling, such as raising as a bluff, though there were only a few.

Data collection
Like I mentioned, I aggregated the responses to a total of 100 LLSNL posts, capturing a total of 535 responses from 199 different users. The raw data is available in this Google Sheet – notice there are multiple sheets. I classified each user’s response as either “call” or “fold”. If the post didn’t specifically recommend a river action, e.g., if they just said “I wouldn’t have arrived here” or “it’s a coin flip”, I didn’t classify it either way.

Of the 535 total responses, 272 recommended a call (51%), while 263 recommended a fold (49%).

The percentage of responses recommending a call was slightly higher when OP was facing a check-raise (58% call responses) than when they were facing a bet (50% call responses) or a raise from behind (49% call responses).

How often did posters reach a consensus?
I wondered how often the responses would reach a clear consensus on whether OP should call or fold. I defined “consensus” as >70% of posters agreeing on the outcome, with at least 3 total responses. If less than 70% of posters agreed on a call/fold, I classified the post as not reaching a consensus.

Of the 93 posts receiving at least 3 responses:
-39% of the time, the responses reached a “call” consensus (>70% “call” responses)
-31% of the time, the responses reached a “fold” consensus (>70% “fold” responses)
-30% of the time, the responses did not reach a consensus

I found it interesting that so many hands failed to reach a consensus. Many of the hands posted here are close spots, i.e., calling isn’t clearly much better or worse than folding. But calling and folding can’t both be +EV. When the responses are divided evenly between calling and folding, at least 30% of us are making a decision that loses us money in the long run. And since many of these river bets are 100+ BB, we don't have to make the wrong decision very often to have a significant impact on our win rate.

How did responses vary when OP won versus lost?
The original poster followed up with results for 43/100 hands. Of the 43 hands with results, OP called the river bet 33/43 times, winning 15/33 times and losing 18/33 times.

Among hands where OP had the winning hand, an average of 46% of responses said OP should call.

But among hands where OP had the losing hand, an average of 68% of responses said OP should call.

This was really surprising. There are a few possible explanations for this. The most likely is probably posting bias: OP makes an obvious call, loses, then goes and posts on LLSNL to make sure everyone else would make the same losing call.

Obviously we don’t have to always have to win at showdown for a call to be profitable, so we shouldn’t expect to always win when the consensus is to call. But we should expect to have a greater number of “call” responses when OP has the winner than when OP has the loser.

Results by user
The five users with the most logged responses were Javanewt (30 responses), sixsevenoff (24 responses), Spanishmoon (19 responses), gobbledygeek (16 responses), and Minatorr (15 responses).

Of these users, the most likely to recommend calling was Spanishmoon (15 “call” and 4 “fold” recommendations, i.e., 79% calls), and the most likely to recommend folding was gobbledygeek (7 “call” and 9 “fold” recommendations, i.e., 44% calls). Of course, not everyone replied to the same posts, so it doesn’t make sense to directly compare these values.

We could rank users by their accuracy, by comparing the percentage of hands where they recommended calling with the winning hand or folding the losing hand. I’m hesitant to share these results, because (I emphasize again) just because someone called with the worst hand and lost doesn’t mean it wouldn’t be a profitable play in the long run, and I don’t want to call anyone out. So I won’t share the rankings of the most and least-likely-to-be-correct users. However, I will say that among the top five users I shared above, gobbledygeek had the highest percentage of “correct” river calls/folds, at 67%.

Takeaways
I feel more certain than before that even good players don’t really know how to evaluate whether a river call will be profitable. I didn’t record in the spreadsheet whether people recommended a “clear call/fold” or a “sigh call/fold”, but even on the controversial hands, there were often people responding “obvious fold” and “obvious call” to the same hand. This is just one piece of evidence, but maybe this will help drive home that these spots are often really uncertain and most of us don’t know how to decide whether a call will be profitable.

Very interesting post, and good analysis of the surprising results.

I definitely agree that hands posted here are biased towards hands lost, and particularly towards ones where the results made the OP question their logic.

It should also be noted that 50% correct wouldn’t at all be the breakeven mark for calling river basically ever.

Thanks OP, interesting and different post.

I laughed twice, once when we discover that gg was the nittiest poster and once again, more ruefully, when he was most right.

I think it’s an indication that he’s less right. You’d need to measure EV. Someone who “correctly“ folds too often is actually leaking and making bad folds due to pot odds.

And it’s nothing more than a slight indication, since the sample size is so low.

Right. Obviously in terms of dollar value, folding the winning hand is a bigger mistake than calling with the losing hand, while this metric (percentage correct) weights them equally, as I mention several times in the original post.

There's nothing wrong with having a high folded-losing-hand percentage, except that it means we're probably (but not necessarily!) also folding the winning hand too often.

Another way of putting it: if the opponent bets pot ($100) and we fold the winner, we lose out on making $200, but if we call with the loser, it only costs us $100. Both folding the winner and calling with the loser are mistakes (in the sense of the fundamental theorem), but calling with the loser is a less expensive mistake, so we prefer to err in the direction of calling, aka "pot odds". But I think we should strive to eliminate both types of mistakes to the extent that it's possible.

So firstly thank you for taking time to do the analysis. Interesting stuff.

I think your conclusion would be true if the posts were a genuine random sample, but the posts here just aren't, and so we have to be very careful. That 68% who said call may be right, because there may well be a clear selection bias to posting hands where villain is at the top of his/her range and we lose.

Yeah I was going to say a lot of posts are just bad beat or cooler stories in disguise.

So you'd often be incorrect results wise making the right decision.

I think you’re making a really simple thing confusing by talking this way.

I would never consider calling with the worst hand a mistake in a vacuum. That’s extremely misleading to others even if it makes sense to you. At a minimum you need to consider your hand vs the villain’s entire range. If we call a pot sized bet on the river closing action we need to win 1/3 times ignoring chops. If we end up with 40% vs villain’s range I wouldn’t say we’re making a calling mistake 60% of the time. We’re correctly calling 100% of the time. The former way of thinking is what leads players to believe that there was some way they should have known or played a hand differently just because they lost. The better way of thinking acknowledges that losing is part of good poker.

Right. I understand that. I assume everyone does since that’s the standard language for talking about this stuff. This is just an alternative way of thinking about the decision-making process, which I find illuminating (and perhaps others might too. If you don’t, that’s ok.)

I figured someone would complain about the term “mistakes”, which is why I specified “in the sense of the fundamental theorem of poker”, i.e., it’s not the play we would make if we could see our opponent’s cards, and it was the less profitable decision *for this specific hand*, but perhaps not given our estimate of what the opponent probably has. I agree that calling with what ends up being the losing hand can be the best play, even though it might have been a “mistake” for this particular hand. I don’t really care if we use a different word for this.

As an aside: if we’re using the language of ranges, I’m also skeptical of this idea that we can reliably determine that we have (say) 40% equity against an opponent’s range. For about a third of the hands posted, we couldn’t come to a consensus on whether a call was profitable. Like I said in the OP, many of these decisions are going to be close, but calling and folding can’t both be profitable, and about half of us are making the wrong decision for each hand.

No, No, No. That is the definition of outcome bias. All of those decisions could have been correct; you're using the outcome to judge them. Sure over a sufficiently large unbiased sample set that may be true, but here we have both a small and a biased sample set. You simply cannot draw the conclusions you are drawing from it.

No, I’m even talking about hands with no result posted.

Take any hand where half of posters said call, and half said fold, and OP never followed up with a result.

Half of posters thought they had the required equity against the opponent’s range, so they called. The other half didn’t think so, so they folded. They can’t both be right. Either we have the required equity, or we don’t. Half of the posters made a call or fold that will be unprofitable in the long run, regardless of the results for this specific hand.

We don’t know whether we’re going to win this specific hand. I think we agree on that. I’m going a step further by observing that often we can’t even agree on if we’re *probably* ahead this specific hand.

Ok, one last time, because it's a lot of work you've done, and it's interesting, you just can't draw the conclusions you're drawing. After this I'm done and someone can pick up the mantle

Let's say every one of your sample was the same.

1/2, Folds to me on the button, I raise to 15 with KK, SB folds, BB jams for 80. What do I do?

And 90% of posters say call, but hey 70% of the time the results are revealed by poster was villain had AA *and* and held.

Would you say that in general we're not good at making the right decision?

FYI, I slightly edited the post you quoted to clarify, after you quoted it.

Thanks for your reply. But (and please don’t take this personally) I think your question suggests I haven’t made my point clear enough.

I’m not trying to make ANY claims about our predictive power based on results, because of the obvious concerns about posting bias, as I alluded to in the OP and as you’ve pointed out here.

I AM suggesting that we aren’t great at making these decisions, but NOT because of results — it’s because we so often come to different conclusions of whether we’re ahead of our opponent’s range or not.

Just to emphasize, from my last post:

Take any hand where half of posters said call, and half said fold, and *OP never followed up with a result*.

Half of posters thought they had the required equity against the opponent’s range, so they called. The other half didn’t think so, so they folded. They can’t both be right. Either we have the required equity, or we don’t. Half of the posters made a call or fold that will be unprofitable in the long run, regardless of the results for this specific hand.

Hand histories have limited information and people tend to fill in the gaps differently based on their personal experience. Two people might reply differently and each be correct for their respective player pools, although there should only be one correct answer generally. Common terms which describe villains also have different meanings to each person. If OP labels villain as “decent 1/2 reg” every person is going to have a different conception of villain. To me almost everyone is a joke at 1/2 and I think the only decent players I’ve ever met there were waiting for a higher game. To someone else the tight ABC reg that wins a little might be considered decent. Maybe that means I’m too critical of people but you get the point.

The easier explanation is that most people that play poker aren’t very good at it, including the people on this forum. Generally it’s less likely to reach a consensus the less skill a group has. Some posters pretty consistently give bad advice. Some pretty consistently give good advice. Most are a mixed bag.

There are at least 2 considerations. First, people don't write a HH unless the decision is unclear to them. So by definition, the decision is close much of the time. Second, the decision is close because the OP has little to no reads on the villain.

Only 43% of posts the OP had the decency to even finish the hands... that should be a penalty. Ridiculous.

For many of the hands I'm assuming OP didn't provide results because they folded and didn't see what the other person had. And if they said they folded but didn't say what the other person had, I also recorded that as an "unknown" result, and just counted "fold" as OP's opinion on what to do for the hand. But I agree there were quite a few without results.

it's this thought process that makes betting for thin value on the river so profitable.
folding isn't sexy, folding isn't going to give you an adrenaline rush SO
most players make the hero call.
most thinking players widen the hand range on V to convince themselves calling is plus EV.
lowering variance at the cost of 8¢ - 10¢ an hour is well worth it.

not all calls are created equal.

calling the guy who hasn't made a river bet in 10yrs just because we have perceived equity is just letting ego control your play.

great post btw

haha I instantly thought of GG as most likely making the best river decisions. I think he plays too tight but there's no harm in playing it safe.

I believe column I in your stats-by-hand sheet is inverted (it's saying the consensus is the opposite of what it was).

I think the main value in this sort of analysis isn't really identifying accuracy or posters inclined towards call or fold decisions -- I think the thing I'd be most curious about, is are there posters who only post after there's a clear consensus? There's always a risk of herding in forums like this, it's tough to be the odd person out. I think I'd be most interested in knowing the posters who are willing to either A) go against an existing consensus or B) usually chime in early before there's a clear consensus.

Great catch about the calls versus folds consensus being flipped — that’s embarrassing! I’m out of town right now so I can’t look but I assume this was just due to a typo in my code.

This error doesn’t change any of the results really, since I didn’t use the “clear call” vs. “clear fold” consensus for anything other than noting how many times we made each decision, and these numbers were similar anyway.

I agree that “accuracy” isn’t the most useful thing to take away from this (and I almost didn’t include it because I knew it would be controversial, as you can see in the comments above).

I can do some more digging to look at which users were most likely to disagree with others. I guess we also have post order, since I mostly recorded the responses in order from top to bottom. I’ll see if there’s anything else interesting to report.

I know I tend not to respond late, unless I perceive a problem with how the discussion has gone so far. Sometimes if the hand is fascinating, I'll grunch it if it has a lot of responses, but usually if it already has 4+ responses I just read them to make sure the conversation is on track and then don't respond.

No biggie on the error, just wanted to compare my own responses to the consensus on the hands I responded and noted the inversion as a result.

I'd say the consensus/herding thing would be something more applicable to the non-river decisions, if you ever decided to go back and "scrape" more data (which given what the scraping process entailed, I'd say you shouldn't! very painful! when I first opened this thread I'd thought you'd somehow managed to chuck all of the raw text into an algorithm and done some sort of call/fold sentiment analysis on it).

Wow, how long did it take you to collect / analyze the data? I've done some hamfisted data collecting / analyzing projects in my game and I find it quite taxing / difficult, and lol usually end up wondering if it shows anything worthwhile / lol sample size.

Sorta surprised that I show up in a lot of the posts since I self quarantine all my coronaresponses to just 1/3 NL hands.

Not surprised that I'm on the "fold" side of things. I think against a lot of opponents that should be our overriding initial instinct and we should have very good reason to take the other route. It may also be because I come from Limit, where bet/folding later streets (where we're getting insanely good odds of typically 10++:1 to make the call) is a bread & butter play. Pot odds / schmot odds, imo.

Haven't really dived into the responses / etc., but I think it might (???) re-enforce my believe that it's typically best to aim to get yourself into an easy peasy / wheelhouse spot as often as possible (often at the cost of what many will cry as "value"), cuz it's unlikely any of us are remotely as good as we think we are in non easy peasy spots (i.e. experts can do whatever they want).

Ghasn'tplayedasinglehandofpokerin4monthsG