there's no real reason why this would be more pronounced in 2020 than in 2016, if anything less so because a female candidate as the opposition would make the effect more pronounced.

the polls are off for sure. lets look at rcp averages vs tentative results (top figure an average of rcp and 270towin, bottom is actual results) for states that have >95% precincts reporting.

florida
biden +.9
trump +3.4
(underestimated trump by 4.5)

ohio
trump +1
trump +8.1
(underestimated trump by 7.1)

colorado
biden +10
biden +3.7
(underestimated trump by 3.4)

north carolina
even
trump +1.4
(underestimated trump by 1.4)

wisconsin
biden +8
biden +0.6
(underestimated trump by 7.4)

michigan
biden +5
biden +1.2
(underestimated trump by 3.8)

georgia
trump +0.5
trump +1.4
(underestimated trump by 0.9)

texas
trump +0.3
trump +5.9
(underestimated trump by 5.6)


the average they're off by is a whopping 4.25 percentage points, and for the popular vote, rcp called it +7.2 biden off by a comparable margin.


The magnitude of how much they're off has actually gone up since 2016 by a lot and not in the direction that nate silver was expecting.

Is much of the problem for pollsters to do with turnout rather than honesty about preference? It's the lack of turnout error that is one of the key reasons I here cited for the accuracy of exit polls in the UK.

Minnesota, a state that looked flippable for Rs after 2016 and that Trump spent a lot of time in, bucked the trend.
RCP: D+4.3
Actual: D+7.2

Arizona still has more votes to count, but they're also currently D+3.4 vs. RCP saying D+0.3.

I wonder if Mason can admit that "under reported Trump voters" happen to correlate with an increase in all voters.

Hi chezlaw:

The pollster are supposed to determine through their questions who is likely to vote. So, this could be another place that needs some fixing.

Best wishes,
Mason

I'm not sure what you even mean. But my purpose of starting this thread was to explore areas where the polls may have been introducing errors into their data.

It also appears that pollsters in general did a poor job. In addition, people like Nate Silver, and there are a bunch of these, now look low notch instead of top notch.

Mason

but isn't this the area where people are most likely to be swayed by the last few days of campaigning and ground game. Can pollsters do much to account for this?

How about polls including a turnout range disclaimer alongside the margin of error? Or show different predictions for different turnout ranges (which i appreciate would be different in different areas)

Would follow from this that the polls may not be done badly as much as currently attempting to do something that fundamentally cannot be done.

Your first post. Contextualize man. You posted something that alludes to republicans/conservatives being under reported or represented in polls. Seems like a rather silly claim now. Republicans still comprise a minority in the popular vote.

Also, lol, at trying to dunk on pollsters (right now) when this election could end up with 290 EVs for Biden and a bigger popular vote margin. Once the dust clears some of their polling might be vindicated to a degree that they were operating within proper CIs.

I agree that the polls were once again off. I'm certainly not disputing that.
I think the big problem that Trump poses tfor pollsters is that he is impossible to poll.
Simply because of his persistent rhetoric throughout his whole presidency of delegitimizing all media. His die hard supporters simply distrust everything but Fox News. So they don't respond to polls. At least I think that has something to do with it.

Paul:

I never said that. What I did say was that that was the claim made by certain pollsters who did better in their 2016 forecasts.

I suggest you go back and reread what I wrote.

Again, the purpose of this thread was to explore where possible errors could be creeping into some of the polls. It was not to predict the outcome of the election or where the Electoral Votes might end up.

MM

Yeah, the "shy Trump supporter" thesis is hopefully an idea that will die now. The firms that went out to look for it of course found it. But in doing so, they significantly understated Biden support by 6 in Michigan, probably 5 in Pennsylvania etc. Also, the biggest polling misses were republicans in quite a few house districts, undercutting the notion that embarrassment over supporting Trump is a driving issue in polling.

Overall, polling was okay I guess, definitely not great. 2016 was right at the average error. 2018 was better than average and 2020 was a little more than the standard historical error. Aggregators like 538 did a really good job.

You are not stating this correctly. The average for Trump in (for example) Wisconsin was 43.5. He got 48.9%, so they underestimated Trump by 5.4%. If they were only off 4.25 for the margin, that is actually pretty decent. Standard error for 1 candidate in one state poll is about 3.5 from sampling noise alone so for the margin between 2 candidates it would be roughly double that. That's because the error for Trump and Biden in a given state is highly correlated.

Although I wouldn't do everything the same way as Nate Silver, assuming the remaining races go the way the betting markets (Betfair) think:

Biden: AZ, GA, MI, NV, PA, WI
Trump: AK, NC

then 538 (or the polls it gives most weight to) will have correctly called 48 out of 50 states (the exceptions being FL and NC) for a 96% record. Yes the polls are off in some places but the criteria on which we have been invited to admire Trafalgar's efforts in 2016 is that they called states correctly, not that they got closest to the actual result.

There are a lot of votes still to be counted in CA and NY but it looks like part of the polling miss is again down to people polling Libertarian and voting Republican.

I have to admit Mason's theory is more accurate than Nate silver 538. AZ and NV seem to be off for Trump support though.

I don't see it. Here's Trafalgar's "model". I don't think they polled all the states and they don't actually show you their inputs like 538.



Looks like they'll miss 5 states 2 districts as well as the overall winner. If Nate goes from top notch to low notch based on 2020, Trafalgar is surely no notch at all with the poor performance and constantly harping on on nonexistent voter fraud.

Yeah, I think we can now officially put Trafalgar in the debunked junk, biased pollster category. 303 electoral votes for Trump, give me a f!kin break.
Meanwhile, everyone has written off Nate Silver; justifiably so, while he's on ABC every day giving his "insights" and cashing in the monies .

Edit: no-one expected such a huge voter turnout on both sides. But shouldn't that be part of the pollsters job also?

i'll defend Nate here. He had Biden as a huge favorite, there was large polling error, mostly to his detriment and he still won. That's what big favorites do. Given he doesn't actually poll, I'm not sure what else he could have done. His model said Trump needed historic eror in his favor(likely unprecedented in the era of mass polling) to win and that was correct.

Looks like Trafalgar’s predicted 44% of African-Americans voting Trump was utter nonsense.

I've seen no reason to think 538's model failed, but if the inputs into that model lose credibility, then the model's output also lose credibility. That is, Nate Silver's model does a fine job AFAICT of putting a probability on a Biden victory given the polls, but if polling is becoming less accurate then we should increase the background chance that the real-world probability of a Biden victory diverges from the probability given by 538.

/\
Yes, that's the real dinger. But even worse, Mason believed it!
These people are not living in reality. Thank you Trafalgar for making it so glaringly obvious. And thanks Masson for this website but it will be a cold day in hell when I buy another book from you guys.

good topic. thx for starting it Mason,

I am a big polling/stats junkie and always will be.

but it seems fairly apparent the polls did a pretty poor overall job in 2016 and now in 2020

a few comments/(questions),

1) I do not believe in "the shy trump voter" in 2020. maybe in 2016

2) trafalgar/rasmussen I think do not run legit polls. I think they are somewhat correct by accident. like asking a kindergarten class what's going to happen. some kid picks major trump positive surprise both years. what are the odds of the kid making those predictions? 5%-10%

3) the 2020 polls fought the last war and probably fixed it. the statistical categories matched the voting populace.

4) BUT do the people responding to the polls represent the average person in their demographic group?.... SO FEW PEOPLE REPLY TO POLLS. ARE THE PEOPLE MORE CIVIL-MINDED AND BUREAUCRATIC THAN THE AVERAGE PERSON IN THEIR COHORT?....... is it the latino doctor/lawyer/teacher who responds to surveys?.. strikes me there's a limit to slice/dice where it eventually becomes meaningless.

5) turnout is so big..... people thought D's would get many votes from people who didn't vote in 2016. not sure many thought the same would happen with uneducated whites. after all 2016 election was like their super-bowl. but maybe that's only after the fact. maybe many didn't think trump had a chance to win or hadn't followed it that closely. post 2016 election there was a deluge of trump coverage.

how many states is Nate silver in danger of missing?

I am doing the math.

he got Florida, Maine 2 wrong.... likely NC will be wrong.

the remaining 4 states (N,A,P,G) he all predicted D. and biden is favourite in all 4. prohibitive favourite in 3 of them.

most 270towin pundit maps were more bullish on trump than that.... EDIT: I stand corrected. he was at 348, in line with many of those other pundits.

Wasn't Silver predicting a blow out for Biden and Democrats in general? That certainly didn't happen and iIf this wasn't the case, he wouldn't of been so critical of Trafalgar.

One thing positive towards Silver, he was willing to put his reputation on line, and deserves credit for that. He didn't phrase his wording in a vague manner.

Best wishes,
Mason

Silver has a good chance of only getting Florida and ME's 1 electoral vote wrong

Popular vote margin is going to be pretty substantial. But NOT 7-8%.. And I question whether polling error wouldn't have skewness (ie normal distribution no good)

Pollsters will fight the last war in 2024... Not sure they will ever get turnout

No not really, he was predicting what happened but he also though FL and NC and ME-2 would go Dem. So his score is 48/50 for states (or 53/56 if you also count districts). Really it's not him predicting it, it's the polls he uses. I had the same 48 states also using the same polls - though with different probabilities.

https://projects.fivethirtyeight.com...tion-forecast/

Larry Sabatos, of the University of Virginia, one of the pundits on 270 seems to be closer

https://www.270towin.com/maps/sabato...2020-president

AFAICS he has only got North Carolina wrong. He had GA Dem and FL Rep - I'm not sure many others will have had both of those right.

Trafalgar were wrong for NV, AZ, NE-2, MI, PA, and GA. So 45/50 or 50/56.

(FWIW though the markets still have GA and AZ as Dem but 4-1 and 3-1 races so we can't really "call" them yet)