Hi Everyone:

Years ago, I worked in Statistical Methods Division of The United States Census Bureau doing survey design. I still have my finite sampling theory books. And for those who don't know, designing statistical surveys and polling are the same thing.

Today, there are a lot of questions about whether the polls are any good, and if some are good, which ones are those? Now, that's a question I can't answer since I don't know their methodology, but I can answer some general questions about polling that some of you might have. And I'll check this thread everyday to see if there is anything I can answer.

And for starters, here's the first question I'll answer:

Question: Some polls seem to oversample Democrats, does this mean the result will be inaccurate and give the Democrat a higher number than he deserves.

Answer: As long as the weighting of the Democrats is adjusted for the over sampling, it should have no effect on the estimate. What it will do is create a slightly higher variance to be associated with the estimate. So, when you hear or read that the Democrats (or Republicans) are over sampled, you should ignore it.

Best wishes,
Mason

What happened in 2016 (when groupthink or some other phenomenon caused most pollsters and pundits to call the race for HRC)? Did Trump voters lie to pollsters? What of the pundits and pollsters who insist that, this time, the polls are accurate and favor Biden? Could they get fooled again?

Hi eobmtns:

I can't tell you what happened. But questions need to be carefully worded so that the response is accurate.

One way of determining this is to reinterview a subsample of those included in the original sample and see how their answers now compare to what they originally said. If there is a big difference, known as response variance, then it implies the wording needs to be changed or perhaps additional questions should be added that will encourage the respondent to give concise, consistent answers. If this was the problem in 2016 then it might be fixed today.

There's also another issue called response bias. This is when for some reason repondents are encouraged, not necessarily by the questions, to give a wrong answer. This is a much tougher to fix.

My guess is that this second issue was more of the problem than the first issue.

Another possibility would have to do with non-response. Those who don't respond are still counted in the survey. It's assumed that their response will be the same as some subset of the people in the poll who have other known similar characteristics. For instance, if you live in a wealthy suburb your respo0nse is more likely to be the same as someone else who lives in a wealthy suburb than someone who lives in a poor neighborhood. But it doesn't have to be, and if non-response rates are high, the accuracy of the poll can be affected.

And as for whether the polls are more accurate today, I wouldn't know.

Best wishes,
Mason

Hey Mason,

How much has the quality of polling changed over time? Is it like physics (where more data gives more accurate results,) or, is it like economics (where human behaviour can blow the models)?

Not to tread on Mason's turf but I'm not sure either physics or economics is a great analogy. The theory of polling hasn't really changed since..Gauss??? or whoever was the first to understand how much information you can get from random small samples of a large population, as long as they are actually random. The only challenge of polling is getting a random sample. That's a moving target that gets complicated by societal changes ie cell phones, caller ID etc. But I think the simple nature of what polling is and how obvious it is that 95% of random samples fall within X range makes polling unlike broad disciplines like physics or economics.

Reality is if Trump wins again Polling is pretty much useless .

This question is kind of a head scratcher to me. The better models were saying Clinton was around 80% to win. That means if Trump doesn't win around 20% of the time the model is wrong. Given the nature of Trump's win (lost comfortably in the popular vote, won by very thin margins in the swing states, without breaking 50% in any) isn't that the exact kind of win you expect an underdog to pull out 20% of the time? If Trump won like Obama did in 2012; not only winning but with over 50% of the popular vote and comfortable margins in all the swing states I would ask the same questions you did. But this is more like a football game where the favorite loses but they gave up a TD on a kickoff, lost 2 fumbles but still had a chance to pull it out in the last 2 minutes.

Who actually gets polled? I’m 49, vote often, and have never been polled. Is it standing on a corner asking people? Who actually answers the phone to an unknown phone # these days?

I'm guilty of sowing chaos with pollsters.

I'm one of those boomers with a landline still. In the past 2 weeks it feels like I get no fewer than 3 political calls a night. Some are just democrats trying to get us to vote Biden, but most are pollsters.

My wife was in the teachers union, so maybe that gets us on more lists than your average citizen.

Wrong.

Party ID is not an appropriate attribute to weight on, the way one weights on education, age, gender, race, income, etc. For one, party ID is fluid, if things are going well for Republicans you'll likely see more people self-identify as republicans. For another, it's an element of the thing you're trying to measure since party ID so it's so closely aligned to candidate choice.

You sound like the UNSKEWED guy from 2012.

I'm 62, and voted in every Presidential election since 1976 (except 2008).

I have NEVER been polled.

For sure. And the fact that it came down to a fraction of a percent of a Trump win in 3 states shows just how freakish his win was.

The best model imo was 538, which had Trump as a 30% dog. So it was in no way some kind of crazy event. Essentially, there were 3 possible scenarios: a small Trump win, a small HRC win, or an HRC blowout. All equally likely.

It's just that the punditry in 2016 was in such denial that our country would vote for Donald Trump that most estimation had her as a 90%+ favorite which was just way off obviously.

Mason:

This is an interesting thread. I have two questions.

The first question involves potential bias of the sponsoring organization/entity that pays a polling firm to conduct a poll. The New York Times bills itself as "the paper of record" but is generally regarded (outside of New York) as a "liberal" newspaper. Some people, especially Trump supporters, believe The Washington Post is biased to the left. Flip the coin to the other side and some people believe the Fox News Channel is biased to the right. (A "librul" like me might say the Fox News Channel is "fair and balanced" - to the right! )

I recall a statistics expert who worked for an insurance company that I was talking to on an airplane flight. We were discussing politics, so I asked him about the accuracy of polling. He responded (paraphrasing slightly): "Tell me the result you want and I'll give you the poll questions to produce that result." So, when these organizations commission a poll, what assurance do we have that the polling firm producing the results is more interested in satisfying their [paying] client than producing an accurate result?

The second question is from my girlfriend who is a very conservative Trump supporter. (That's right, I've hooked up with a very politically conservative lady friend.) With a tip of the hat to Paula Abdul

https://www.youtube.com/watch?v=xweiQukBM_k

I suppose it's a case of opposites attracting.

I don't know if she's been watching too much of the Fox News Channel, but she tells me she believes Trump supporters are intentionally lying to pollsters because they don't want to admit - to a pollster - that they intend to vote for Trump. Assuming there is some truth to this belief, how do pollsters detect (and adjust) for such a bias? More generally, how inaccurate can a poll be if a substantial percentage of the respondents choose to intentionally lie to the pollster?

Side note: This will never happen, but I think politics and elections would be much more interesting if all public polling was either banned or outlawed. People would be more inclined to actually go to the polls and vote if they didn't "know" (in advance) who was going to win.

Candidates could produce their own "private" polls - for internal use only - but public polling published in newspapers and on the internet should be banned. When it comes to politics, the only poll that really counts is the poll conducted on election day when voters cast their ballots.

I was wondering the same thing. My gut instinct is that the answer is more complicated than a simple yes or no.

Mason, I may know someone who overlapped with you at the Census Bureau. The person I know was working in the same area.

It is entirely possible that a publicized poll near election time will have less accuracy than an unpublicized poll, especially considering that people may or may not vote. The direction of the inaccuracy is unclear though. Common sense might say that being behind might make more people vote. But there is also the opposite theory that people are more likely to vote when the poll says their preference is ahead because it feels good to vote for a winner.

Hi Doctor:

The statistical theory for polling hasn't changed. But I do believe reliable data is more difficult to obtain. One area is non-response. It's probably harder to track down those selected for the survey and my guess is that the refusal rate has also gone up as people seem to be more busy today with their own affairs than they use to be.

Best wishes,
Mason

Hi ecriture:

I think you have this right. But just to add on, cell phones, caller ID probably increase the non-response rate and the higher that is, even though the poller will take this into account, the less accurate the poll should be.

Best wishes,
Mason

This may be true. If Trump wins it will again show how off these polls were. On the other hand, groups like Trafalgar and Rassmusen who have consistently showed Trump doing better may breathe new life into polling as other groups begin to copy their methods.

Best wishes,
Mason

Hi ecriture:

In the last chapter of my book, The History of the World from a Gambler's Perspective, written with Antonio Carrasco, I explain why Trump won. You may want to look at that:

https://www.amazon.com/History-World...s=books&sr=1-1

Best wishes,
Mason

Participants are selected randomly, usually from a list. However, the main reason you haven't been included in a poll is that the sample size to achieve a relatively accurate poll, which is determined by statistical theory, is much smaller than most people think.

Best wishes,
Mason

Hi Inso:

Your list comment is probably correct.

Best wishes,
Mason

I don't agree. If the pollster sees that there is something wrong with his sample, he will most likely adjust for it. Otherwise, he'll be putting out a result that he knows is not accurate.

By the way, while it was many years ago, I have some experience doing this.

Best wishes,
Mason

Hi DD:

My understanding is that 538 looks at polls done by others and then comes to a conclusion. They don't do their own polling.

Best wishes,
Mason

Hi DJ:

Regarding your first question, your statistics expert is right. For instance, if you give respondents a choice between "corrupt" Biden and "successful businessman" Trump, your results would probably be skewed towards Trump. In the world of finite sampling theory, this is known as "conditioning." And questions need to be carefully worded and often tested ahead of time to make sure this doesn't happen.

Regarding your second question, the Trafalgar Group, which was the most accurate polling organization in 2016 where they predicted a Trump win in the exact manner that he won, claims that this is exactly what happened. They also are predicting Trump to win again.

The way to possibly determine this would be to ask additional question to the respondent that would possibly indicate that his answers are not accurate. For instance, if the respondent was to say that capitalism is great while socialism is terrible, this might be, along with some other responses, that his presidential perference answer might not be accurate and then the pollster would adjust his results accordingly.

Best wishes,
Mason

Hi Rococo:

Send me his name by PM and I'll see if I knew him. Also, we have a poster on this site who I was actually good friends with when I worked at Census. After leaving Census I had lost contact with him until the poker boom and then I heard from him through 2+2. I wonder if we're talking about the same person.

Best wishes,
Mason