HI DJ:

Concerning your last issue the answer is no. And without going into an explanation, this is what statistical theory tells us, and what seem like small sample sizes relative to the size of the population are correct providing the sampling itself is done properly.

Best wishes,
Mason

If they're biased garbage, can you tell us where the biases appear? I bet you can't.

It's my opinion that response bias has creep into many of these polls, and remember, I do have experience doing work in survey design. But exactly how these biases creep in and to what degree, would only be speculation on my part.

Mason

They produce consistently biased results, and unlike most pollsters, they hide their crosstabs so that people cannot see where their biases have crept in. Any outfit that is consistently biased and far more secretive than everyone else should not be trusted.

Let's update with more results coming in:

Nevada can go in the "bad" category, as it will finish about Biden +2

We know Georgia, Pennsylvania, and Arizona are all bad and now we can attach numbers to them: Biden +a hair in GA, Biden +1 in PA, Biden +1 in AZ

You can't either until you get their datasets and look over the questions they ask. So this is a really weird jibe.

They also just tell you how they are biased

So they go in thinking that Trump's support is going to be understated because people don't like to freely admit they are voting for Trump. Whatever secretive method they use to get Trump supporters to admit it just ended up under counting Biden's support by 5-7 in AZ, PA, MI and WI. Completely predictable and not super surprising.

And you're a scientist? but certainly not a businessman. If they're doing things better, or at least think they are, why would they release information to let their competitors catch up?

MM

This is correct. In fact, even if I had their data as well as their questions, I wouldn't be able to tell. That's not how population sampling works.

To accurately answer things like response variance, response bias, and conditioning, reinterviews will need to be done and this might include asking similar but somewhat different questions.

Mason

This is incorrect. If you saw questions like "Are you voting for our president or the socialists?" you would obviously concede that the poll is biased. There's more subtle and unintentional types of questions where people with proper training could determine biases. I would have to go back and look in my old statistic books to remember their specific names as I haven't dealt with polls outside of college.

Then there's the case if you found a poll that seemed 50% republican inside a county that was 70% registered democrats or vice versa. That would obviously look like a sample that wasn't random at all. You could also find errors by whoever used the data in the first place.

No. This would show that the questions have been worded in such a way that the respondents are being conditioned to produce certain answers. That's not the same as a bias.

I don't think this is true either. Based on my experience to determine whether a bias actually exists, a reinterview of many of the respondents would be required.

I doubt that this is true either. Very few schools, unless there have been a lot of changes over the years, have courses in finite sampling theory. When I worked for the Census Bureau, I had to go at night to American University to take a graduate level course in finite sampling theory. My school, Va Tech, at least at that time, did not offer such a course in their statistics department.

Not necessarily. This could mean that while the sample was random, the population from which the sample came from was not the total population from which the sample should have come from.

This might be correct depending on exactly what you mean which isn't completely clear to me.

Mason

We have more data now. In 2020 it seems the Trafalgar people got it wrong when most everyone got it right, especially with respect to Biden's vote % in key swing states. Consensus was ~50% and right on the money. Trafalgar was in the 44-46% range which is about a 20:1 shot to occur without systemic error/bias. This was the result predicted by many here and exactly what you would expect it Trafalgar got it "right" in 2016 by luck rather than something that carries over election to election like skill. Maybe you can ask that professional statistician friend of yours how to interpret this new, but wholly predictable data.

Remember when America rejected trump?

This is just an absolutely incredible misconstruing of my post. Believe me, I understand pollster business models. There are two.

1. Pollsters sell themselves as a means for getting an accurate assessment of public opinion, so that people in politics and marketing have actionable information that they can use for targeting and messaging. In order to convince people that they are accurate, they show a long track record of successes and disclose sampling and methodological information that shows that they are doing everything on the up and up and that their samples are representative. They do not have to disclose proprietary sample weighting methodologies, which they understandably would keep close to the vest.

2. Pollsters can also sell themselves as a source of telling the customer what they want to hear, regardless of the truth of the matter. These people will trumpet their successes, hide their misses, and focus only on the top line without demonstrating that their samples are indeed representative of public opinion instead of just what the customer wants to hear.

Trafalgar is clearly in category 2. There's plenty of room for opprobrium for pollsters in category 1, like Quinnipiac, who had some major whiffs, but that doesn't mean that Trafalgar is anything other than a category 2 pollster.

In addition to keeping their cross tabs secret, Trafalgar should consider keeping the rest of the polling data a trade secret to keep the competition from learning their top-flight methodologies.

Again, are their methods superior or not? This is almost exactly what the dishonest companies like Tralagar do. Constantly hype up how they have it correct before an election, then disappear if they were wrong. They'll make no changes and be back next cycle to do the same thing despite the huge miss.

How would I know. I would have to have access to not only their methodology but would need similar access to the methodology of other pollsters, and this assumes I would even understand it.

Why do you ask such silly questions?

MM

Heh

You said pretty clearly before the election

What changed, other than Trafalgar missing badly? Seems like a classic "heads I win, tails we really need to delve into the philosophical and scientific justification of coin flipping"

I really don't understand. If Trafalgar's results are worse than the top polls over 50+ pieces of data, why are you spending so much time entertaining the very slim possibility that they are a great polling firm? It's very contrarian/Davidian/long tail discussion interest of you to continue to engage in that manner.

Don't the calculations of sample size suppose spatially homogeneity to a certain degree? Miami is blue but with very localized sharp red peaks. That variation could be missed by insufficiently fine sampling.

There's a big difference between sampling from a uniform distribution and sampling from an al lost uniform distribution with highly localized concentrations of mass.

He's not willing to admit that Trafalgar is pretty bad despite all the obvious evidence. Before the election it was "the election will determine whether they are good" and now it's "silly question, how could I possibly determine whether or not they are good". FWIW I don't think he's trolling given his history and he actually can't reason his way out of the obvious self-made trap he fell into.

Fwiw I still can't tell if Mason is just trolling you guys lol

In a real shocker, it looks like you would have won.

Cherry pick data much? When you Cherry pick you can make Jingle Bells sound like Che Gelida Manina......

They were not within the margin of error in PA. They had Biden at 46 and he is at 49.9, they used a 1000 person sample which generates a margin of error of ~3. And for Arizona they had Biden at 46 and he really got 49.4, so off by 3.4.