Yeah thats a great question. Short answer, no, I don’t. Long answer, is that its complicated and too hard to know. Safe answer is, just assume the above as the best guess for what biases will look like on election day.
The problem with being able to compensate for what the above data show is that you have to have extremely good demographic models, specifically for demographics you didn’t capture in your original sample. I think part of the reason why stochastic modeling misses these things is that its not really a forwards-in-time facing type of analysis. You can’t compensate for a future state if that state is unknown, you can only go backwards to account for your prior (but even that is still facing backwards).
However, I don’t agree that stochastic models are where we should stop with trying to understand these kinds of things. There are plenty of phenomena where we engage with a range of classes of models to try to get an idea of where things should be. Some examples of these are things like process based models, which are a kind of simulation to estimate based on some parameterization, how things came to be. You’ll often do a kind of bayesian filtering on these kinds of models to get down to results that match your data, then use the priors to hopefully understand something about the system. So in the context of electoral politics, it would be trying to understand why someone gets off the couch to vote, or join a movement, or whatever.
So I think that the data in these stochastic samples are good, but the problem is that voting really isn’t a random effect. I think the results are likely good, but they are only going to be as good as the last time the voter demographics were sampled (if they were even updated for that), and then as relevant as those demographics are to the actually electorate who shows up when November 5th rolls around.
A great example of this phenomena in play was the Bernie/ Hillary primary race in 2016. Hillary had the support of basically every mainstream media outlet on the left, all of the DNC, all of Washington. Yet, she was on-track to lose until the DNC stepped in and put their thumbs on the scales. Why? How was that possible? How was Bernie out-performing all of his polls?
Bernie was outperforming his polls because he wasn’t drawing on the same distribution of voters for whom polls are focused. He was turning disengaged, non-voters, into engaged participants in a process. And you can’t measure that with your last demographic sample, because according to your best most recent measurement: those people don’t vote.
Trump does something very similar. He is gathering disenfranchised, disengaged, non-voters and turning them into voters. And you’ll never capture that with a polling model based on last elections voter demographics, when the strategy is to fundamentally shift the demographics.
If pollsters were to massively weight their numbers as I’m describing, Democrats would be getting thunked right now. Its why having a >5% polling advantage going into election day is so important for Democrats.
I still think Trump is such a well-known commodity now and all of this is nothing new. We’ve been talking about his “hidden voters” so much for so long that I actually think polls may be overcompensating a bit for that. Or at least they could be pretty well calibrated for it at this point. Guess we’ll see in less than a month.
I still think Trump is such a well-known commodity now and all of this is nothing new. We’ve been talking about his “hidden voters” so much for so long that I actually think polls may be overcompensating a bit for that.
I would be ecstatic for that to be the case. Unfortunately, both the 2016, and 2020 polling disagree. But right now, the data we have at our disposal do not support that case.
I’m curious what you think pollsters are doing when you say:
Or at least they could be pretty well calibrated for it at this point.
Like, in stochastic modeling, you have to do things like having a truly random sample to develop your statistics on. Pollsters hands are kind-of tied in this regards and the data is mostly available for download. I’m curious if you think there is some kind of demographic weighting that you think pollsters are doing on the back end?
Yes, I definitely think pollsters are compensating for Trump’s hidden voters by now. Like you say, they’ve had both 2016 and 2020 to get it worked into the polling. It’s rare to get three tries to work it out. I’d be very surprised if they undercount it again.
Don’t you think the pollsters have compensated for that by now? This has been known for years and years.
Yeah thats a great question. Short answer, no, I don’t. Long answer, is that its complicated and too hard to know. Safe answer is, just assume the above as the best guess for what biases will look like on election day.
The problem with being able to compensate for what the above data show is that you have to have extremely good demographic models, specifically for demographics you didn’t capture in your original sample. I think part of the reason why stochastic modeling misses these things is that its not really a forwards-in-time facing type of analysis. You can’t compensate for a future state if that state is unknown, you can only go backwards to account for your prior (but even that is still facing backwards).
However, I don’t agree that stochastic models are where we should stop with trying to understand these kinds of things. There are plenty of phenomena where we engage with a range of classes of models to try to get an idea of where things should be. Some examples of these are things like process based models, which are a kind of simulation to estimate based on some parameterization, how things came to be. You’ll often do a kind of bayesian filtering on these kinds of models to get down to results that match your data, then use the priors to hopefully understand something about the system. So in the context of electoral politics, it would be trying to understand why someone gets off the couch to vote, or join a movement, or whatever.
So I think that the data in these stochastic samples are good, but the problem is that voting really isn’t a random effect. I think the results are likely good, but they are only going to be as good as the last time the voter demographics were sampled (if they were even updated for that), and then as relevant as those demographics are to the actually electorate who shows up when November 5th rolls around.
A great example of this phenomena in play was the Bernie/ Hillary primary race in 2016. Hillary had the support of basically every mainstream media outlet on the left, all of the DNC, all of Washington. Yet, she was on-track to lose until the DNC stepped in and put their thumbs on the scales. Why? How was that possible? How was Bernie out-performing all of his polls?
Bernie was outperforming his polls because he wasn’t drawing on the same distribution of voters for whom polls are focused. He was turning disengaged, non-voters, into engaged participants in a process. And you can’t measure that with your last demographic sample, because according to your best most recent measurement: those people don’t vote.
Trump does something very similar. He is gathering disenfranchised, disengaged, non-voters and turning them into voters. And you’ll never capture that with a polling model based on last elections voter demographics, when the strategy is to fundamentally shift the demographics.
If pollsters were to massively weight their numbers as I’m describing, Democrats would be getting thunked right now. Its why having a >5% polling advantage going into election day is so important for Democrats.
Thank you for a good write-up. Much appreciated.
I still think Trump is such a well-known commodity now and all of this is nothing new. We’ve been talking about his “hidden voters” so much for so long that I actually think polls may be overcompensating a bit for that. Or at least they could be pretty well calibrated for it at this point. Guess we’ll see in less than a month.
I would be ecstatic for that to be the case. Unfortunately, both the 2016, and 2020 polling disagree. But right now, the data we have at our disposal do not support that case.
I’m curious what you think pollsters are doing when you say:
Like, in stochastic modeling, you have to do things like having a truly random sample to develop your statistics on. Pollsters hands are kind-of tied in this regards and the data is mostly available for download. I’m curious if you think there is some kind of demographic weighting that you think pollsters are doing on the back end?
Yes, I definitely think pollsters are compensating for Trump’s hidden voters by now. Like you say, they’ve had both 2016 and 2020 to get it worked into the polling. It’s rare to get three tries to work it out. I’d be very surprised if they undercount it again.
I hope you’ve got the right of it.