The many problems with public opinion

[Ptarmigan]

The many problems with public opinion
https://www.econlib.org/the-many-problems-with-public-opinion/

A commenter named Ted once asked me a series of interesting questions, and I’ve since done a few “Ted talks” over at MoneyIllusion. Today I’ll do one here, in response to this request:

More posts on how public opinion isn’t real (for me, this was a big takeaway from your writing)

I’ve already done a few posts on this, but I don’t believe I’ve put all my reservations into one place.  Here are a few:

1.  Political correctness

2.  Framing effects

3.  Innumeracy

4.  Budget constraints

5.  Fragility

Interesting read. Polls should be taken with a grain of salt.

[Ptarmigan]

The polling industry doesn’t measure public opinion – it produces it
https://www.theguardian.com/commentisfree/2019/sep/20/polls-public-opinion-polling-industry

The polls keep coming, one after another. But the polls are all over the place. For example, they can’t agree on where the competing parties stand. One gives the Tories a 10-point lead, another gives Labour a 2% lead. Polling has never been an exact science, but political volatility, the growth of new polling firms and the extraordinary ubiquity of conflicting polls, has put it under new strains and new scrutiny.

Polling was once a specialised sector of market research. Now it is a niche area of the much bigger data industry, using the same Bayesian techniques of probabilistic analysis that stock markets employ in financial forecasting. Its customers include huge financial firms that can extract commercial advantage from the slightest margin of predictive accuracy.

When it comes to prediction, the neoliberal economist Frank Knight made an important distinction between “risk” and “uncertainty”. The data revolution has helped capitalism manage risk, but it manages uncertainty far less effectively. One byproduct of this is that, although we have more data than ever before, political outcomes are no easier to predict: hence, the current crisis of polling.

Polls rely on history.

By relying on past outcomes to guide their assumptions, pollsters made no room for political upsets. Since poll numbers are often used as a kind of democratic currency – a measure of “electability” – the effect of these methodological assumptions was to ratify the status quo. They reinforced the message: “There is no alternative.” Now that there are alternatives, polling firms are scrabbling to update their models.

Yet, what do polls actually measure? In 1947, as modern polling was becoming a major industry, researchers canvassed American public opinion on something called the “Metallic Metals Act”. No such act existed, metallic metals being akin to ironic irony, or tautologous tautology. But 70% of respondents took a firm view for or against. They weren’t stupid: they were just acting as most of us do in an interview situation, under pressure to have definite views about something of which we may be uncertain, conflicted or even ignorant.

These researchers weren’t measuring opinion. They were producing it. This is what the polling industry does: that’s why it is an industry.

The artifice of the phone interview, or online survey, resembles no real-life circumstance in which opinions are formed. It is an assembly line designed to produce quantifiable opinions: that is, to express sentiments, preferences. This may make sense in the context of market research to measure consumer preferences. But formulating a preference, or even purchasing an item, is quite unlike casting a vote. The latter is closer to a major life decision – often rooted in collective experiences like class and race – than to a brand choice.

[freedumb2003b]

Pollsters use the same methodology as AGW adherents:

Draw your lines THEN plot your points.