5 ms·

It's an interesting study in terms of how they conducted it, but isn't it obvious? If political ads didn't swing votes, no one would run them.

tqi 7y ago
This study would also indicate that political polling has the ability to influence elections?
- Nasrudith 7y ago
  I thought everyone knew that already given that push polls are a thing.
- ekianjo 7y ago
  Of course, anything that brings focus potentially changes the final metric.
  - tqi 7y ago
    I would agree, but I think most political reporters and pollsters view themselves as impartial observers rather than active participants...
    - ekianjo 7y ago
      What they view themselves as does not change the fact that they skew things in one way or another. The only observer that is neutral is the one that is not seen and not heard of.
gwern 7y ago
It's not at all obvious that political ads (or ads in general) work. The correlation of political ads (and all other kinds of political stumping) with success is confounded by donations (because more popular candidates are able to get more donations), and looking at randomized political interventions, we see tiny effects - you might've seen some coverage of "The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments", Kalla & Broockman 2018 https://web.stanford.edu/~dbroock/published%20paper%20PDFs/kalla_broockman_minimal_persuasive_effects_of_campaign_contact_in_general_elections_evidence_from_49_field_experiments.pdf https://web.stanford.edu/~dbroock/published%20paper%20PDFs/k... which says bluntly: "a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections." Political scientists often find minimal effects of advertising spend in general (how much did Trump spend compared to Hillary? to use a particularly pointed example). And to the extent that some kinds of ads do work, they may be self-negating ('advertising wearout' and political examples in https://www.nytimes.com/2018/05/25/business/what-advertising-history-says-about-the-future-of-fake-news.html https://www.nytimes.com/2018/05/25/business/what-advertising... ). Ultimately, 'if people don't want to go to the ball game, how are you gonna stop'em?' More broadly, there are many experiments on online advertising showing that the correlational biases are huge and you get wildly misleading estimates if you use anything but rigorous randomized experiments, and the actual causal effects appear to usually be, again, close to if not indistinguishable from zero, which I link in https://www.gwern.net/Ads#discussion https://www.gwern.net/Ads#discussion It is extremely easy to fool yourself in this subject. For an even broader argument that we greatly overrate the manipulability of people, see Mercier 2017's "How Gullible are We? A Review of the Evidence from Psychology and Social Science" https://www.gwern.net/docs/psychology/2017-mercier.pdf https://www.gwern.net/docs/psychology/2017-mercier.pdf .
- shay_ker 7y ago
  I don't think anyone would suggest that there's always a linear correlation between ad spend and conversions. But are you saying that ads never influence decisions or actions? That seems.... impossible. Ads must work sometimes. The degree to which they may or may not, however, is extremely hard to figure out, as you note. The study that this HN thread is about doesn't shed much light either, other than saying that "people can sometimes have their decisions influenced." Same as always. People can certainly change over time. 30 years ago, gay marriage would've been inconceivable by the majority of the US population. How did that change? Many of those same people who were against gay marriage are still alive! They must've been influenced by something - advertisements or not.
  - gwern 7y ago
    > But are you saying that ads never influence decisions or actions? That seems.... impossible. Ads must work sometimes. The degree to which they may or may not, however, is extremely hard to figure out, as you note. The study that this HN thread is about doesn't shed much light either, other than saying that "people can sometimes have their decisions influenced." Same as always. Nobody is saying that ads have literally 0.00000% effect. (I would strongly disagree with such a claim: https://www.gwern.net/Everything https://www.gwern.net/Everything ) What I am disagreeing with is the claim that they are so powerful and so obviously pass a cost-benefit test that they are known to work and further, that the mere fact that people buy ads is sufficient to prove that they are highly effective and worth buying. ("Isn't it obvious? If political ads didn't swing votes, no one would run them.") All of these claims are quite likely to be wrong: the best evidence is that the average effect is minute (and one can't rule out even negative effects the opposite of desired), they probably do not pass a cost-benefit test because much commercial advertising does not, it is easy to be mistaken about their value, and given how 'obvious' it is that they work and which casual analysis would support such interpretations ('candidates who spend more on ads win more! isn't it obvious that ads work?!') we should not be surprised if people buy advertising even when the true effects are unknown, unknowable within reasonable possible experiments, and probably minute.
    - 7y ago

tqi 7y ago

This study would also indicate that political polling has the ability to influence elections?

Nasrudith 7y ago
I thought everyone knew that already given that push polls are a thing.
ekianjo 7y ago
Of course, anything that brings focus potentially changes the final metric.
- tqi 7y ago
  I would agree, but I think most political reporters and pollsters view themselves as impartial observers rather than active participants...
  - ekianjo 7y ago
    What they view themselves as does not change the fact that they skew things in one way or another. The only observer that is neutral is the one that is not seen and not heard of.

gwern 7y ago

It's not at all obvious that political ads (or ads in general) work. The correlation of political ads (and all other kinds of political stumping) with success is confounded by donations (because more popular candidates are able to get more donations), and looking at randomized political interventions, we see tiny effects - you might've seen some coverage of "The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments", Kalla & Broockman 2018 https://web.stanford.edu/~dbroock/published%20paper%20PDFs/kalla_broockman_minimal_persuasive_effects_of_campaign_contact_in_general_elections_evidence_from_49_field_experiments.pdf https://web.stanford.edu/~dbroock/published%20paper%20PDFs/k... which says bluntly: "a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections." Political scientists often find minimal effects of advertising spend in general (how much did Trump spend compared to Hillary? to use a particularly pointed example). And to the extent that some kinds of ads do work, they may be self-negating ('advertising wearout' and political examples in https://www.nytimes.com/2018/05/25/business/what-advertising-history-says-about-the-future-of-fake-news.html https://www.nytimes.com/2018/05/25/business/what-advertising... ). Ultimately, 'if people don't want to go to the ball game, how are you gonna stop'em?' More broadly, there are many experiments on online advertising showing that the correlational biases are huge and you get wildly misleading estimates if you use anything but rigorous randomized experiments, and the actual causal effects appear to usually be, again, close to if not indistinguishable from zero, which I link in https://www.gwern.net/Ads#discussion https://www.gwern.net/Ads#discussion It is extremely easy to fool yourself in this subject. For an even broader argument that we greatly overrate the manipulability of people, see Mercier 2017's "How Gullible are We? A Review of the Evidence from Psychology and Social Science" https://www.gwern.net/docs/psychology/2017-mercier.pdf https://www.gwern.net/docs/psychology/2017-mercier.pdf .

shay_ker 7y ago
I don't think anyone would suggest that there's always a linear correlation between ad spend and conversions. But are you saying that ads never influence decisions or actions? That seems.... impossible. Ads must work sometimes. The degree to which they may or may not, however, is extremely hard to figure out, as you note. The study that this HN thread is about doesn't shed much light either, other than saying that "people can sometimes have their decisions influenced." Same as always. People can certainly change over time. 30 years ago, gay marriage would've been inconceivable by the majority of the US population. How did that change? Many of those same people who were against gay marriage are still alive! They must've been influenced by something - advertisements or not.
- gwern 7y ago
  > But are you saying that ads never influence decisions or actions? That seems.... impossible. Ads must work sometimes. The degree to which they may or may not, however, is extremely hard to figure out, as you note. The study that this HN thread is about doesn't shed much light either, other than saying that "people can sometimes have their decisions influenced." Same as always. Nobody is saying that ads have literally 0.00000% effect. (I would strongly disagree with such a claim: https://www.gwern.net/Everything https://www.gwern.net/Everything ) What I am disagreeing with is the claim that they are so powerful and so obviously pass a cost-benefit test that they are known to work and further, that the mere fact that people buy ads is sufficient to prove that they are highly effective and worth buying. ("Isn't it obvious? If political ads didn't swing votes, no one would run them.") All of these claims are quite likely to be wrong: the best evidence is that the average effect is minute (and one can't rule out even negative effects the opposite of desired), they probably do not pass a cost-benefit test because much commercial advertising does not, it is easy to be mistaken about their value, and given how 'obvious' it is that they work and which casual analysis would support such interpretations ('candidates who spend more on ads win more! isn't it obvious that ads work?!') we should not be surprised if people buy advertising even when the true effects are unknown, unknowable within reasonable possible experiments, and probably minute.
  - 7y ago