6 ms·

Hi, I'm the author of the article, and I don't assume all online advertising is Google search. I also cite research on display advertising (meta-study of 432 di

Hi, I'm the author of the article, and I don't assume all online advertising is Google search. I also cite research on display advertising (meta-study of 432 display experiments on Google): https://www.ssrn.com/abstract=2701578 https://www.ssrn.com/abstract=2701578 And on Facebook advertising: https://www.kellogg.northwestern.edu/faculty/gordon_b/files/fb_comparison.pdf https://www.kellogg.northwestern.edu/faculty/gordon_b/files/... The problem with all online advertising is that there are huge selection effects, which are hard to correct for using conventional statistical methods. So you need to do experiments. And when economists do experiment, they find that advertising effects are so small that they are hard to measure.

glofish 7y ago
Do you think that the situation is due to an arms race kind of scenario? -If everyone but one stopped advertising then the one still doing it would reap massive benefits? So advertising is a form of obstructing the competitor.
marrone12 7y ago
There is a selection bias in these studies as well -- that you are focusing on large, already established brands that have good brand recall and people who had intent to purchase there anyways. For smaller companies that are just starting out, it's relatively impossible to have prior intent when people don't know who you are or what you offer. There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.
- sharkmerry 7y ago
  > There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned. But is that number statistically significant? some do, how many others tried the same route and failed, how many grew without it, etc
graycat 7y ago
> using conventional statistical methods. Experimental design, analysis of variance are such. E.g., for the farmers and from the corn fields and hog pens of Iowa: George W. Snedecor and William G. Cochran, Statistical Methods, The Iowa State University Press, Ames, Iowa. These methods have been widely used in the social sciences -- e.g., my wife, Ph.D. in mathematical sociology from Hopkins, got quite good with that material. The field is quite serious and mature and goes well beyond just A/B testing. For the practical challenges of the article, academic fields closer than economics include statistics and optimization. For the Lagrange multipliers in the article, those likely would be from the Kuhn-Tucker (Karush-Kuhn-Tucker) conditions. There without some special assumptions, e.g., having to do with cases of convexity, the conditions are only necessary for optimality and not sufficient. Generally in practice, it is more difficult to get sufficient conditions. Yes, correlation does not necessarily mean causality. Usually showing causality needs a mechanism; in practice showing causality just from data and/or statistical methods is difficult and rare. But in practice, correlation can be powerful enough to take money to the bank.
mochomocha 7y ago
I can confirm this. I've run similar studies at my current employer (some of them with one of the persons you cite in your article) on multiple very large advertising platforms for display advertising. The causal ad effect is very often indistinguishable from statistical noise.

glofish 7y ago

Do you think that the situation is due to an arms race kind of scenario? -If everyone but one stopped advertising then the one still doing it would reap massive benefits? So advertising is a form of obstructing the competitor.

marrone12 7y ago

There is a selection bias in these studies as well -- that you are focusing on large, already established brands that have good brand recall and people who had intent to purchase there anyways. For smaller companies that are just starting out, it's relatively impossible to have prior intent when people don't know who you are or what you offer. There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.

sharkmerry 7y ago
> There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned. But is that number statistically significant? some do, how many others tried the same route and failed, how many grew without it, etc

graycat 7y ago

> using conventional statistical methods. Experimental design, analysis of variance are such. E.g., for the farmers and from the corn fields and hog pens of Iowa: George W. Snedecor and William G. Cochran, Statistical Methods, The Iowa State University Press, Ames, Iowa. These methods have been widely used in the social sciences -- e.g., my wife, Ph.D. in mathematical sociology from Hopkins, got quite good with that material. The field is quite serious and mature and goes well beyond just A/B testing. For the practical challenges of the article, academic fields closer than economics include statistics and optimization. For the Lagrange multipliers in the article, those likely would be from the Kuhn-Tucker (Karush-Kuhn-Tucker) conditions. There without some special assumptions, e.g., having to do with cases of convexity, the conditions are only necessary for optimality and not sufficient. Generally in practice, it is more difficult to get sufficient conditions. Yes, correlation does not necessarily mean causality. Usually showing causality needs a mechanism; in practice showing causality just from data and/or statistical methods is difficult and rare. But in practice, correlation can be powerful enough to take money to the bank.

mochomocha 7y ago

I can confirm this. I've run similar studies at my current employer (some of them with one of the persons you cite in your article) on multiple very large advertising platforms for display advertising. The causal ad effect is very often indistinguishable from statistical noise.