4 ms·

Overstated. This article assumes that: - All online advertising is Google search - That brand search advertising is representative of the entire industry The

Overstated. This article assumes that: - All online advertising is Google search - That brand search advertising is representative of the entire industry There are plenty of companies making double-digit ROAS (return over ad spend), especially direct response advertisers.

neonate 7y ago
Maybe overstated, but if "all" the article shows is that Google search ads don't work, that's a big deal.
nabla9 7y ago
How do you know it's not selection effect and the ROAS number are BS just like they are with Google? Have you tested? Google and Facebook are 60% of the market, so if it would be only them this is important.
jessefrederik 7y ago
Hi, I'm the author of the article, and I don't assume all online advertising is Google search. I also cite research on display advertising (meta-study of 432 display experiments on Google): https://www.ssrn.com/abstract=2701578 https://www.ssrn.com/abstract=2701578 And on Facebook advertising: https://www.kellogg.northwestern.edu/faculty/gordon_b/files/fb_comparison.pdf https://www.kellogg.northwestern.edu/faculty/gordon_b/files/... The problem with all online advertising is that there are huge selection effects, which are hard to correct for using conventional statistical methods. So you need to do experiments. And when economists do experiment, they find that advertising effects are so small that they are hard to measure.
- glofish 7y ago
  Do you think that the situation is due to an arms race kind of scenario? -If everyone but one stopped advertising then the one still doing it would reap massive benefits? So advertising is a form of obstructing the competitor.
- marrone12 7y ago
  There is a selection bias in these studies as well -- that you are focusing on large, already established brands that have good brand recall and people who had intent to purchase there anyways. For smaller companies that are just starting out, it's relatively impossible to have prior intent when people don't know who you are or what you offer. There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.
  - sharkmerry 7y ago
    > There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned. But is that number statistically significant? some do, how many others tried the same route and failed, how many grew without it, etc
- graycat 7y ago
  > using conventional statistical methods. Experimental design, analysis of variance are such. E.g., for the farmers and from the corn fields and hog pens of Iowa: George W. Snedecor and William G. Cochran, Statistical Methods, The Iowa State University Press, Ames, Iowa. These methods have been widely used in the social sciences -- e.g., my wife, Ph.D. in mathematical sociology from Hopkins, got quite good with that material. The field is quite serious and mature and goes well beyond just A/B testing. For the practical challenges of the article, academic fields closer than economics include statistics and optimization. For the Lagrange multipliers in the article, those likely would be from the Kuhn-Tucker (Karush-Kuhn-Tucker) conditions. There without some special assumptions, e.g., having to do with cases of convexity, the conditions are only necessary for optimality and not sufficient. Generally in practice, it is more difficult to get sufficient conditions. Yes, correlation does not necessarily mean causality. Usually showing causality needs a mechanism; in practice showing causality just from data and/or statistical methods is difficult and rare. But in practice, correlation can be powerful enough to take money to the bank.
- mochomocha 7y ago
  I can confirm this. I've run similar studies at my current employer (some of them with one of the persons you cite in your article) on multiple very large advertising platforms for display advertising. The causal ad effect is very often indistinguishable from statistical noise.
dontich 7y ago
If a company has incrementality tested and is really making double digit ROAS they aren't spending enough haha.
- mr_toad 7y ago
  Not sure if you’re joking or not, but it makes obvious economic sense to keep investing in something that has a higher rate of return than other investments - and you can’t reliably get a 50% return on any other investment I’m aware of. And it only takes a moments thought to realise that once everyone starts doing this, the marginal rate of return is going to be driven down to the going rate in the rest of the market. Like any other investment, if it sounds too good to be true it almost certainly is.
buboard 7y ago
Worth mentioning the conflict of interest that you work for facebook's Ads Growth.
- davidedicillo 7y ago
  Ah, not anymore. Outdated information, I work on a different advertising platform :)
tomrod 7y ago
As the other commentator identified, you work @ FB -- could you please provide the following three scatter plots to support or disprove your claims?: (1) Size of Company vs. ROA (2) Advertising Expenditure by Company vs. ROA (3) [Advert %] vs. ROA I hypothesize you'll see a rapidly decreasing return on advertising due to size, and would love to be proven wrong.
- davidedicillo 7y ago
  Not at FB anymore but I would assume your hypothesis isn't too far from the truth. It's also true that there are hundreds of thousands of small businesses that thrive thanks to online advertising.
  - tomrod 7y ago
    The US currently has about 25M small businesses. How many of them use digital advertising? What testing is done to ensure their ROA isn't a waste?
mochomocha 7y ago
The ROAS numbers given by FB (your employer) are far from being a real estimate of the causal effect of adverts. For example, there's no way to have global holdback against other advertising channels not controlled by FB. Double-digit anything are easy to get on any advertising perfomance metric if you don't scale your spend, so it's not really an argument. (Disclaimer: I've built multiple bidders & ads incrementality models over the years so I'm probably biased against the "supply" side of ad inventory).

neonate 7y ago

Maybe overstated, but if "all" the article shows is that Google search ads don't work, that's a big deal.

nabla9 7y ago

How do you know it's not selection effect and the ROAS number are BS just like they are with Google? Have you tested? Google and Facebook are 60% of the market, so if it would be only them this is important.

jessefrederik 7y ago

Hi, I'm the author of the article, and I don't assume all online advertising is Google search. I also cite research on display advertising (meta-study of 432 display experiments on Google): https://www.ssrn.com/abstract=2701578 https://www.ssrn.com/abstract=2701578 And on Facebook advertising: https://www.kellogg.northwestern.edu/faculty/gordon_b/files/fb_comparison.pdf https://www.kellogg.northwestern.edu/faculty/gordon_b/files/... The problem with all online advertising is that there are huge selection effects, which are hard to correct for using conventional statistical methods. So you need to do experiments. And when economists do experiment, they find that advertising effects are so small that they are hard to measure.

glofish 7y ago
Do you think that the situation is due to an arms race kind of scenario? -If everyone but one stopped advertising then the one still doing it would reap massive benefits? So advertising is a form of obstructing the competitor.
marrone12 7y ago
There is a selection bias in these studies as well -- that you are focusing on large, already established brands that have good brand recall and people who had intent to purchase there anyways. For smaller companies that are just starting out, it's relatively impossible to have prior intent when people don't know who you are or what you offer. There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned.
- sharkmerry 7y ago
  > There are a number of companies who started with nothing and grew their business via online ads and this seems like a giant blind spot in the article as well as the studies that you mentioned. But is that number statistically significant? some do, how many others tried the same route and failed, how many grew without it, etc
graycat 7y ago
> using conventional statistical methods. Experimental design, analysis of variance are such. E.g., for the farmers and from the corn fields and hog pens of Iowa: George W. Snedecor and William G. Cochran, Statistical Methods, The Iowa State University Press, Ames, Iowa. These methods have been widely used in the social sciences -- e.g., my wife, Ph.D. in mathematical sociology from Hopkins, got quite good with that material. The field is quite serious and mature and goes well beyond just A/B testing. For the practical challenges of the article, academic fields closer than economics include statistics and optimization. For the Lagrange multipliers in the article, those likely would be from the Kuhn-Tucker (Karush-Kuhn-Tucker) conditions. There without some special assumptions, e.g., having to do with cases of convexity, the conditions are only necessary for optimality and not sufficient. Generally in practice, it is more difficult to get sufficient conditions. Yes, correlation does not necessarily mean causality. Usually showing causality needs a mechanism; in practice showing causality just from data and/or statistical methods is difficult and rare. But in practice, correlation can be powerful enough to take money to the bank.
mochomocha 7y ago
I can confirm this. I've run similar studies at my current employer (some of them with one of the persons you cite in your article) on multiple very large advertising platforms for display advertising. The causal ad effect is very often indistinguishable from statistical noise.

dontich 7y ago

If a company has incrementality tested and is really making double digit ROAS they aren't spending enough haha.

mr_toad 7y ago
Not sure if you’re joking or not, but it makes obvious economic sense to keep investing in something that has a higher rate of return than other investments - and you can’t reliably get a 50% return on any other investment I’m aware of. And it only takes a moments thought to realise that once everyone starts doing this, the marginal rate of return is going to be driven down to the going rate in the rest of the market. Like any other investment, if it sounds too good to be true it almost certainly is.

buboard 7y ago

Worth mentioning the conflict of interest that you work for facebook's Ads Growth.

davidedicillo 7y ago
Ah, not anymore. Outdated information, I work on a different advertising platform :)

tomrod 7y ago

As the other commentator identified, you work @ FB -- could you please provide the following three scatter plots to support or disprove your claims?: (1) Size of Company vs. ROA (2) Advertising Expenditure by Company vs. ROA (3) [Advert %] vs. ROA I hypothesize you'll see a rapidly decreasing return on advertising due to size, and would love to be proven wrong.

davidedicillo 7y ago
Not at FB anymore but I would assume your hypothesis isn't too far from the truth. It's also true that there are hundreds of thousands of small businesses that thrive thanks to online advertising.
- tomrod 7y ago
  The US currently has about 25M small businesses. How many of them use digital advertising? What testing is done to ensure their ROA isn't a waste?

mochomocha 7y ago

The ROAS numbers given by FB (your employer) are far from being a real estimate of the causal effect of adverts. For example, there's no way to have global holdback against other advertising channels not controlled by FB. Double-digit anything are easy to get on any advertising perfomance metric if you don't scale your spend, so it's not really an argument. (Disclaimer: I've built multiple bidders & ads incrementality models over the years so I'm probably biased against the "supply" side of ad inventory).