3 ms·
This is an open secret among techonomists and others in the arcane space of digital campaign measurement.[0][1- preprint] Digital advertising has a lot of smoke
by tomrod 8y ago
This is an open secret among techonomists and others in the arcane space of digital campaign measurement.[0][1- preprint] Digital advertising has a lot of smoke and mirrors. There is _some_ incrementality, but it seems oversold.
[0] https://academic.oup.com/qje/article-abstract/130/4/1941/1914592 https://academic.oup.com/qje/article-abstract/130/4/1941/191...
[1] https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2498290_code1130222.pdf?abstractid=2367103&mirid=1 https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2498290_cod...
- everdev 8y agoAnd how much of the internet is powered by digital advertising? The web will look a lot different if that bubble bursts.
- qha 8y agoGoing back to what it was 20 years ago? I'm sold!
- bryanrasmussen 8y agoIf the web went back to what it was 20 years ago that would be economically catastrophic, I mean if it happened like nearly instantly, if it happened over a space of 10-15 years OK.
- hedora 8y agoIn the context of the quote, let’s ignore non-ad based stuff, like amazon’s shipping network, and online banking. In what way would resetting the ad based parts 20 years be catastrophic? Were RSS and semi-curated news portals like yahoo and google news that much worse for publishers? (I thought they were actually better, economically, than FB...)
- bryanrasmussen 8y agothe context of the quote was that the web went back to what it was 20 years ago, ignoring the non-ad based stuff is cherry picking after the fact.
- manigandham 8y agoThere was basically nothing on the internet 20 years ago.
- humanrebar 8y agoMapQuest and Amazon both started around then. People stood up personal sites at places like GeoCities. There were chat services like ICQ.
- joering2 8y agoOh boy memories.. i still remember my six digit ICQ number. It had to be circa 2000?
- qha 8y agoI vividly remember lots of things... what I wouldn't give for seeing IRC rise up again
- AznHisoka 8y agoThere were just a bunch of DragonBall webmasters trying to rank #1 for DBZ in Altavista and Yahoo, but couldnt take that spot from VegettoEX.
- checkyoursudo 8y ago
- dependsontheq 8y agoThere is no going back, we are now talking about a 90% mobile network, that is used by some people every two minutes. This is not even comparable to the network we had. If we switch to a non digital ad world, it will be a net with different classes of citizens. I‘ll gladly pay for my 30 subscriptions but a whole global class of people will not be able to.
- trothamel 8y agoWe'd have websites sponsored and run by people willing to pay to put them online to push their agenda. Versus now, when we have websites sponsored by advertising and run by people trying to push their agenda.
- joejerryronnie 8y agoI've been awaiting the return of marching ants and "under construction" gifs for years.
- tomrod 8y agoMuch.
- jammygit 8y agoIt seems like that particular article is mostly about variance and methodological concerns. I only saw the abstract though
- mlthoughts2018 8y agoI dispute what you are saying, as someone who has studied that Lewis & Rao paper in detail. Advertising academia is pretty much just as mixed up as advertising in practice. Here is another paper worth reading on it [0] (which cites Lewis & Rao), but provides fairly clear evidence of a distribution of outcomes of digital ad campaigns with a lot of probability mass on the side of positive lift. The place where issues come in via Lewis & Rao is on the side of poor statistical methodology, particularly in the calculation of necessary statistical power. But those discussions are all stuck in bad frequentist estimation techniques, and in industry there are practitioners using Bayesian methods, for example such as Gelman’s proposed “type sign” and “type magnitude” errors, with well-calibrated priors that treat the possibility of negative lift seriously. I strongly disagree with the idea that “ads don’t work” is an “open secret” like you say. Rather, “measuring ads is extremely complicated” is the known conclusion, and marketers are just coming around to the fact that you have to invest in extremely advanced statistical methods to even know if ads are working, let alone to optimize them for an audience, especially inclusive of things like audience privacy, opt-out policies, paychological well being and so on. [0]: https://marketing.wharton.upenn.edu/wp-content/uploads/2017/08/Johnson-Garrett-PAPER-VERSION-2.pdf https://marketing.wharton.upenn.edu/wp-content/uploads/2017/...
- tomrod 8y agoThe open secret is that the incremental impact is small, not that it is non-existent. In experience, small enough to be questionable economically. The more interesting work has been showing where a consumer's offline transactions are impacted by online ads.
- mlthoughts2018 8y agoThe cited paper above, and even the influential 2018 Facebook paper by Gordon et al that tries to lobby for randomized controlled trials both disagree with your claim. The observed incremental lift on digital conversion outcomes (particularly account signups and purchases from an online shopping cart) is (median) somewhere around 8% - 17% based on close to 500 different very large field experiments. The distribution is skewed so that the ~30% standard deviation represents mostly outcomes higher and only somewhere between 9% and 22% showed negative lift (which, as the Facebook paper points out, can also be related to the difficulty of observational statistical methods used for post facto causal inference). 8% lift is quite significant. There’s no open secret of “small impact” at all. https://www.kellogg.northwestern.edu/faculty/gordon_b/files/fb_comparison.pdf https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...