9 ms·
Google created a fake pizza brand to test creative strategies for YouTube ads
- lalos 8y agoThis concept is very creepy. This particular pizza example reported is not but who knows if they come clean on ALL their experiments. They have a lot of power in their hands and now they are running experiments on people without them even knowing? This can get out of hand just like ads were used to push certain narratives during elections all over the world in all platforms and just like Facebook was playing with people's emotions with different posts shown [0]. No regulation, no ethic panel, no external audit, no transparency and they still show it off innocently. [0] https://www.theguardian.com/technology/2014/jun/30/facebook-emotion-study-breached-ethical-guidelines-researchers-say https://www.theguardian.com/technology/2014/jun/30/facebook-...
- ThomPete 8y agoI am trying to understand the moral/ethical issue here. Care to elaborate? There might be something I am missing here cause I just don't see it. It's a genuine question.
- Bluestrike2 8y agoGoogle ran ads under a false brand name to test how viewers reacted to them. From what I can tell, the marketing experiment was fairly similar to standard A/B tests advertisers on ad design and copy or the kind of tests used by designers and developers for new site functionality. The biggest difference being that--and this is the possible ethical concern from the parent comment--the the brand wasn't real. Users had no way of knowing who the ad was produced by, or who was attempting to analyze their behavior. They aren't totally analogous, but there are circumstances where obscured identity is tested: focus groups market research tests that ask consumers opinions about possible product lines and/or changes, political polling asking how potential voters would perceive an unnamed politician (such as might done when developing a response plan prior to a major scandal breaking...if they have time), etc. In those examples, however, the subject at least knows who is doing the research if not who commissioned it. And most importantly, they give consent. The Facebook paper was a much more significant ethical concern, largely because it was (1) clearly a psychological study and was, in fact, published as such[0]; (2) had the explicit goal of eliciting changes in subject emotional responses and behavior in order to test the hypothesis; and (3) made no effort to comply with standard ethical practices for informed consent in human research. The paper got around the lack of informed consent because Facebook was a private company and was able to point to their Data Use Policy. That was enough of a fig leaf for Cornell's Institutional Review Board to accept because the Cornell professor was only given access to completed data after the experiment was independently run by Facebook. They accepted it in those circumstances, but I doubt anyone was very happy about it. Google's experiment is quite tame in comparison, and there are obvious parallels to the sort of benign A/B testing marketers use. There's also--arguably--a pretty major difference in the type of experimentation going on, and it's not intended for academic publication. Also, YouTube terms of use presumably contain language that gives them enough cover to avoid any legal consequences. Ethicists and university IRBs put a lot of thought into where the boundaries lie and how proposed experiments can cross them unexpectedly. Some of those boundaries have been learned about after some truly horrendous experiments in our past that nobody ever wants to even consider repeating. Others have been much less destructive, but nevertheless with their own flaws and problems. Personally, I think Google ought to have at least labeled it as a research ad and put their name on it at the end or on the page users clicked through to. While this example was fairly benign with only relatively minor ethical concerns, I understand the parent comment's concerns. There's a lot of interesting questions that can be researched, and I'd imagine some pretty abusive ways in relevant data could be discovered. In that context, the discussion's a good one to have for both the public and the companies themselves. Not because they're likely to jump into those abusive methods or because they'll repeat something along the lines of the Milgram experiment (or far, far worse). It's simply useful to understand the pitfalls beforehand. I'm not normally one for slippery slopes, but there are instances where considering them beforehand helps avoid the problematic outcomes altogether. 0. http://www.pnas.org/content/pnas/111/24/8788.full.pdf http://www.pnas.org/content/pnas/111/24/8788.full.pdf
- iambenjones 8y agoJust a note of response here: We did put our name on it--there was a landing page with details on the aims of the research, the methodology used, etc. It included contact info, and we responded personally to any and all questions. Ben (the creative director on this research)
- anonymfus 8y agoThese ads were lying.
- kowdermeister 8y agoAren't all ads lying?
- DowsingSpoon 8y ago>Aren't all ads lying? Not at all. Please refer to the Wikipedia page on "False advertising" for an overview of what does and does not constitute false advertising. https://en.wikipedia.org/wiki/False_advertising https://en.wikipedia.org/wiki/False_advertising
- Skunkleton 8y agoDon't you deserve to be free of Symptom X? Try drug Y! *warning results vary, and may result in sudden death.
- anyfoo 8y agoI keep being baffled at how transgressions tend to get handwaved away with "everybody does it" by some commenters on Hacker News. First, there are plenty of locations where advertising is heavily regulated. Second, even in locations where that is not true, "everybody does it" does not justify anything. Every surgeon used to perform their job without washing their hands, and yet we got rid of that behavior.
- kowdermeister 8y ago> "everybody does it" does not justify anything Indeed, it doesn't, I just found the commenter being way too captain obvious :)
- oh_sigh 8y agoWhat did the ads lie about? That fresh mozzarella tastes good?
- lalos 8y agoThere is a lot to it. You can start here: https://en.wikipedia.org/wiki/Nuremberg_Code#The_ten_points_of_the_Nuremberg_Code https://en.wikipedia.org/wiki/Nuremberg_Code#The_ten_points_... Specially point number 1. Also read on the history of Nuremberg Code and why it was triggered after WW2. Also you could read this explanation by a professor of law at U of Maryland http://laboratorium.net/archive/2014/06/28/as_flies_to_wanton_boys http://laboratorium.net/archive/2014/06/28/as_flies_to_wanto... which talks about Facebook research which is similar to this one. In general: you are tricking the end user, this is not a simple A/B test this is more of here is a company that does not exist providing nothing but let's see how you react to it. Today is fake pizza, tomorrow is a fake HIV med that cures it in one day (you click on it, there is a strong inference that you have it), etc the possibility is endless. Who is deciding this? That's why I mention ethic panels or regulation or whatever way of putting limits on it. The only good thing here is now we know that we are test subjects when using Youtube/Google.
- philwelch 8y agoThis is almost literally a Godwin argument. Your argument basically unpacks into, "there's a dangerously slippery slope between fake door tests and war crimes", and it's most often used as a self-serving justification by bureaucrats who want you to do hours of busy work before giving people a questionnaire: http://slatestarcodex.com/2017/08/29/my-irb-nightmare/ http://slatestarcodex.com/2017/08/29/my-irb-nightmare/ Look, marketing researchers are not going to just accidentally vivisect concentration camp inmates to death just because you leave them unregulated. It's not like people just accidentally go around committing crimes against humanity because there aren't any regulations stopping them from doing so. In fact, at Nuremberg, it turned out that most of them committed crimes against humanity specifically because there were regulations requiring them to do so! This isn't to completely pooh-pooh the necessity of research ethics and regulation. But those regulations need to be based on realistic risk models.
- lalos 8y agoIt is not my fault that the rules that govern research and interaction between researchers and tests subjects occurred because of WW2 and that Godwin's law exists. I mentioned that as a historic fact. So nobody can talk about how this started because it is literally Godwin's law? Great idea.
- Apocryphon 8y agoThere's definitely been creepy experiments done on Facebook and OkCupid when they play with peoples' emotions and social responses- especially on platforms they themselves control, but this specific example just seems like market research at its purest: seeing how people respond to a typical product. Agreed that such experiments should be published, or at least made to follow ethical guidelines.
- rhizome 8y ago25 years ago I'd regularly (every month or two) get focus group invitations that paid $100+/hr. Along with informing the subjects, compensating participants is a rock-bottom ethical standard that Google apparently cannot meet.
- joshuamorton 8y agoDo you consider all A/B tests to be studies one should be compensated for? Not to mention that they are compensating you by sponsoring the content you're viewing, but that should go without saying. (Focus groups and A/B tests are different, and afaik, Google does compensate for focus groups)
- rhizome 8y agoOf course not, but where do you draw the line?
- joshuamorton 8y agoThat's a good question. So let's clarify: Would this be unethical if it was a real pizza company doing an A/B test for their real brand? Would this be unethical if it was a real pizza company doing the A/B test for a fake brand? Is it unethical for Google to have done this? You've just said no to the first question, and previously yes to the last. so I'm curious where you lie on the second, and how you justify those different answers. I don't see a difference between any of them. (in other words: the line is "when its no longer an A/B test and they're actively soliciting your feedback", but you draw it differently. Why?)
- norswap 8y agoNow? Anyone's willing to spend ad $ has been able to do this for ages. It's basic A/B testing. It's impossible to prevent.
- jacquesm 8y agoThere's a Dutch IT company (Topicus) that has an actual Pizza brand, and they brew their own beer too!
- drdeadringer 8y agoThis reminds me of a hotel that elected to advertise its own "Room Service Pizza" as a local commercial business. It went something like this... A given guest checks in, unpacks, and finds themselves ready for dinner. They skim over the room service menu, considers Room Service Pizza, and then dials a nearby pizza chain for delivery. Reviews for Room Service Pizza were not good. At the time, "Dominoes Is Cardboard" among similar chain-pizza love was going on so it wasn't looking up for Room Service Pizza even though it was right there on site and arguably not that bad as the story goes. The hotel tried "everything" but it would never work; recipes, surveys, fresh ingredients... "One Chain Pizza, Please" was always the result. One day someone desperate and with authority had an idea. They had changed "everything else" by this point, so let's go "marketing". This person drums up a separate phone number, business name, menu, logo, uniforms, the whole bit for Room Service Pizza. Drops the flier menu into the "local restaurants" courtesy spread in their rooms. It's not "Room Service Pizza" anymore, it's "Tony's Tower Pizza" or whatever. Orders pick up. Reviews go up. Deliveries for Chain Pizza start to drop a bit on a noticeable level. Every time the special phone line for "Room Service Pizza", aka "Tony's Tower Pizza", rang up they knew something had finally broke free for themselves. ... So, I guess my point is that this isn't the first time pizza and playing masquerade have come together for a business. Here, a hotel uses ads and "rebranding" to sell their own pizza vs Google using fake pizza to sell more ads. If pizza were a coin, these might be opposite sides of it.
- dawnerd 8y agoUniversal Orlando has their own in house pizza that’s totally branded differently and they even throw flyers under the hotel room doors to make it look like a local chain. Thing is, it’s actually pretty decent pizza and the price isn’t too bad either. In their case I think most people have caught on to theme park resort pizza being awful so branding was essential.
- nerdponx 8y agoit’s actually pretty decent pizza and the price isn’t too bad either. Most room service pizza is. Suggestion is a powerful force.
- legitster 8y ago1. It's hard to understand these conclusions without knowing what they used to measure the effectiveness. Clicks? Skips? 2. From the headline, I assumed that they used a fake brand to track the number of searches for it later. Perhaps to see how paid ads could influence organic search. It seems like that would have been a more interesting result.
- ectospheno 8y agoI read the article and I also can't figure out how they measured what they claim to have measured. The results seem made up in much the same way other advertising metrics and results seem made up.
- alentodorov 8y agoWhenever I used to run Youtube ads ($100k campaigns) the account manager at Google Ads informed us about an automated option of neasuring assisted and unassisted recall. People whom seen the ads were presented with one question surveys on Youtube. It was pretty interesting as you could see what would be a good reach and frequency for your ads.
- iambenjones 8y agoHey--This is Ben--I did the research. We used a survey-based tool called Brandlift that measures post-exposure lifts in recall and favorability against control audiences using a defined competitive set. The tool automatically separates out the control audiences and the various test cells so there is no cross contamination. There's detail on the tool here: https://www.thinkwithgoogle.com/products/brand-lift/ https://www.thinkwithgoogle.com/products/brand-lift/ Happy to answer any other questions you have--glad you found it interesting...
- marzell 8y agoIs this even legal, given false advertising guidelines? They have provided an advertisement for products by a company that doesn't exist. I don't know the legal details, but I am questioning whether this is actually permitted in the U.S. if you go by the letter of the law.
- mindfulhack 8y agoMy thoughts exactly. I agree that companies should to have the freedom to be able to test markets to optimise their systems designs (free trade is important), but surely a multi-billion-dollar company can use a real business and spend a little extra dough to test their system without breaking the law. This seems lazy, and legally it should be investigated. And this doesn't look good for their brand either, now that we live in an age of 'fake ads' swinging entire US elections.
- oh_sigh 8y agoI am not an expert in this area, but none of the ads I watched seem to actually advertise anything. There is basically just a picture of a pizza and a saying like "The tangy sweetness of fresh mozzarella" https://www.youtube.com/playlist?list=PLFfnLmpHEyLYLqHdyOmVElmy_5isisklx https://www.youtube.com/playlist?list=PLFfnLmpHEyLYLqHdyOmVE...
- daeken 8y agoFor this to be false advertising, there would need to be an offer that's extended to customers. For instance, "$10 large pizza" would be an offer: give us $10 and we give you a large pizza. If they failed to follow through on that deal (by, say, not accepting money and not giving pizzas) then that would be false advertising. There's no such offer in these ads, or anything that could even be remotely construed as an offer. This is not an advertisement for chocolate cake: https://tastesbetterfromscratch.com/wp-content/uploads/2016/12/Dark-Chocolate-Cake-5.jpg https://tastesbetterfromscratch.com/wp-content/uploads/2016/...
- modells 8y agoDamn, I'm jonesing for a Celeste four-cheese microwave pizza really bad rn. Where might a lowly member of the public borrow magical 1970's microwave technology in the Mountain View area? PS: Anyone have one of those huge Radarange RR-9 microwaves with that (capacitive?) touch-panel in the late 70's or early 80's? Yup, I'm ooooold.
- sarcasmic 8y agoThis is not without precedent in the ad industry, although the level of subtlety varies a lot. Billboards are a notable example, because the medium is in-house and there's almost always latent inventory. Adams Outdoor once advertised the fictitious, toilet humor brand 'Outhouse Springs', then years later the seemingly personality-enhancing wonder drug 'Reachemol'. Lamar's Milwaukee division advertised a cat doctor with a preference for chocolate who healed 'boo-boos with nom-noms'. This is essentially a big A/B test, and unlike billboards, they can collect fine-grained data on how people interact with the ad.
- trumped 8y agolooks like engineers or robots decided on the store name... "Doctor Fork"? for a pizza restaurant?
- ronilan 8y agoThey ran an experiment. Users reported that eating pizza with a fork was much preferred to eating it with chopsticks. The Chinese version may differ though. They’ll test this later if the party allows.
- thefounder 8y agoWho eats pizza with chopsticks?
- partiallypro 8y agoI do wonder if some of the outcomes were dictated by people wondering about what this new brand was. It would be different if they were serving small populations, but they served it 20 Million times. I guess that is a good case study for small brands, but do the results stay the same when people recognize the brand?
- sushid 8y agoIt seems like this might be the precursor to doing a real test (probably small/regional) with a real brand. If the response was extremely negative I'd think that they'd shelve it altogether.
- fipple 8y agoThis is like the age-old "hack" of creating a landing page for a B2B startup that doesn't exist, to gauge demand and build a marketing list.
- rhizome 8y agoThat's the canonical definition of an MVP.
- philwelch 8y agoIt's usually called a "fake door test". Fake door tests are neither viable nor products :)
- cookingrobot 8y agoAn MVP has just enough features to satisfy early customers. If all you've built is a landing page to gauge interest, that's useful for validating the idea, but that's not a product.
- lucasmullens 8y agoThe iconic Dropbox MVP was just a video, so I'd argue an MVP can often refer to something like a simple landing page.
- mygo 8y agoI mean... just because someone can call a video an MVP doesn’t mean it was an MVP. An MVP is a functioning product/service. It’s something that provides enough utility / functionality that the user can actually use. Unless dropbox was a media company demoing a new film, the video was not an MVP. It was a marketing campaign.
- amelius 8y agoTl;dr: Google did a bunch of experiments to see how best to manipulate people. They used a fake pizza brand "Doctor Fork" for this purpose.
- gukov 8y agoIf you're not paying, you're the prod... test subject.
- in_hindsight 8y agoIs it the pizza they called during voice assistant demo? ;)
- wiradikusuma 8y agoThe 1st and 2nd conclusions seem to contradict each other: one said put everything together while the other one said no?
- tabtab 8y ago"Totally fork news, believe me!"