11 ms·

Facebook Overestimated Key Video Metric for Two Years

283 points by tshtf 10y ago

cft 10y ago
We are using Facebook Audience Network in random order with other ad network for mobile ads. FB ads earnings are about 3-5% of other networks, with exactly the same traffic. Other networks disclose the percentage the revenue split between them and the publishers, but FB does not. What do they overestimate in this case, I wonder?
codehusker 10y ago
On my programmer hand, I can see myself making a similar technical choice. We have auto-playing videos, we shouldn't count the views less than 3 seconds because that wasn't really an intentional view. But that means we should also exclude them from the overall view count. I don't know if FB did that as well. On my shareholder hand, this seems slightly like fraud. On my advertiser foot, this seems slightly like fraud.
- karmel 10y ago
  One wonders whether this was an intentional filter to avoid scans of the video that shouldn't count as watching, or just a bug – some deeply nested, vestigial filter that everybody forgot about until some random engineer stumbled across it...
- daveguy 10y ago
  That is an important question that wasn't clear in the article. If they counted those sub 3 sec views in the count but not in the duration then it would definitely be fraudulent. If they didn't include sub 3 sec in the count or in the average then it is a valid technical decision. I am guessing they were included in the count but not in the duration. They should have a histogram of durations.
  - arcticfox 10y ago
    That distinction makes all the difference to me. IMO one answer is fraud and the other is borderline valid / no big deal.
  - dismantlethesun 10y ago
    It looks like they didn't count it in either metric. Peoples' complaint is that they excluded it from all averages and all reports, which inflates their average duration of video watched because videos less than 3 seconds aren't included at all.
  - prostoalex 10y ago
    > If they counted those sub 3 sec views in the count but not in the duration It sounds like this was exactly what was happening. From the article: "Video views of under three seconds were not factored in, thereby inflating the average. Video views of under three seconds were not factored in, thereby inflating the average. Facebook’s new metric, “Average Watch Time”, will reflect video views of any duration."
- nol13 10y ago
  See just get rid of the auto-playing videos problem solved, and make the world a better place while you're at it.
  - colejohnson66 10y ago
    I bet Facebook gets a decent enough amount from people who scroll, look away for a second, video starts playing, then user pauses video, but not before the 3 second mark. What really should be done is getting rid of auto playing videos at the HTML spec level. While there are legitimate uses for autoplaying videos (YouTube for example), it has too much potential for abuse by ad networks.
    - cpeterso 10y ago
      Advertisers will find a workaround and it will inevitably be much worse for the user. In the following talk, someone from the Google Chrome team describes how, when they tried to not autoplay videos on Android, video advertisers worked around the restriction by compiling video codecs to JavaScript and then rendering their video-like ads to an HTML canvas. https://youtu.be/wQGa_6CRc9I?t=23m52s https://youtu.be/wQGa_6CRc9I?t=23m52s
      - colejohnson66 10y ago
        That's where NoScript comes in
        NoGravitas 10y ago
        Or blocking 3rd party scrips with uBlock.
      - raverbashing 10y ago
        Don't ad networks charge for ad size as well? (or they should start charging for this - would cut on abuse) If your ad is > 50k then you're paying a fee per display, even if you're "pay per click"
    - 10y ago
- mod 10y ago
  On my programmer hand, if I made the cutoff 3 seconds for the metrics, I would make it 3 seconds to count a view. If the metrics and the view counts match, I don't see a problem. If they calculate the average for > 3 seconds, but count EVERY view, I agree--fraudish.
  - AstralStorm 10y ago
    This sort of Windsorizing will inflate the average anyway. Making decisions based on "average" view time is still silly.
    - luiscarloscb 10y ago
      Why would it be silly? If I have an ad that has double the average view time than other wouldn't it make sense to invest more in it and less in the second one?
      - LeifCarrotson 10y ago
        The distribution is what they really need to include. The average view time doesn't mean much when the distribution is super skewed, which it's likely to be for these ads. If 0.1% watch all 30 seconds, 0.9% watch to 28 seconds and quit at the credits, and the rest all watch 0.2 seconds as they scroll through, that tells you something that "Average watch time = 0.5 seconds" doesn't tell you. Especially if last week, no one watched the whole thing, but a bunch of people watched for 2 seconds and then quit, and your average was 0.9 seconds.
bonniemuffin 10y ago
I don't think average watch time is a particularly useful metric, because no matter how they count views, there's some kind of arbitrary cutoff. If they count a view as soon as the player finishes loading, they'll include a whole bunch of people who didn't intend to watch the video, and in fact only "watched" a few milliseconds of it before bouncing. So that will drag the metric down artificially. I bet if they count views as soon as the video starts playing, they could drag down average watch time by improving player load time, because they'd count more unintentional "views" of very short duration. I hope/assume this is part of a suite of metrics that they provide to advertisers, so you can understand it in the context of other things like video completion rate or counts of views that reach X% through the video.
breatheoften 10y ago
What was the actual bug? Does this mean that no video had an average view time less than 3 seconds?
- sharkmerry 10y ago
  They dont count views for less than 3 seconds in their view count. They didnt include views of less than 3 seconds in their "average view time" metric, which is very misleading. Their metric was "average view time of video's in our view count". Which is not the same thing and the latter vastly inflates the effectiveness of facebook video
emcq 10y ago
This seems slightly overblown. I suspect this was a metric useful for engineering that found its way to external usage. That's because in many recommendation systems in practice you want to filter out the spurious views, and a simple way to do that is with dwell time. It's not uncommon to require at least 50% of pixels in view for 1 second before you have an impression for static images [0, 1]. AOL defines an impression requiring 2 seconds for images [1]. Facebook likely did some analyses to find that 3 seconds was a good cutoff for their site. There are more sophisticated ways to estimate dwell time but they seem uncommon in practice; perhaps due to their difficulty communicating to advertisers what the impression metric actually means. For sites with many bots or inbound marketing you often find users bounce quickly which drives some of this timing. I'm a bit surprised it needs to be that high for Facebook without many bots or users bouncing quickly. Perhaps this is for mobile users scrolling quickly. [0] http://advertising.aol.com/specs/terms/aol-viewability-terms http://advertising.aol.com/specs/terms/aol-viewability-terms [1] http://mediaratingcouncil.org/063014%20Viewable%20Ad%20Impression%20Guideline_Final.pdf http://mediaratingcouncil.org/063014%20Viewable%20Ad%20Impre...
- ma2rten 10y ago
  I thought that this metric is supposed to measure the quality of the views that the advertiser is paying for. In that case maybe advertisers shouldn't pay for views under 3 seconds, but if they do it should count towards the average.
  - jdoliner 10y ago
    Based on the stat in the article about it resulting in 60% to 80% difference in total viewing time, I think they were probably reporting total views including views under 3 seconds and reporting avg view length discounting views under 3 seconds. This results in a pretty misleading viewing time. Still quite possibly a mistake, but I can see why advertisers would be pissed.
  - honkhonkpants 10y ago
    Isn't it the advertisers' job to measure the quality of the channel? Or does this kind of advertising have an intangible outcome like brand awareness?
nostrademons 10y ago
My data-weenie hat tells me that this is a silly metric anyway. As an advertiser, you really care about the distribution of viewing times, and in particular what fraction of viewers watch the video all or most of the way through. Arithmetic mean is virtually useless when the data has a power-law or other non-linear distribution, and it's highly likely video viewing times exhibits this.
- sharkmerry 10y ago
  Doesnt this vastly affect the distribution though? It excludes all views less than 3 seconds.
  - pg_is_a_butt 10y ago
    you mean the views that don't count at all? why would you want metrics about non-views to pull down the average view time? shut facebook down. you're all idiots.
- teej 10y ago
  As someone who helps make decisions around ad spend... not really. I would take a distribution if I was given it but it wouldn't substantially drive my advertising strategy. Views (over 3 sec views is fine), clicks, viewthrough conversion, clickthrough rate, and conversion rate are orders of magnitude higher signal measures than the specifics of how far they got through the video. I'd imagine that view time distributions are way more important to new media companies that rely heavily on video media like Buzzfeed or anyone putting out content on Youtube.
  - cwilkes 10y ago
    What's the clickthru rate on a movie trailer? What is that going to tell you? Do I click on ads on TV or in the newspaper? Most video ads I see are more informational than a call to action. Who really wants to click on an ad while in facebook on a phone?
    - 10y ago
pducks32 10y ago
I don't believe for a second that this was unintentional. The amount of press and mind share that Facebook has received for their "rocketing video efforts that rival YouTube" was totally worth pissing off some ad exec. Very few people will see this news compared to "Facebook dominates video" headlines that have been everywhere. This was a smart play on their part.
- donohoe 10y ago
  "Never attribute to malice that which can be adequately explained by stupidity." I can see exactly how things like this could happen in large corps. You'd be surprised.
  - reitanqild 10y ago
    This might not be totally faor towards Facebook, but: When it comes to Facebook I guess there are slightly different rules: A good rule of thumb for Facebook might be: expect malice. We have seen it with the way the made everything public. We have seen how they decided to sell out its own user a year after proclaiming their love and how this wouldn't happen.
  - thr0waway1239 10y ago
    While your point is very fair, a long track record of behavior which pushes the boundaries of ethics becomes indistinguishable from malice. In these matters, people always judge you more harshly if you have taken advantage of their trust (many many times in FB's case) in the past. Facebook wants to have the cake (the hype surrounding its success to be a self-perpetuating thing) and simultaneously eat it too (always expecting the benefit of doubt around its integrity around its self-reported success metrics). While I welcome people with means doing philanthropy, I can't help but wonder if the recent announcement of philanthropy from Mark Z is an attempt to generate good press. If Mark Z is so keen on doing charity, perhaps he should divert all that money to the top 10 privacy watchdogs in the world. In today's world, that is as much a just cause as any other, particularly because no one really wants to talk about it. Note: I am not saying the other tech giants don't indulge in these practices. Two wrongs can't make a right.
aaron695 10y ago
I don't get it, it makes sense? Less than 3 seconds is not a view any more than seeing the picture of the video is a view. It doesn't count I'd say? Or do advertisers have to still pay for a 2 second view?
bluetwo 10y ago
"likely overestimated average time spent watching videos by between 60% and 80%" Soooooooo... fraud? They say it didn't impact billings/revenue, but I have to imagine they will now be in a position to give discounts, if not refunds.
- sean_patel 10y ago
  Yes. straight up fraud. "How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q https://www.youtube.com/watch?v=t7tA3NNKF0Q
  - bbctol 10y ago
    That video is discussing a slightly different problem than the recently discovered issue the article's talking about.
jtchang 10y ago
I don't think this is overblown at all. I expect to see a class action lawsuit because of this. If you are a big advertiser and you can somehow show damages because of this error there may be a case. There are some adtech companies recently who focus purely on reporting the metrics of your ad purchases. Kind of like acting as a third party. I forgot exactly what they are called.
- shostack 10y ago
  This assumes most advertisers can prove the value of a video view of a certain length. I assure you as someone who deals with attribution on a daily basis that most cannot and the ones that claim to would have a hard time proving it in court. Attribution is hard.
sean_patel 10y ago
"How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q https://www.youtube.com/watch?v=t7tA3NNKF0Q This is neither a bug nor a "mistake". It's straight up fraud. Fraud because Facebook uses these false #s to prove to their advertizing clients that they are getting a HUGE ROI, when they really aren't. Also the investors and analysts bump up their ratings and the stock goes higher and higher on false ad view #s. By counting a view as a legitmate view after the video plays only for 3 seconds, their "algorithm" counted billions of views even when the user has not even seen it, because Auto-play is enabled by default and you have to opt-out / disable it and no-one does it. It takes a person roughly 4 seconds to scroll off their feed as they quickly "scan" their friends posts.
- yladiz 10y ago
  Do you have a citation for the "4 seconds" figure? If it is 4 seconds, and it logs a view after 3 seconds, that's pretty damning.
  - AstralStorm 10y ago
    That it logs invisible videos at all would be a problem.
  - sean_patel 10y ago
    It's not publicly discussed for obvious reasons, but I worked closely with Mudd for a couple of months last year. Here's a related interview where he touches upon it. http://digiday.com/platforms/advertisingweek2015-facebook-finds-that-fast-scrolling-millennials-consume-ads-2-5-times-faster/ http://digiday.com/platforms/advertisingweek2015-facebook-fi... Facebook spend an awful lot of resources tracking all sorts of user behaviour inside it's "walled garden". I got really sick of it after a while and decided to stop working for them (even as a contractor)
- hanspeter 10y ago
  ROI from FB ads can only be calculated from tracking conversions on the advertiser's site or in their app. View count alone can never be attributed a ROI as there is no measurable return. If you pay solely for video views on Facebook or Youtube without any tracking, you're not being a smart advertiser and it doesn't really matter if view counts are more or less correct - you're blindly throwing your money away. If you do however track your ROI correctly it doesn't really matter how many true view counts you get.
throwanem 10y ago
It's like watching a wasp land on a nettle. You know somebody's going to get stung, but you just don't care who.
yueq 10y ago
Will there be a class action on this matter?
restlessdesign 10y ago
It is unreasonable, even for a developer, to count 3 seconds as a play. Anyone working on a video product would know they were inflating the numbers.
- prostoalex 10y ago
  The top-voted comment on this thread, by emcq, has a link to AOL's viewability guidelines which states "For purposes of Video inventory, an impression is considered “In-View” to a user when at least fifty percent (50%) of the pixels in the ad are in the viewable browser window for a minimum of two (2) consecutive seconds." Google here https://support.google.com/dfp_premium/answer/4574077?hl=en https://support.google.com/dfp_premium/answer/4574077?hl=en says "For in-stream video ads, 50% of the ad’s pixels must be visible in the browser window for a continuous 2 seconds."
  - swish_bob 10y ago
    That's the IAB's standard viewable impression metric: https://www.iab.com/wp-content/uploads/2015/06/MRC-Viewable-Ad-Impression-Measurement-Guideline.pdf https://www.iab.com/wp-content/uploads/2015/06/MRC-Viewable-...
tn13 10y ago
It might be inaccurate but does not matter much in advertiser perspective. Given the conversions etc. eCPMs conversion to the real expected value.
bruinbread 10y ago
This isn't anything new. Several YouTubers have "exposed" how Facebook tracks views. It's not accurate, but it's the system that has been in place for a while and they've defended it as such.
smithunsero 10y ago
In 2012 there were reports that 80% of traffic from Facebook Ads were automatic bots. So this is not the first time Facebook use tricks to get more money from clients.
- plopilop 10y ago
  From what I read [1] it's only one startup that claimed this, and I failed to see any confirmation from any other company. For me, no replication from companies that spend thousands every month on advertising = the startup messed up with its stat tools or wanted to create a buzz around its name. [1]: https://techcrunch.com/2012/07/30/startup-claims-80-of-its-facebook-ad-clicks-are-coming-from-bots/ https://techcrunch.com/2012/07/30/startup-claims-80-of-its-f...
- hanspeter 10y ago
  Running campaigns on Facebook's advertising platform does seem to attract bots. It's not 80 % though and it's obviously not Facebook doing it. Why would they sabotage their own business?
Arkaad 10y ago
I am disappointed by FB. Next we'll learn that they do tax evasion.
supermatt 10y ago
If face book only count a view as over 3 seconds, and they measure the average view as only those videos over 3 seconds, what is the problem? Surely now its less representative?
coldtea 10y ago
>the tech giant vastly overestimated average viewing time for video ads on its platform for two years, according to people familiar with the situation They didn't "overestimate". They plainly gave false numbers to drive ad sales. Coldtea's law: Never attribute to incompetence what can be explained by profit.
- meira 10y ago
  I'd call it SV's Law, but you can get the merits of it :)
- mtgx 10y ago
  Same goes for Facebook's user tracking "bugs".
- jbooth 10y ago
  I'd qualify that. Things that appear to be generating a ton of money don't get looked at that closely, because everything's going great. Things creating losses get looked at much faster. So it's not pure deceptiveness. Probably nobody made the call "let's defraud everyone". However, a bug that undercounted the viewing time would have been uncovered much sooner.
- jbverschoor 10y ago
  + there's a reason videos auto-play....
- billmalarky 10y ago
  "It is difficult to get a man to understand something, when his salary depends upon his not understanding it!" -Upton Sinclair
  - jgalt212 10y ago
    That is basically how Wall Street works. If a trader or salesperson is printing money, management doesn't want to know how it's done. Plausible deniability will at worst get you a failure to supervise penalty, whereas if you understand how these folks are making money you could become complicit in any illicit acts and thus exposure yourself to a much larger penalty and/or jail time. Of course, the government has proven themselves time and time again unwilling to punish management so management might as well figure out how the illicit gains are being made so they can replicate such processes when the big producing trader or salesperson is invariably poached by a competing shop.
shostack 10y ago
While this is concerning, smart advertisers were looking at the distribution of view length percentages anyway as you can easily pull those columns in and their derived metrics. Likewise, smart advertisers look at lift in conversion metrics when possible, in which case this stat is irrelevant. That said, FB has not exactly helped things by making metric definitions a little obfuscated in general. Personally, I see a bigger concern is them giving 100% view through conversion credit with a 1 day window by default as part of any website conversion action tracking. There are very few cases (like some retargeting situations) where you'd ever want to give full weighting for a VT, and while there is likely value in VT's, I probably wouldn't give 100% credit to them by default given the rate at which people scroll on mobile. But advertisers like to see big numbers, agencies like to show big numbers, and so you have platforms like FB aggressively try to push metrics like this and some of their rather loose definitions of "engagement" without great explanation of the nuances or pros and cons. These are largely left up to the advertiser to determine since, to be fair, they are very subjective. Savvy buyers know this and configure their reporting and tracking settings accordingly because FB and PMDs give you those options. They are sometimes just buried. Ultimately, IMHO FB and Google's greatest defense towards any of these sorts of claims is better and (more importantly) transparent attribution data and tools. If they can prove their value on the bottom line, other things often don't matter to many advertisers. Attribution is a tough nut to crack, but for advertisers spending large sums, it is critical to be successful in these channels.
laurihy 10y ago
Then again, if a "video view" is defined as "watched over 3 seconds (50% of the video visible in the screen, IIRC)", then it sort of makes sense that "Average Duration of Video Viewed" doesn't include non-views. For sure Facebook should attempt to name their metrics as descriptively as possible, but also advertisers should make sure they understand how different conversions are measured, what's included and what's not. Another example would be "Clicks" metric, which included all engagement (i.e. likes, shares etc), instead of just "Link clicks".
- TazeTSchnitzel 10y ago
  It doesn't seem all that unreasonable, yeah. If a user scrolls right past a video, should that really count towards the average?
  - RodericDay 10y ago
    A user below summarized the issue the way I understand it: > what happens is that views under 3 seconds aren't counted towards the average. So if you have 999 999 views of <1 second each and 1 view of 3 seconds, the reported average is 3 seconds / 1 view = average view time of 3 seconds, while the true average is something like 500 000 seconds / 1 000 000 views. This is massive fraud, especially if you're shouting from the rooftops "Look at how much more attention people are paying to your videos in our platform!".
- Naritai 10y ago
  I thought about this as well, and concluded that FB must be reporting 'video views' as any time a video starts autoplaying (or something similar). In this way I could see the total estimated viewing time getting significantly overestimated.
mikejb 10y ago
I didn't fully understand what the bug was, but I found on a German news page[1] with a slightly more detailed description: Aufgezeichnet wurde zwar die Gesamtsehdauer aller User, geteilt wurde dieser Wert allerdings nur durch diejenigen User, die das Video länger als drei Sekunden ansahen So the average was not calculated correctly: They accumulated the duration of all video views (including of those shorter than 3 seconds), but divided it by the number of 'legitmate' views - i.e. only those longer than 3 seconds. So you get a pretty big offset if you have many views under 3 seconds (which they probably do, thanks to autoplay). [1] https://www.heise.de/newsticker/meldung/Facebook-Unklare-Video-Metrik-fuehrt-Werbekunden-aufs-Glatteis-3330188.html https://www.heise.de/newsticker/meldung/Facebook-Unklare-Vid...
Bedon292 10y ago
Ok, what am I not understanding here? If they are only counting a view as over 3 seconds. And then only averaging the those views what is the problem? They tell me 1000 people viewed it (over 3 seconds), and then say the average view time was 10 seconds for those 1000 users. How is this an issue? Why do I care that 9,000 people scrolled past it and weren't counted in the viewer numbers? Now if they were reporting this as 10,000 viewers and saying 10 second average, I see the problem. Is that what was happening? Or what am I missing?
- oneloop 10y ago
  My suspicion (based on nothing) is that they averaged viewing over viewings with length >3 seconds, but _counted_ all views. I.e. suppose your ad was displayed 5 times for 0.1 seconds and 5 times for 5 seconds. They would report this as "10 views, with average viewing time of 5 seconds".
  - tedmiston 10y ago
    I wish your comment was at the highest level. Everyone seems to be attempting to make the same point in the other subthreads. The example here makes it crystal clear.
    - oneloop 10y ago
      My guess was almost correct. The issue was indeed that they use "views with >3 secs" in some metrics and use "all views" in other metrics. Do one of the other, don't mix them up. http://www.adweek.com/news/technology/facebook-gave-ad-agencies-inflated-video-viewing-metric-2-years-173676 http://www.adweek.com/news/technology/facebook-gave-ad-agenc... > UPDATE: Facebook authored a blog post today to futher explain the metric-based miscue. David Fischer, vp of advertising and global operations, wrote: "About a month ago, we found an error in the way we calculate one of the video metrics on our dashboard – average duration of video viewed. The metric should have reflected the total time spent watching a video divided by the total number of people who played the video. But it didn't – it reflected the total time spent watching a video divided by only the number of 'views' of a video (that is, when the video was watched for three or more seconds). And so the miscalculation overstated this metric. While this is only one of the many metrics marketers look at, we take any mistake seriously."
randomgyatwork 10y ago
When will Facebook 'content creators'(users) get compensated for the value they create?
malloreon 10y ago
how much of their video income will they be refunding to advertisers?
sidcool 10y ago
This looks like a flat out lie. Startups have to sometimes do that to woo potential investors. Although I like Facebook overall, I do not agree with many of their business practices.

283 points by tshtf 10y ago

cft 10y ago

We are using Facebook Audience Network in random order with other ad network for mobile ads. FB ads earnings are about 3-5% of other networks, with exactly the same traffic. Other networks disclose the percentage the revenue split between them and the publishers, but FB does not. What do they overestimate in this case, I wonder?

codehusker 10y ago

On my programmer hand, I can see myself making a similar technical choice. We have auto-playing videos, we shouldn't count the views less than 3 seconds because that wasn't really an intentional view. But that means we should also exclude them from the overall view count. I don't know if FB did that as well. On my shareholder hand, this seems slightly like fraud. On my advertiser foot, this seems slightly like fraud.

karmel 10y ago
One wonders whether this was an intentional filter to avoid scans of the video that shouldn't count as watching, or just a bug – some deeply nested, vestigial filter that everybody forgot about until some random engineer stumbled across it...
daveguy 10y ago
That is an important question that wasn't clear in the article. If they counted those sub 3 sec views in the count but not in the duration then it would definitely be fraudulent. If they didn't include sub 3 sec in the count or in the average then it is a valid technical decision. I am guessing they were included in the count but not in the duration. They should have a histogram of durations.
- arcticfox 10y ago
  That distinction makes all the difference to me. IMO one answer is fraud and the other is borderline valid / no big deal.
- dismantlethesun 10y ago
  It looks like they didn't count it in either metric. Peoples' complaint is that they excluded it from all averages and all reports, which inflates their average duration of video watched because videos less than 3 seconds aren't included at all.
- prostoalex 10y ago
  > If they counted those sub 3 sec views in the count but not in the duration It sounds like this was exactly what was happening. From the article: "Video views of under three seconds were not factored in, thereby inflating the average. Video views of under three seconds were not factored in, thereby inflating the average. Facebook’s new metric, “Average Watch Time”, will reflect video views of any duration."
nol13 10y ago
See just get rid of the auto-playing videos problem solved, and make the world a better place while you're at it.
- colejohnson66 10y ago
  I bet Facebook gets a decent enough amount from people who scroll, look away for a second, video starts playing, then user pauses video, but not before the 3 second mark. What really should be done is getting rid of auto playing videos at the HTML spec level. While there are legitimate uses for autoplaying videos (YouTube for example), it has too much potential for abuse by ad networks.
  - cpeterso 10y ago
    Advertisers will find a workaround and it will inevitably be much worse for the user. In the following talk, someone from the Google Chrome team describes how, when they tried to not autoplay videos on Android, video advertisers worked around the restriction by compiling video codecs to JavaScript and then rendering their video-like ads to an HTML canvas. https://youtu.be/wQGa_6CRc9I?t=23m52s https://youtu.be/wQGa_6CRc9I?t=23m52s
    - colejohnson66 10y ago
      That's where NoScript comes in
      - NoGravitas 10y ago
        Or blocking 3rd party scrips with uBlock.
    - raverbashing 10y ago
      Don't ad networks charge for ad size as well? (or they should start charging for this - would cut on abuse) If your ad is > 50k then you're paying a fee per display, even if you're "pay per click"
  - 10y ago
mod 10y ago
On my programmer hand, if I made the cutoff 3 seconds for the metrics, I would make it 3 seconds to count a view. If the metrics and the view counts match, I don't see a problem. If they calculate the average for > 3 seconds, but count EVERY view, I agree--fraudish.
- AstralStorm 10y ago
  This sort of Windsorizing will inflate the average anyway. Making decisions based on "average" view time is still silly.
  - luiscarloscb 10y ago
    Why would it be silly? If I have an ad that has double the average view time than other wouldn't it make sense to invest more in it and less in the second one?
    - LeifCarrotson 10y ago
      The distribution is what they really need to include. The average view time doesn't mean much when the distribution is super skewed, which it's likely to be for these ads. If 0.1% watch all 30 seconds, 0.9% watch to 28 seconds and quit at the credits, and the rest all watch 0.2 seconds as they scroll through, that tells you something that "Average watch time = 0.5 seconds" doesn't tell you. Especially if last week, no one watched the whole thing, but a bunch of people watched for 2 seconds and then quit, and your average was 0.9 seconds.

bonniemuffin 10y ago

I don't think average watch time is a particularly useful metric, because no matter how they count views, there's some kind of arbitrary cutoff. If they count a view as soon as the player finishes loading, they'll include a whole bunch of people who didn't intend to watch the video, and in fact only "watched" a few milliseconds of it before bouncing. So that will drag the metric down artificially. I bet if they count views as soon as the video starts playing, they could drag down average watch time by improving player load time, because they'd count more unintentional "views" of very short duration. I hope/assume this is part of a suite of metrics that they provide to advertisers, so you can understand it in the context of other things like video completion rate or counts of views that reach X% through the video.

breatheoften 10y ago

What was the actual bug? Does this mean that no video had an average view time less than 3 seconds?

sharkmerry 10y ago
They dont count views for less than 3 seconds in their view count. They didnt include views of less than 3 seconds in their "average view time" metric, which is very misleading. Their metric was "average view time of video's in our view count". Which is not the same thing and the latter vastly inflates the effectiveness of facebook video

emcq 10y ago

This seems slightly overblown. I suspect this was a metric useful for engineering that found its way to external usage. That's because in many recommendation systems in practice you want to filter out the spurious views, and a simple way to do that is with dwell time. It's not uncommon to require at least 50% of pixels in view for 1 second before you have an impression for static images [0, 1]. AOL defines an impression requiring 2 seconds for images [1]. Facebook likely did some analyses to find that 3 seconds was a good cutoff for their site. There are more sophisticated ways to estimate dwell time but they seem uncommon in practice; perhaps due to their difficulty communicating to advertisers what the impression metric actually means. For sites with many bots or inbound marketing you often find users bounce quickly which drives some of this timing. I'm a bit surprised it needs to be that high for Facebook without many bots or users bouncing quickly. Perhaps this is for mobile users scrolling quickly. [0] http://advertising.aol.com/specs/terms/aol-viewability-terms http://advertising.aol.com/specs/terms/aol-viewability-terms [1] http://mediaratingcouncil.org/063014%20Viewable%20Ad%20Impression%20Guideline_Final.pdf http://mediaratingcouncil.org/063014%20Viewable%20Ad%20Impre...

ma2rten 10y ago
I thought that this metric is supposed to measure the quality of the views that the advertiser is paying for. In that case maybe advertisers shouldn't pay for views under 3 seconds, but if they do it should count towards the average.
- jdoliner 10y ago
  Based on the stat in the article about it resulting in 60% to 80% difference in total viewing time, I think they were probably reporting total views including views under 3 seconds and reporting avg view length discounting views under 3 seconds. This results in a pretty misleading viewing time. Still quite possibly a mistake, but I can see why advertisers would be pissed.
- honkhonkpants 10y ago
  Isn't it the advertisers' job to measure the quality of the channel? Or does this kind of advertising have an intangible outcome like brand awareness?

nostrademons 10y ago

My data-weenie hat tells me that this is a silly metric anyway. As an advertiser, you really care about the distribution of viewing times, and in particular what fraction of viewers watch the video all or most of the way through. Arithmetic mean is virtually useless when the data has a power-law or other non-linear distribution, and it's highly likely video viewing times exhibits this.

sharkmerry 10y ago
Doesnt this vastly affect the distribution though? It excludes all views less than 3 seconds.
- pg_is_a_butt 10y ago
  you mean the views that don't count at all? why would you want metrics about non-views to pull down the average view time? shut facebook down. you're all idiots.
teej 10y ago
As someone who helps make decisions around ad spend... not really. I would take a distribution if I was given it but it wouldn't substantially drive my advertising strategy. Views (over 3 sec views is fine), clicks, viewthrough conversion, clickthrough rate, and conversion rate are orders of magnitude higher signal measures than the specifics of how far they got through the video. I'd imagine that view time distributions are way more important to new media companies that rely heavily on video media like Buzzfeed or anyone putting out content on Youtube.
- cwilkes 10y ago
  What's the clickthru rate on a movie trailer? What is that going to tell you? Do I click on ads on TV or in the newspaper? Most video ads I see are more informational than a call to action. Who really wants to click on an ad while in facebook on a phone?
  - 10y ago

pducks32 10y ago

I don't believe for a second that this was unintentional. The amount of press and mind share that Facebook has received for their "rocketing video efforts that rival YouTube" was totally worth pissing off some ad exec. Very few people will see this news compared to "Facebook dominates video" headlines that have been everywhere. This was a smart play on their part.

donohoe 10y ago
"Never attribute to malice that which can be adequately explained by stupidity." I can see exactly how things like this could happen in large corps. You'd be surprised.
- reitanqild 10y ago
  This might not be totally faor towards Facebook, but: When it comes to Facebook I guess there are slightly different rules: A good rule of thumb for Facebook might be: expect malice. We have seen it with the way the made everything public. We have seen how they decided to sell out its own user a year after proclaiming their love and how this wouldn't happen.
- thr0waway1239 10y ago
  While your point is very fair, a long track record of behavior which pushes the boundaries of ethics becomes indistinguishable from malice. In these matters, people always judge you more harshly if you have taken advantage of their trust (many many times in FB's case) in the past. Facebook wants to have the cake (the hype surrounding its success to be a self-perpetuating thing) and simultaneously eat it too (always expecting the benefit of doubt around its integrity around its self-reported success metrics). While I welcome people with means doing philanthropy, I can't help but wonder if the recent announcement of philanthropy from Mark Z is an attempt to generate good press. If Mark Z is so keen on doing charity, perhaps he should divert all that money to the top 10 privacy watchdogs in the world. In today's world, that is as much a just cause as any other, particularly because no one really wants to talk about it. Note: I am not saying the other tech giants don't indulge in these practices. Two wrongs can't make a right.

aaron695 10y ago

I don't get it, it makes sense? Less than 3 seconds is not a view any more than seeing the picture of the video is a view. It doesn't count I'd say? Or do advertisers have to still pay for a 2 second view?

bluetwo 10y ago

"likely overestimated average time spent watching videos by between 60% and 80%" Soooooooo... fraud? They say it didn't impact billings/revenue, but I have to imagine they will now be in a position to give discounts, if not refunds.

sean_patel 10y ago
Yes. straight up fraud. "How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q https://www.youtube.com/watch?v=t7tA3NNKF0Q
- bbctol 10y ago
  That video is discussing a slightly different problem than the recently discovered issue the article's talking about.

jtchang 10y ago

I don't think this is overblown at all. I expect to see a class action lawsuit because of this. If you are a big advertiser and you can somehow show damages because of this error there may be a case. There are some adtech companies recently who focus purely on reporting the metrics of your ad purchases. Kind of like acting as a third party. I forgot exactly what they are called.

shostack 10y ago
This assumes most advertisers can prove the value of a video view of a certain length. I assure you as someone who deals with attribution on a daily basis that most cannot and the ones that claim to would have a hard time proving it in court. Attribution is hard.

sean_patel 10y ago

"How Facebook is Stealing Billions of Views" - Kurzgesagt – In a Nutshell exposed this with detailed analysis and evidence. Watch it here => https://www.youtube.com/watch?v=t7tA3NNKF0Q https://www.youtube.com/watch?v=t7tA3NNKF0Q This is neither a bug nor a "mistake". It's straight up fraud. Fraud because Facebook uses these false #s to prove to their advertizing clients that they are getting a HUGE ROI, when they really aren't. Also the investors and analysts bump up their ratings and the stock goes higher and higher on false ad view #s. By counting a view as a legitmate view after the video plays only for 3 seconds, their "algorithm" counted billions of views even when the user has not even seen it, because Auto-play is enabled by default and you have to opt-out / disable it and no-one does it. It takes a person roughly 4 seconds to scroll off their feed as they quickly "scan" their friends posts.

yladiz 10y ago
Do you have a citation for the "4 seconds" figure? If it is 4 seconds, and it logs a view after 3 seconds, that's pretty damning.
- AstralStorm 10y ago
  That it logs invisible videos at all would be a problem.
- sean_patel 10y ago
  It's not publicly discussed for obvious reasons, but I worked closely with Mudd for a couple of months last year. Here's a related interview where he touches upon it. http://digiday.com/platforms/advertisingweek2015-facebook-finds-that-fast-scrolling-millennials-consume-ads-2-5-times-faster/ http://digiday.com/platforms/advertisingweek2015-facebook-fi... Facebook spend an awful lot of resources tracking all sorts of user behaviour inside it's "walled garden". I got really sick of it after a while and decided to stop working for them (even as a contractor)
hanspeter 10y ago
ROI from FB ads can only be calculated from tracking conversions on the advertiser's site or in their app. View count alone can never be attributed a ROI as there is no measurable return. If you pay solely for video views on Facebook or Youtube without any tracking, you're not being a smart advertiser and it doesn't really matter if view counts are more or less correct - you're blindly throwing your money away. If you do however track your ROI correctly it doesn't really matter how many true view counts you get.

throwanem 10y ago

It's like watching a wasp land on a nettle. You know somebody's going to get stung, but you just don't care who.

yueq 10y ago

Will there be a class action on this matter?

restlessdesign 10y ago

It is unreasonable, even for a developer, to count 3 seconds as a play. Anyone working on a video product would know they were inflating the numbers.

prostoalex 10y ago
The top-voted comment on this thread, by emcq, has a link to AOL's viewability guidelines which states "For purposes of Video inventory, an impression is considered “In-View” to a user when at least fifty percent (50%) of the pixels in the ad are in the viewable browser window for a minimum of two (2) consecutive seconds." Google here https://support.google.com/dfp_premium/answer/4574077?hl=en https://support.google.com/dfp_premium/answer/4574077?hl=en says "For in-stream video ads, 50% of the ad’s pixels must be visible in the browser window for a continuous 2 seconds."
- swish_bob 10y ago
  That's the IAB's standard viewable impression metric: https://www.iab.com/wp-content/uploads/2015/06/MRC-Viewable-Ad-Impression-Measurement-Guideline.pdf https://www.iab.com/wp-content/uploads/2015/06/MRC-Viewable-...

tn13 10y ago

It might be inaccurate but does not matter much in advertiser perspective. Given the conversions etc. eCPMs conversion to the real expected value.

bruinbread 10y ago

This isn't anything new. Several YouTubers have "exposed" how Facebook tracks views. It's not accurate, but it's the system that has been in place for a while and they've defended it as such.

smithunsero 10y ago

In 2012 there were reports that 80% of traffic from Facebook Ads were automatic bots. So this is not the first time Facebook use tricks to get more money from clients.

plopilop 10y ago
From what I read [1] it's only one startup that claimed this, and I failed to see any confirmation from any other company. For me, no replication from companies that spend thousands every month on advertising = the startup messed up with its stat tools or wanted to create a buzz around its name. [1]: https://techcrunch.com/2012/07/30/startup-claims-80-of-its-facebook-ad-clicks-are-coming-from-bots/ https://techcrunch.com/2012/07/30/startup-claims-80-of-its-f...
hanspeter 10y ago
Running campaigns on Facebook's advertising platform does seem to attract bots. It's not 80 % though and it's obviously not Facebook doing it. Why would they sabotage their own business?

Arkaad 10y ago

I am disappointed by FB. Next we'll learn that they do tax evasion.

supermatt 10y ago

If face book only count a view as over 3 seconds, and they measure the average view as only those videos over 3 seconds, what is the problem? Surely now its less representative?

coldtea 10y ago

>the tech giant vastly overestimated average viewing time for video ads on its platform for two years, according to people familiar with the situation They didn't "overestimate". They plainly gave false numbers to drive ad sales. Coldtea's law: Never attribute to incompetence what can be explained by profit.

meira 10y ago
I'd call it SV's Law, but you can get the merits of it :)
mtgx 10y ago
Same goes for Facebook's user tracking "bugs".
jbooth 10y ago
I'd qualify that. Things that appear to be generating a ton of money don't get looked at that closely, because everything's going great. Things creating losses get looked at much faster. So it's not pure deceptiveness. Probably nobody made the call "let's defraud everyone". However, a bug that undercounted the viewing time would have been uncovered much sooner.
jbverschoor 10y ago
+ there's a reason videos auto-play....
billmalarky 10y ago
"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!" -Upton Sinclair
- jgalt212 10y ago
  That is basically how Wall Street works. If a trader or salesperson is printing money, management doesn't want to know how it's done. Plausible deniability will at worst get you a failure to supervise penalty, whereas if you understand how these folks are making money you could become complicit in any illicit acts and thus exposure yourself to a much larger penalty and/or jail time. Of course, the government has proven themselves time and time again unwilling to punish management so management might as well figure out how the illicit gains are being made so they can replicate such processes when the big producing trader or salesperson is invariably poached by a competing shop.

shostack 10y ago

While this is concerning, smart advertisers were looking at the distribution of view length percentages anyway as you can easily pull those columns in and their derived metrics. Likewise, smart advertisers look at lift in conversion metrics when possible, in which case this stat is irrelevant. That said, FB has not exactly helped things by making metric definitions a little obfuscated in general. Personally, I see a bigger concern is them giving 100% view through conversion credit with a 1 day window by default as part of any website conversion action tracking. There are very few cases (like some retargeting situations) where you'd ever want to give full weighting for a VT, and while there is likely value in VT's, I probably wouldn't give 100% credit to them by default given the rate at which people scroll on mobile. But advertisers like to see big numbers, agencies like to show big numbers, and so you have platforms like FB aggressively try to push metrics like this and some of their rather loose definitions of "engagement" without great explanation of the nuances or pros and cons. These are largely left up to the advertiser to determine since, to be fair, they are very subjective. Savvy buyers know this and configure their reporting and tracking settings accordingly because FB and PMDs give you those options. They are sometimes just buried. Ultimately, IMHO FB and Google's greatest defense towards any of these sorts of claims is better and (more importantly) transparent attribution data and tools. If they can prove their value on the bottom line, other things often don't matter to many advertisers. Attribution is a tough nut to crack, but for advertisers spending large sums, it is critical to be successful in these channels.

laurihy 10y ago

Then again, if a "video view" is defined as "watched over 3 seconds (50% of the video visible in the screen, IIRC)", then it sort of makes sense that "Average Duration of Video Viewed" doesn't include non-views. For sure Facebook should attempt to name their metrics as descriptively as possible, but also advertisers should make sure they understand how different conversions are measured, what's included and what's not. Another example would be "Clicks" metric, which included all engagement (i.e. likes, shares etc), instead of just "Link clicks".

TazeTSchnitzel 10y ago
It doesn't seem all that unreasonable, yeah. If a user scrolls right past a video, should that really count towards the average?
- RodericDay 10y ago
  A user below summarized the issue the way I understand it: > what happens is that views under 3 seconds aren't counted towards the average. So if you have 999 999 views of <1 second each and 1 view of 3 seconds, the reported average is 3 seconds / 1 view = average view time of 3 seconds, while the true average is something like 500 000 seconds / 1 000 000 views. This is massive fraud, especially if you're shouting from the rooftops "Look at how much more attention people are paying to your videos in our platform!".
Naritai 10y ago
I thought about this as well, and concluded that FB must be reporting 'video views' as any time a video starts autoplaying (or something similar). In this way I could see the total estimated viewing time getting significantly overestimated.

mikejb 10y ago

I didn't fully understand what the bug was, but I found on a German news page[1] with a slightly more detailed description: Aufgezeichnet wurde zwar die Gesamtsehdauer aller User, geteilt wurde dieser Wert allerdings nur durch diejenigen User, die das Video länger als drei Sekunden ansahen So the average was not calculated correctly: They accumulated the duration of all video views (including of those shorter than 3 seconds), but divided it by the number of 'legitmate' views - i.e. only those longer than 3 seconds. So you get a pretty big offset if you have many views under 3 seconds (which they probably do, thanks to autoplay). [1] https://www.heise.de/newsticker/meldung/Facebook-Unklare-Video-Metrik-fuehrt-Werbekunden-aufs-Glatteis-3330188.html https://www.heise.de/newsticker/meldung/Facebook-Unklare-Vid...

Bedon292 10y ago

Ok, what am I not understanding here? If they are only counting a view as over 3 seconds. And then only averaging the those views what is the problem? They tell me 1000 people viewed it (over 3 seconds), and then say the average view time was 10 seconds for those 1000 users. How is this an issue? Why do I care that 9,000 people scrolled past it and weren't counted in the viewer numbers? Now if they were reporting this as 10,000 viewers and saying 10 second average, I see the problem. Is that what was happening? Or what am I missing?

oneloop 10y ago
My suspicion (based on nothing) is that they averaged viewing over viewings with length >3 seconds, but _counted_ all views. I.e. suppose your ad was displayed 5 times for 0.1 seconds and 5 times for 5 seconds. They would report this as "10 views, with average viewing time of 5 seconds".
- tedmiston 10y ago
  I wish your comment was at the highest level. Everyone seems to be attempting to make the same point in the other subthreads. The example here makes it crystal clear.
  - oneloop 10y ago
    My guess was almost correct. The issue was indeed that they use "views with >3 secs" in some metrics and use "all views" in other metrics. Do one of the other, don't mix them up. http://www.adweek.com/news/technology/facebook-gave-ad-agencies-inflated-video-viewing-metric-2-years-173676 http://www.adweek.com/news/technology/facebook-gave-ad-agenc... > UPDATE: Facebook authored a blog post today to futher explain the metric-based miscue. David Fischer, vp of advertising and global operations, wrote: "About a month ago, we found an error in the way we calculate one of the video metrics on our dashboard – average duration of video viewed. The metric should have reflected the total time spent watching a video divided by the total number of people who played the video. But it didn't – it reflected the total time spent watching a video divided by only the number of 'views' of a video (that is, when the video was watched for three or more seconds). And so the miscalculation overstated this metric. While this is only one of the many metrics marketers look at, we take any mistake seriously."

randomgyatwork 10y ago

When will Facebook 'content creators'(users) get compensated for the value they create?

malloreon 10y ago

how much of their video income will they be refunding to advertisers?

sidcool 10y ago

This looks like a flat out lie. Startups have to sometimes do that to woo potential investors. Although I like Facebook overall, I do not agree with many of their business practices.