7 ms·
Looking back on “Look Back” videos
- notastartup 13y agoThey should make a video on how they served a billion Facebook likes and where they came from for the people who bought them.
- jzwinck 13y agoI assumed the videos were created lazily, on first view. This article makes it pretty clear that they decided from the beginning that they needed to render all the videos in advance. I wonder how many videos they rendered which were never viewed, and whether rendering on first demand would have been viable.
- aragot 13y agoI was also expecting amazing optimization tricks, like: - Designing an MPEG encoder which can take a couple of pictures and render them on the fly, so only saving the pictures, - Carefully spreading the views over a period of time, - Associating the soundtrack at view time, since the soundtrack is the same for everyone, - Using EC2 for the purpose, deleting all vids afterwards, But no, they just used the brute force method. It would be a good job interview question: "How would you serve 700m videos?"
- ruchirablog 13y ago- Using EC2 for the purpose, deleting all vids afterwards Facebook has their own infrastructure and very well capable of hosting their own. And of course deleting the videos isn't a smart idea
- frostmatthew 13y ago> Using EC2 for the purpose, deleting all vids afterwards Facebook content isn't expected to disappear at some later point in time.
- yoavz 13y agoIt unbelievable to think about how much synchronization must have gone into this project for all the pieces to come together successfully. Just some more evidence that goes to show facebook has some of the best engineering teams in the world.
- jw2013 13y agoIs there really the need to use 80 people for this feature? Why it seems to me that is just because of excessive programming hours in total available at Facebook rather than the real need for this new feature. Just curious.
- xixixao 13y agoThere are no excessive programming hours at Facebook. I assume most of those people worked on the massive backend challenge, not on making the videos (but that's my guess).
- jw2013 13y agoI understand that the difficulty of making this new feature is scaling the backend, but at this point wouldn't Facebook already have sort of some processes or "frameworks" to follow when making a new feature.
- shuw 13y agoAbout 3-10 people worked on this full time depending on which day in the 3 week development cycle you looked at. Most of the others helped out in their spare time in one way or another
- vjeux 13y agoTo get a sense of scale, the largest DDoS ever recorded by Cloudflare is 400Gb/s[1]. Lookback video was 450Gb/s. [1] http://arstechnica.com/security/2014/02/biggest-ddos-ever-aimed-at-cloudflares-content-delivery-network/ http://arstechnica.com/security/2014/02/biggest-ddos-ever-ai...
- stedaniels 13y agoReally annoyingly, Facebook's selection of photos was way off for me. I've been married over 3 years, with my wife for over 4.5 years, yet it still chose to include photos of ex girlfriends. Facebook has all the data it needs to discern that was a bad idea. Quite an oversight in my opinion as I know several people who it affected too so they didn't publish the video.
- NZ_Matt 13y agoI've think they've updated it since then. You can now go through and manually select the photos and statuses that you want to include in the video.
- kawsper 13y agoEx-girlfriends are part of your past whether you like it or not. I can speak with most of my ex-girlfriends, and it was a nice trip down memory-lane to watch the pictures of me and them. I am not running from my past, and I appreciate the time that we had together.
- wavefunction 13y agoThat's a nice sentiment but do you think your wife or their new beaus feel similarly? What exactly is the value of the "social graph" if it is never applied by the one company in the prime position to do so?
- adyus 13y agoA HN-specific analogy would be: would you work for a startup who can't accept what you did prior to employment? A wife or new beau who can't accept that you had a past may not be a good idea for the future.
- marcoagner 13y agotouche
- 13y ago
- asharpe 13y agoWhile I understand they want their own server farms, it is interesting that Facebook does not use any cloud compute. This is almost the perfect usage example and pattern for cloud compute.
- CSDude 13y agoIf you mean cloud providers, it would be very expensive for Facebook scale, cloud is not that cheap, having own infrastructure would be more efficient
- randartie 13y agoFacebook has some of the largest data-centers in the world, with plenty of spare capacity. In the article they describe that running these HUGE jobs put them at risk of not potentially not being able to provide power to all the machines at once. If you're thinking they could have just started up a ec2 job to process a billion videos, I really doubt there's any cloud solution in the world right now that can handle this scale without capacity planning before hand.
- mcintyre1994 13y agoIt's easy to forget given some of the press they get just how active their users are, 720m users who care enough about your product to watch a video about how they use it must be incredible. The algorithm seemed fairly naive but it certainly seemed to work well for some I saw, and really impressive engineering too.
- personlurking 13y ago> 720m users who care enough about your product With the product (and the videos) being (about) the users themselves, it starts to make sense.
- qnk 13y agoThe people from Statigram[1] came up with this idea last year. I have always been interested in how they did it. It wasn't the same scale as Facebook, but I assume it was a big challenge for them, especially not having the resources. The feature consisted in generating a 15 second video with your most liked Instagram photos and sending it to you as and MPEG email attachments. They had to do this because third party apps cannot post to Instagram. I initiated my video generation probably at their peak, and still, in less than five minutes, the generated video was in my inbox. If someone from Statigram is reading this, I would really love to hear your side of the story, being able to compare it to facebook's approach now will make it even more interesting to me. Here's an example of the generated video: http://instagram.com/p/ibh7zXmvqm/ http://instagram.com/p/ibh7zXmvqm/ [1] http://statigr.am http://statigr.am
- chengiz 13y ago> We had anticipated that about 10% of the people who saw their video would share it. Instead, to our surprise, well over 40% of the viewers were sharing. That 10% estimate seems way too low to me. If something like this catches on, it catches on big time. If they didnt expect it to catch on, then why even bother. That said, I never had a problem with videos loading etc during this look back craze. And hardly ever any other time for that matter - it is remarkable how stable that site is.
- harmegido 13y agoThat estimate might be a tacit admission to the problem of all the fake accounts out there.
- randartie 13y agoIf the estimate was for people who would see the video, why would fake accounts see the video?
- rtpg 13y ago10% of the FB population is still enough for it to "catch on", surely.
- jbverschoor 13y agoWhy didn't they just create an html version of this? No need to render a gazillion videos
- sjtrny 13y agoProbably to ensure compatibility for old browsers and mobile users. Obviously the more elegant solution is to do the animation client side and avoid rendering millions of videos but it's not the solution that would have worked for the most people.
- Achshar 13y agoConsidering how facebook has more than a billion users from all the corners of the world accessing it from literally every internet connected device, they had to make something that would work everywhere. Doing it the video way seems like a good decision in this case, at the scale of facebook.
- steadicat 13y agoThey did that as well: https://www.facebook.com/lookback/edit/ https://www.facebook.com/lookback/edit/
- junto 13y ago- Obtain enough computing capacity - Stay within the electrical power limits of our data centers Somehow it feels like the world is so backwards. Here you have all these smart hard working people with billions of dollars of resources at their disposal and they made a load of kooky videos that people could 'like'. The videos I'm sure are cool, and I realise that it was quite an achievement, but still, our priorities as a Hunan race are pretty much fucked up.
- grinich 13y agoWhat are the priorities of the Hunan race?
- rakoo 13y agoAn impressive engineering task, for sure. But this shows how far we are from the fundamental ideas of the internet and the web, and instead fall back to the good ol' feudalism.
- mahouse 13y agoLooks like that Schneier article made a dent...
- rakoo 13y agoIn France we had that thing called Minitel [0], which was heavily widespread before the WWW took off (I think it was given to everyone or something like that). The economic (and technical) model was completely different than the internet: you had a passive, stupid terminal, and you connected to a server that offers services for a fee. So you paid for connecting and using a service that could technically not be done on your own minitel (at least in the earlier version). Just like a browser that is completely stupid and allows you to connect to Facebook, Twitter, Google et al. for things you can't do on your own machine. You were depending on a third-party to do things. We speak a lot about the Minitel model and how it is after all these years a model for all the big WWW players and ISPs to monetize their services. But it's not a common reference, so 'feudalism' works well in this context. [0] http://www.reddit.com/r/netsec/comments/1shi77/textsecure_now_with_10_million_more_users/ce1uwc1 http://www.reddit.com/r/netsec/comments/1shi77/textsecure_no...