4 ms·

High Performance at Massive Scale: Lessons Learned at Facebook

66 points by niels 17y ago

davi 17y ago
More on Haystack: http://www.facebook.com/note.php?note_id=76191543919 http://www.facebook.com/note.php?note_id=76191543919
- xal 17y ago
  The article mentions that Haystack has been open sourced but that doesn't seem to be the case yet.
  - quilby 17y ago
    A few days ago I sent a message to the FB developer that wrote the blog post about Haystack. He told me that they are working on getting it open sourced and that it will be ready 'soon'.
andrewcooke 17y ago
wow. they have no (ie random) partitioning - it's just brute force. i hope their "Graph-based caching and storage systems" is coming along... :o) [edit: although, given the exponential growth they give and the factor of 4 headroom they have left, they're going to top out in about 1.5 years anyway]
- niels 17y ago
  Yep, so basically one request can touch hundreds of servers. It's basically the opposite approach of Twitter. They aggregate the graphs on write.
  - niels 17y ago
    Which also is the reason you can only have 5000 friends on FB. If you had a million friends, you'd probably touch all their servers for one request.
  - aaronblohowiak 17y ago
    Who is "They" in this case? Both a read and a write could be a "request", so I am having some trouble with the ambiguity of your last sentence. Thanks!
    - niels 17y ago
      Twitter is they.
praxxis 17y ago
An impressive statistic: "In aggregate, Memcache at Facebook processes in 120M requests/sec."
kssreeram 17y ago
Interesting quote from the article: "Jeff also relayed an interesting philosophy from Mark Zuckerberg: 'Work fast and don’t be afraid to break things.' Overall, the idea to avoid working cautiously the entire year, delivering rock-solid code, but not much of it. A corollary: if you take the entire site down, it’s not the end of your career."
- codexon 17y ago
  And apparently, don't worry about violating your user's data privacy! I felt compelled to write this since I just received the Facebook Beacon class action lawsuit email a couple of minutes ago. Why the downvotes?

66 points by niels 17y ago

davi 17y ago

More on Haystack: http://www.facebook.com/note.php?note_id=76191543919 http://www.facebook.com/note.php?note_id=76191543919

xal 17y ago
The article mentions that Haystack has been open sourced but that doesn't seem to be the case yet.
- quilby 17y ago
  A few days ago I sent a message to the FB developer that wrote the blog post about Haystack. He told me that they are working on getting it open sourced and that it will be ready 'soon'.

andrewcooke 17y ago

wow. they have no (ie random) partitioning - it's just brute force. i hope their "Graph-based caching and storage systems" is coming along... :o) [edit: although, given the exponential growth they give and the factor of 4 headroom they have left, they're going to top out in about 1.5 years anyway]

niels 17y ago
Yep, so basically one request can touch hundreds of servers. It's basically the opposite approach of Twitter. They aggregate the graphs on write.
- niels 17y ago
  Which also is the reason you can only have 5000 friends on FB. If you had a million friends, you'd probably touch all their servers for one request.
- aaronblohowiak 17y ago
  Who is "They" in this case? Both a read and a write could be a "request", so I am having some trouble with the ambiguity of your last sentence. Thanks!
  - niels 17y ago
    Twitter is they.

praxxis 17y ago

An impressive statistic: "In aggregate, Memcache at Facebook processes in 120M requests/sec."

kssreeram 17y ago

Interesting quote from the article: "Jeff also relayed an interesting philosophy from Mark Zuckerberg: 'Work fast and don’t be afraid to break things.' Overall, the idea to avoid working cautiously the entire year, delivering rock-solid code, but not much of it. A corollary: if you take the entire site down, it’s not the end of your career."

codexon 17y ago
And apparently, don't worry about violating your user's data privacy! I felt compelled to write this since I just received the Facebook Beacon class action lawsuit email a couple of minutes ago. Why the downvotes?