4 ms·

How Facebook keeps 100 petabytes of Hadoop data online

• mpim 14y ago
  Probably better to link to the original article instead of a summary: https://www.facebook.com/notes/facebook-engineering/n/10150888759153920 https://www.facebook.com/notes/facebook-engineering/n/101508...
• halayli 14y ago
  This gigaom article is horrible.
• ajays 14y ago
  It's not 100PB of data; it's 100PB of physical disk space. Given the (standard) 3x replication and filesystem overhead, you're looking at about 30PB of data. Certainly a ginormous amount of data, that's for sure.
  • deleted 14y ago
    [deleted]
• dude_abides 14y ago
  If only the secondary name node did what one would assume a secondary name node is supposed to do, Facebook wouldn't have needed to invent avatar nodes, and even Hitler wouldn't have gotten so upset and fired that intern: http://www.youtube.com/watch?v=hEqQMLSXQlY http://www.youtube.com/watch?v=hEqQMLSXQlY
• matan_a 14y ago
  This doesn't actually discuss how Facebook manages and maintains their 100PB of storage, rather just talks a bit about the namenode SPoF issue. Disappointing article for sure.