3 ms·
That's right, it should be noted however that the need of horizontally scaling the equivalent of the NameNode kicks in only when you really have a very large st
by ithkuil 12y ago
That's right, it should be noted however that the need of horizontally scaling the equivalent of the NameNode kicks in only when you really have a very large storage system, where large can be defined as:
"You know you have a large storage
system when you get paged at 1 AM because you only have a
few petabytes of storage left." [1]
Even if you are below that size you might have a large system, and even if using a single metadata master node might be a sound solution you still have lot of interesting problems to solve. Don't make systems more complex than necessary.
More pointers to google public material on colossus in [2].
[1] http://static.googleusercontent.com/media/research.google.com/it//university/relations/facultysummit2010/storage_architecture_and_challenges.pdf http://static.googleusercontent.com/media/research.google.co...
[2] http://www.highlyscalablesystems.com/3202/colossus-successor-to-google-file-system-gfs/ http://www.highlyscalablesystems.com/3202/colossus-successor...