3 ms·
This is interesting, especially the native windows client, but it's based on the same tired architecture as HDFS, and has all the same limitations. 1) Single M
by GauntletWizard 11y ago
This is interesting, especially the native windows client, but it's based on the same tired architecture as HDFS, and has all the same limitations.
1) Single Master for Metadata
One of the biggest weaknesses in this model is that you can only scale the master so far. For metadata responses to be snappy, everything in your working set has to fit in RAM. That imposes hard limits on your number of files; 100M is achievable, 1B can even be done, but more than that and you're in for hurt.
2) Chunkserver to chunkserver replication
As the moosefs write diagram shows, you're waiting for chunkservers to walk a linear tree, and you get the slowest performance of each one. Which, as it turns out, is on average pretty bad. If you have a 99th percentile latency of 10x your average, you lose 20% of your speed with three replicas. Not to mention the fact that your 'single point of failure' is actually any of the three of those chunkservers, with a long retry stall if one of them errors.
LizardFS supports a xor mode, so it may be avoiding this by having clients talk to more than one chunkserver directly. Combined with proper queueing and retry logic, this reduces the effect of the long tail significantly.