3 ms·
I'm not an expert in this area, but at a glance, an MP cell looks a lot like a GFS cluster, right? It seems like the big differences are that MP is designed for
by jkaptur 10y ago
I'm not an expert in this area, but at a glance, an MP cell looks a lot like a GFS cluster, right? It seems like the big differences are that MP is designed for
a) a huge number of smallish files, accessed by a large number of clients
b) file writes, rather than appends
c) cross-cell, cross-zone replication
Is this impression correct, or am I missing other major design differences?
- james_cowling 10y agoIt's been a very long time since I read the GFS paper but it's similar in many ways - there are only so many ways to design a large-scale blob store and they all share a lot of the same design elements. If I remember correctly GFS has a coordinator node in the critical path, much like the name node in HDFS, which impacts availability if it goes down. The nice thing about the Master in MP is that it's completely disjoint from the data-plane and not a critical component in the system. Your assessment is pretty accurate, and the cross-zone replication component is one distinguishing characteristic. MP also makes heavy use of highly-efficient erasure coding as a first-class design element, which wasn't included in the original implementation of GFS but was added later. The biggest difference from an API perspective is that MP is only a blob store whereas GFS is an actual filesystem (we implement the filesystem at a higher abstraction layer). From what I understand MP is closer in spirit to the underpinnings of Colossus than GFS.
- sroussey 10y agoDo you really create a filesystem? A mountable one? I can't see why you would want that kind of overhead. (I don't mean the mount you provide to users for their files.)