3 ms·
It's been a very long time since I read the GFS paper but it's similar in many ways - there are only so many ways to design a large-scale blob store and they al
by james_cowling 10y ago
It's been a very long time since I read the GFS paper but it's similar in many ways - there are only so many ways to design a large-scale blob store and they all share a lot of the same design elements.
If I remember correctly GFS has a coordinator node in the critical path, much like the name node in HDFS, which impacts availability if it goes down. The nice thing about the Master in MP is that it's completely disjoint from the data-plane and not a critical component in the system.
Your assessment is pretty accurate, and the cross-zone replication component is one distinguishing characteristic. MP also makes heavy use of highly-efficient erasure coding as a first-class design element, which wasn't included in the original implementation of GFS but was added later.
The biggest difference from an API perspective is that MP is only a blob store whereas GFS is an actual filesystem (we implement the filesystem at a higher abstraction layer). From what I understand MP is closer in spirit to the underpinnings of Colossus than GFS.
- sroussey 10y agoDo you really create a filesystem? A mountable one? I can't see why you would want that kind of overhead. (I don't mean the mount you provide to users for their files.)