5 ms·
The two are very different technologies. In AFS, the fundamental unit of sharding is the volume, each volume is controlled by a single volser, and the "master"
by thequux 4y ago
The two are very different technologies. In AFS, the fundamental unit of sharding is the volume, each volume is controlled by a single volser, and the "master" (i.e. vldb) is responsible for resolving volume IDs to the set of volsers that hold a copy (at most one of which is read/write)
In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data.
The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed.
With AFS, on the other hand, the master is much less of a bottleneck as it's only used when mounting volumes. Further, once a volume is mounted, accessing files and data within those files could be done in a single round trip. However, if you have a volume with high write traffic, that will end up bottlenecking on a single server's IO speed.
- lossolo 4y ago> In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data. > The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed. Sound similar to BitTorrent design.