2 ms·
> In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would c
by lossolo 4y ago
> In GFS, on the other hand, individual files are sharded between servers on a chunk level (where 1 block==64MiB IIRC). To access a file's contents, you would contact the master, ask it for the chunkservers for a file, and it would tell you which set of chunkservers were responsible for each chunk in the file; you'd then contact the chunkserver to access the data.
> The impact of this is that the latency of accessing a file was much higher on GFS; you needed to go through the master (which tended to be a bottleneck) to get the metadata, and then establish a new connection with authorization to each chunkserver to access the data. However, because each file was stored by multiple machines and the data location was independent of the position in the hierarchy, there was very little risk of chunkservers limiting the transfer speed.
Sound similar to BitTorrent design.