4 ms·

Yes, I had forgotten that they striped their data across multiple pods... I'm not familiar with how each Pod of their works. Is it really reading/writing direct

Yes, I had forgotten that they striped their data across multiple pods... I'm not familiar with how each Pod of their works. Is it really reading/writing directly to the devices, or is there an intermediate FS that is storing the files (well, chunks of files) and the Backblaze software is orchestrating it? Or do the 20 pods have one central interface? I guess I was assuming it worked like Lustre or BeeGFS where there were smaller FSs (ext4, XFS) at play that handled storing file (chunks) and then a higher level interface that managed it all. Either way -- the original comment was referring to Backblaze swapping 7PBs for 19PBs. Which we agree is not a normal size for a "normal" file system! ZFS is a strange beast... it was designed to support a world that just never showed up. It is a "normal" FS that works from laptops to servers, but was designed to support a big iron world that never really appeared (much). Instead, here we are comparing it not to XFS, but to cloud FS's that were designed for a completely different "cloud" world.

arghwhat 9y ago
The vault is the lowest unit you can interact with. There is some routing on top which picks a vault for you to use, but that's about it. I would assume that the tome, beneath network protocols, interacts directly with the individual block devices. They have implemented things like Reed-Solomon error correction themselves to handle redundancy across the tome (which to recap is their cross-server vdev equivalent), so they have indeed implemented tasks of a filesystem and RAID manager on top of their lower-level primitives. I also believe that what their low-level primitives are is actually a rather irrelevant implementation detail. A filesystem presents a way to store and organize files, and is a filesystem regardless of whether it is implemented on a block device or an egg-engraver. It could be using FAT16 as object store, sharding across files and handling metadata elsewhere. I would still consider such setup to be a true filesystem, as it is not simply a 1:1 network-to-disk protocol. And yes, ZFS won't see the large systems it is designed to handle anytime soon. What Sun did not see coming was the death of large, high-capacity servers, and the rise of distributed solutions. But then again, no one saw that coming in 2001 when ZFS development started. However, considering that storage sizes will continue to increase, who knows what will happen in the future... 128 bit filesystems might come in handy at some point. (Note that my information on backblaze's systems is based on public information—I am not an insider of any sort.)

arghwhat 9y ago

The vault is the lowest unit you can interact with. There is some routing on top which picks a vault for you to use, but that's about it. I would assume that the tome, beneath network protocols, interacts directly with the individual block devices. They have implemented things like Reed-Solomon error correction themselves to handle redundancy across the tome (which to recap is their cross-server vdev equivalent), so they have indeed implemented tasks of a filesystem and RAID manager on top of their lower-level primitives. I also believe that what their low-level primitives are is actually a rather irrelevant implementation detail. A filesystem presents a way to store and organize files, and is a filesystem regardless of whether it is implemented on a block device or an egg-engraver. It could be using FAT16 as object store, sharding across files and handling metadata elsewhere. I would still consider such setup to be a true filesystem, as it is not simply a 1:1 network-to-disk protocol. And yes, ZFS won't see the large systems it is designed to handle anytime soon. What Sun did not see coming was the death of large, high-capacity servers, and the rise of distributed solutions. But then again, no one saw that coming in 2001 when ZFS development started. However, considering that storage sizes will continue to increase, who knows what will happen in the future... 128 bit filesystems might come in handy at some point. (Note that my information on backblaze's systems is based on public information—I am not an insider of any sort.)