4 ms·
Versity open-sources ScoutFS
- jorangreef 8y ago"The super block is stored in two 4KB blocks at a known location at the start of the device." There should be more than 2 slots for the super block, e.g. at least 64 slots for the super block, to allow for better wear-leveling for SSDs. AFAIK this has been a problem for some filesystems, which wear out the device due to the high frequency of super block updates. The latest super block should also be replicated 2-4 times on disk to survive corruption. Otherwise, the loss of a mere two contiguous disk sectors could result in loss of the entire disk. A super block of 4KB also looks like they might be relying too much on the device to guarantee atomic sector writes (this is almost always the case, but not necessarily), instead of using a cryptographic signature to verify that the super block was written atomically. Also, 4KB seems small these days for a super block, especially if the super block needs to include replicated information relating to RAID sibling devices etc.
- a012 8y agoWhat's the difference of archival filesystem vs. distributed filesystem? I assume other distributed filesystems could be used to archive files as well, namely https://github.com/moosefs/moosefs https://github.com/moosefs/moosefs
- notacoward 8y agoThe focus here seems to be on very high scale, not just capacity but also number of files and scan performance which would be important for an archival store and which are often pain points for other distributed filesystems. Similarly, ensuring data integrity is a high priority for such a system. I certainly wouldn't compare it to a single-metadata-server pseudo-distributed filesystem that doesn't even honor O_SYNC writes.
- meghan-versity 8y agoCouldn't have said it better myself! For more details on requirements as we designed the filesystem and why current filesystems didn't meet our needs, see this blog post: http://www.versity.com/blog/versity-open-sources-scoutfs http://www.versity.com/blog/versity-open-sources-scoutfs
- SOLAR_FIELDS 8y agoIf anyone from Versity is reading this, there is a typo in the release: “We are releaing the technology under the GPLv2 open source license.”
- meghan-versity 8y agoDoh! Thanks for letting us know, will get that fixed. Of course, patches/contributions are always welcome.... :)
- notacoward 8y agoLooks interesting, and apparently it's really a filesystem unlike so many thing that just slap "FS" on the name and call it a day. tears of joy White paper etc. is a bit low on details, though.
- patrickg_zill 8y agoWhat I didn't see in the white paper, is whether the contents of each file are checksummed? It seems only the metadata is checksummed. I have used ZFS and the ability to run a check on the entire set of metadata and actual files is very reassuring (though it is definitely time consuming).
- meghan-versity 8y agoBoth metadata and data are checksummed. A unique feature of the filesystem is that it doesn't trust what the storage tells it, it actually verifies it. Thanks for pointing this out - we'll get these details in the next version of the paper.
- patrickg_zill 8y agoThat is great to hear and I look forward to reading more about it, thanks.
- unixhero 8y agoLinux mainline soon?
- meghan-versity 8y agoDefinitely.