3 ms·
Tiered storage and FUSE has existed with Alluxio for years. And NVMe optimisations e.g. NVMeoF in OpenEBS (Mayastor). None of it is particularly ground breaki
by threeseed 1y ago
Tiered storage and FUSE has existed with Alluxio for years.
And NVMe optimisations e.g. NVMeoF in OpenEBS (Mayastor).
None of it is particularly ground breaking just a lot of pieces brought together.
- jamesblonde 1y agoThe difference is scale-out metadata in the filesystem. Alluxio uses Raft, i believe, for metadata - that has to fit on a single server.
- rfoo 1y ago3FS isn't particularly fast in mdbench, though. Maybe our FDB tuning skill is what to blame, or FUSE, I don't know, but it doesn't really matter. The truly amazing part for me is combining NVMe SSD + RDMA + supports reading a huge batch of random offsets from a few already opened huge files efficiently. This is how you get your training boxes consuming 20~30GiB/s (and roughly 4 million IOPS).
- rjzzleep 1y agoFUSE has traditionally been famously slow. I remember there were some changes that supposedly made it faster, but maybe that was just a certain fuse implementation.
- jamesblonde 1y agoThe block size is 4KB by default, which is a killer. We set it to 1MB or so by default - makes a huge difference. https://github.com/logicalclocks/hopsfs-go-mount https://github.com/logicalclocks/hopsfs-go-mount