4 ms·

They have a bunch of claims comparing this to JuiceFS at https://www.zerofs.net/zerofs-vs-juicefs https://www.zerofs.net/zerofs-vs-juicefs I am in no way affil

They have a bunch of claims comparing this to JuiceFS at https://www.zerofs.net/zerofs-vs-juicefs https://www.zerofs.net/zerofs-vs-juicefs I am in no way affiliated with JuiceFS, but I have done a lot of benchmarking and testing of it, and the numbers claimed here for JuiceFS are suspicious (basically 5 ops/second with everything mostly broken).

ChocolateGod 1y ago
I have a juicefs setup and get operations in the 1000/s, not sure how they got such low numbers. JuiceFS also supports multiple concurrent clients making their own connection to the metadata and object storage, allowing near instant synchronization and better performance, where this seems to rely on a single service having a connection and everyone connecting through it with no support for clustering.
- deleted 1y ago
  [deleted]
- Eikon 1y ago
  I have no doubt that JuiceFS can perform “thousands of operations per second” across parallel clients. I don't think that's a useful benchmark because the use cases we are targeting are not embarrassingly parallel. Using a bunch of clients on any system hides the latency profile. You could even get your "thousands of operations per second" on a system where any operation takes 10 seconds to complete.
  - ChocolateGod 1y ago
    I'm referring to one client being able to do 1000s of operations a second. I've not experienced this bad performance with JuiceFS that the article is describing. This is a JuiceFS setup with 10TB of data, where JuiceFS was specifically chosen because of its minimal latency compared to raw object storage.
    - Eikon 1y ago
      > I'm referring to one client being able to do 1000s of operations a second Again, I don't see any problems with "thousands of operations a second" if your single client is issuing parallel requests. Try any sequential workload instead. > because of its minimal latency compared to raw object storage That's not how it works. JuiceFS hits the object store for every file operation (except metadata, because it's backed by an external database). Latency cannot be lower, especially as JuiceFS doesn't seem to be implementing any buffering. Above you also claim "allowing near instant synchronization and better performance", but near instant synchronization of what exactly? Metadata? Data blocks? And under what consistency model? If you wish to run the benchmarks in bench/ on your setup and show better numbers, I'd happily update the JuiceFS benchmarks.
Eikon 1y ago
The bench suite is published at the root of the repo in bench/. Stating “a bunch of claims” seems a bit dismissive when it’s that easy to reproduce. JuiceFS maps operations 1:1 to s3 chunk wise, so the performance profile is entirely unsurprising to me, even untaring an archive is not going to be a pleasant experience.
selfhoster1312 1y ago
I've never used JuiceFS in prod (or any S3 product for that matter), but i was involved in benchmarking juiceFS+garage for archiving based on torrents, and initial results were promising but qBittorrent quickly produced a pathological case where reads/writes dropped to almost zero (and stayed that way long term). The data and method can be found here: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1021 https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1021 Every software's perf is optimized for a usage pattern and maybe TFA's benchmark or torrenting isn't it, but i was certainly disappointed in the performance i found on 12 core / 144GB / NVME (for metadata) / 6x8TB HDD (for data). In the end we didn't move to S3 we stayed with good old ZFS storage. I'd be curious to reproduce my benchmarks with ZeroFS when i find the time.

ChocolateGod 1y ago

I have a juicefs setup and get operations in the 1000/s, not sure how they got such low numbers. JuiceFS also supports multiple concurrent clients making their own connection to the metadata and object storage, allowing near instant synchronization and better performance, where this seems to rely on a single service having a connection and everyone connecting through it with no support for clustering.

deleted 1y ago
[deleted]
Eikon 1y ago
I have no doubt that JuiceFS can perform “thousands of operations per second” across parallel clients. I don't think that's a useful benchmark because the use cases we are targeting are not embarrassingly parallel. Using a bunch of clients on any system hides the latency profile. You could even get your "thousands of operations per second" on a system where any operation takes 10 seconds to complete.
- ChocolateGod 1y ago
  I'm referring to one client being able to do 1000s of operations a second. I've not experienced this bad performance with JuiceFS that the article is describing. This is a JuiceFS setup with 10TB of data, where JuiceFS was specifically chosen because of its minimal latency compared to raw object storage.
  - Eikon 1y ago
    > I'm referring to one client being able to do 1000s of operations a second Again, I don't see any problems with "thousands of operations a second" if your single client is issuing parallel requests. Try any sequential workload instead. > because of its minimal latency compared to raw object storage That's not how it works. JuiceFS hits the object store for every file operation (except metadata, because it's backed by an external database). Latency cannot be lower, especially as JuiceFS doesn't seem to be implementing any buffering. Above you also claim "allowing near instant synchronization and better performance", but near instant synchronization of what exactly? Metadata? Data blocks? And under what consistency model? If you wish to run the benchmarks in bench/ on your setup and show better numbers, I'd happily update the JuiceFS benchmarks.

Eikon 1y ago

The bench suite is published at the root of the repo in bench/. Stating “a bunch of claims” seems a bit dismissive when it’s that easy to reproduce. JuiceFS maps operations 1:1 to s3 chunk wise, so the performance profile is entirely unsurprising to me, even untaring an archive is not going to be a pleasant experience.

selfhoster1312 1y ago

I've never used JuiceFS in prod (or any S3 product for that matter), but i was involved in benchmarking juiceFS+garage for archiving based on torrents, and initial results were promising but qBittorrent quickly produced a pathological case where reads/writes dropped to almost zero (and stayed that way long term). The data and method can be found here: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1021 https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1021 Every software's perf is optimized for a usage pattern and maybe TFA's benchmark or torrenting isn't it, but i was certainly disappointed in the performance i found on 12 core / 144GB / NVME (for metadata) / 6x8TB HDD (for data). In the end we didn't move to S3 we stayed with good old ZFS storage. I'd be curious to reproduce my benchmarks with ZeroFS when i find the time.