3 ms·

I do not get it. Why would I use zfs for this? Isn't the power of zfs that it's a filesystem with checksum and stuff like encryption? Why would I use it for s

I do not get it. Why would I use zfs for this? Isn't the power of zfs that it's a filesystem with checksum and stuff like encryption? Why would I use it for s3?

deleted 9mo ago
[deleted]
bakies 9mo ago
I've got a massive storage server built that I want to run s3 protocol on it. It's already running ZFS. This is exactly what I want. zfs-share already implements SMB and NFS.
- 0x457 9mo ago
  This is not what it is. This is building zpool on top of an S3 backend (vdev). Not sure what is the use case out of my ignorance, but I guess one can use it to `zfs send` backups to s3 in a very neat manner.
  - lkjdsklf 9mo ago
    One use case that comes to mind is backups. I can have a zpool created backed by a S3 vdev and then use zfs send | zfs recv to backup datasets to S3 ( or the billion other S3 like providers) Saves me the step of creating an instance with EBS volumes and snapshotting those to S3 or whatever haven't done the math at all on whether that's cost effective, but that's the usecase that comes to mind immediately
- suprasam 9mo ago
  I hope you are not having that massive storage storage on public-cloud then you would need MayaNAS to reduce storage costs. For S3 as frontend use MinIO gateway - serves S3 API from your ZFS filesystem
mustache_kimono 9mo ago
> Why would I use it for s3? You have it the wrong way around. Here, ZFS uses many small S3 objects as the storage substrate, rather than physical disks. The value proposition is that this should be definitely cheaper and perhaps more durable than EBS. See s3backer, a FUSE implementation of similar: https://github.com/archiecobbs/s3backer https://github.com/archiecobbs/s3backer See prior in kernel ZFS work by Delphix which AFAIK was closed by Delphix management: https://www.youtube.com/watch?v=opW9KhjOQ3Q https://www.youtube.com/watch?v=opW9KhjOQ3Q BTW this appears to be closed too!
- suprasam 9mo ago
  Yes that is the value prop. Cheap S3 instead of expensive EBS. EBS limitations: - Per-instance throughput caps - Pay for full provisioned capacity whether filled or not S3: - Pay only for what you store - No per-instance bandwidth limits as long as you have network optimized instance

I do not get it. Why would I use zfs for this? Isn't the power of zfs that it's a filesystem with checksum and stuff like encryption? Why would I use it for s3?

deleted 9mo ago

[deleted]

bakies 9mo ago

I've got a massive storage server built that I want to run s3 protocol on it. It's already running ZFS. This is exactly what I want. zfs-share already implements SMB and NFS.

0x457 9mo ago
This is not what it is. This is building zpool on top of an S3 backend (vdev). Not sure what is the use case out of my ignorance, but I guess one can use it to `zfs send` backups to s3 in a very neat manner.
- lkjdsklf 9mo ago
  One use case that comes to mind is backups. I can have a zpool created backed by a S3 vdev and then use zfs send | zfs recv to backup datasets to S3 ( or the billion other S3 like providers) Saves me the step of creating an instance with EBS volumes and snapshotting those to S3 or whatever haven't done the math at all on whether that's cost effective, but that's the usecase that comes to mind immediately
suprasam 9mo ago
I hope you are not having that massive storage storage on public-cloud then you would need MayaNAS to reduce storage costs. For S3 as frontend use MinIO gateway - serves S3 API from your ZFS filesystem

mustache_kimono 9mo ago

> Why would I use it for s3? You have it the wrong way around. Here, ZFS uses many small S3 objects as the storage substrate, rather than physical disks. The value proposition is that this should be definitely cheaper and perhaps more durable than EBS. See s3backer, a FUSE implementation of similar: https://github.com/archiecobbs/s3backer https://github.com/archiecobbs/s3backer See prior in kernel ZFS work by Delphix which AFAIK was closed by Delphix management: https://www.youtube.com/watch?v=opW9KhjOQ3Q https://www.youtube.com/watch?v=opW9KhjOQ3Q BTW this appears to be closed too!

suprasam 9mo ago
Yes that is the value prop. Cheap S3 instead of expensive EBS. EBS limitations: - Per-instance throughput caps - Pay for full provisioned capacity whether filled or not S3: - Pay only for what you store - No per-instance bandwidth limits as long as you have network optimized instance