4 ms·

Apart from all these other products that implement s3? MinIO, Ceph (RGW), Garage, SeaweedFS, Zenko CloudServer, OpenIO, LakeFS, Versity, Storj, Riak CS, Juic

by _joel 6mo ago

Apart from all these other products that implement s3? MinIO, Ceph (RGW), Garage, SeaweedFS, Zenko CloudServer, OpenIO, LakeFS, Versity, Storj, Riak CS, JuiceFS, Rustfs, s3proxy.

0x457 6mo ago
Riak CS been dead for over a decade which makes me question the rest. Some of these also do not have the same behaviors when it comes to paths (MinIO is one of those IIRC). Also, none of them implement full S3 API and features.
- Datagenerator 6mo ago
  What does RadosGW miss?
- skrtskrt 6mo ago
  There's a difference between S3 API spec and what Amazon does with S3 - for isntance, the new CAS capabilities with Amazon are not part of the spec. Ceph certainly implements the full API spec, though it may lag behind some changes. It's mostly a question of engineering time available to the projects to keep up with changes.
  - 0x457 6mo ago
    > There's a difference between S3 API spec and what Amazon does with S3 - for isntance, the new CAS capabilities with Amazon are not part of the spec. Sure, but those are S3 APIs and features that provided by AWS. We not talking about S3 spec, we're talking about s3 product.
    - skrtskrt 6mo ago
      You’re wrong. “Implementing the S3 API” means the spec, not an Amazon product.
      - 0x457 6mo ago
        We're talking about using AWS S3 and using something that implements S3 spec. Really not that hard to understand. When customer wants to switch away from AWS S3, customer cares about AWS S3 feature coverage and not the spec.
        ovaistariq 6mo ago
        Spec and features are intertwined. Customers who switch away from AWS S3 still want to use the same SDKs, libraries, etc that support S3 API. They don't want to rewrite their applications to use a new API. So then is it feature coverage or spec coverage?
        0x457 6mo ago
        It's both? Customer doesn't care if spec is 100% covered if feature that they are using in AWS S3 isn't supported. Also, who is rewriting their application to change interaction with an object storage? People that directly use some S3 sdk all around the app should read a book on software engineering or at least a blog post.
        skrtskrt 6mo ago
        Wrong again. Not only have I worked on DigitalOcean’s S3 implementation but I currently work on an open source product that targets S3 spec and can be used with any cloud provider and any other spec-compliant drop-in, like Garage.
        0x457 6mo ago
        All that means: you only used AWS S3 features that are in the S3 spec.
ovaistariq 6mo ago
Add Tigris to the list as well please. We maintain a page that shows our compatibility with S3 API. It's at https://www.tigrisdata.com/docs/api/s3/ https://www.tigrisdata.com/docs/api/s3/. The test runner is open source at https://github.com/tigrisdata-community/s3-api-compat-tests https://github.com/tigrisdata-community/s3-api-compat-tests

by _joel 6mo ago

Apart from all these other products that implement s3? MinIO, Ceph (RGW), Garage, SeaweedFS, Zenko CloudServer, OpenIO, LakeFS, Versity, Storj, Riak CS, JuiceFS, Rustfs, s3proxy.

0x457 6mo ago

Riak CS been dead for over a decade which makes me question the rest. Some of these also do not have the same behaviors when it comes to paths (MinIO is one of those IIRC). Also, none of them implement full S3 API and features.

Datagenerator 6mo ago
What does RadosGW miss?
skrtskrt 6mo ago
There's a difference between S3 API spec and what Amazon does with S3 - for isntance, the new CAS capabilities with Amazon are not part of the spec. Ceph certainly implements the full API spec, though it may lag behind some changes. It's mostly a question of engineering time available to the projects to keep up with changes.
- 0x457 6mo ago
  > There's a difference between S3 API spec and what Amazon does with S3 - for isntance, the new CAS capabilities with Amazon are not part of the spec. Sure, but those are S3 APIs and features that provided by AWS. We not talking about S3 spec, we're talking about s3 product.
  - skrtskrt 6mo ago
    You’re wrong. “Implementing the S3 API” means the spec, not an Amazon product.
    - 0x457 6mo ago
      We're talking about using AWS S3 and using something that implements S3 spec. Really not that hard to understand. When customer wants to switch away from AWS S3, customer cares about AWS S3 feature coverage and not the spec.
      - ovaistariq 6mo ago
        Spec and features are intertwined. Customers who switch away from AWS S3 still want to use the same SDKs, libraries, etc that support S3 API. They don't want to rewrite their applications to use a new API. So then is it feature coverage or spec coverage?
        0x457 6mo ago
        It's both? Customer doesn't care if spec is 100% covered if feature that they are using in AWS S3 isn't supported. Also, who is rewriting their application to change interaction with an object storage? People that directly use some S3 sdk all around the app should read a book on software engineering or at least a blog post.
      - skrtskrt 6mo ago
        Wrong again. Not only have I worked on DigitalOcean’s S3 implementation but I currently work on an open source product that targets S3 spec and can be used with any cloud provider and any other spec-compliant drop-in, like Garage.
        0x457 6mo ago
        All that means: you only used AWS S3 features that are in the S3 spec.

ovaistariq 6mo ago

Add Tigris to the list as well please. We maintain a page that shows our compatibility with S3 API. It's at https://www.tigrisdata.com/docs/api/s3/ https://www.tigrisdata.com/docs/api/s3/. The test runner is open source at https://github.com/tigrisdata-community/s3-api-compat-tests https://github.com/tigrisdata-community/s3-api-compat-tests