4 ms·

Just here to rant about 'S3 compliance' a little - I have no real awareness on the topic at hand. Tangentially this is lamenting at the lack of choice Last ni

Just here to rant about 'S3 compliance' a little - I have no real awareness on the topic at hand. Tangentially this is lamenting at the lack of choice Last night I tried to set up BackBlaze (S3 'compatible') behind Mattermost. This doesn't go very far when clients use/rely on headers that specifically mention vendorization BackBlaze won't work with clients that try to send x-amz-sdk-checksum-algorithm, Mattermost taught me this TLDR: S3 isn't always S3

toomuchtodo 3y ago
Did you happen to submit this bug to Backblaze for triage? If not, I'll get it to the right people! > When you're using an SDK, you can set the value of the x-amz-sdk-checksum-algorithm parameter to the algorithm that you want Amazon S3 to use when calculating the checksum. Amazon S3 automatically calculates the checksum value. > When you're using the REST API, you don't use the x-amz-sdk-checksum-algorithm parameter. Instead, you use one of the algorithm-specific headers (for example, x-amz-checksum-crc32). https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/checki...
- bravetraveler 3y ago
  I hadn't considered it! Your username is appropriate, too much to do :D My requirements are pretty low so I was likely to just let it slide... ...but with your help (and perhaps from others on understanding the pieces), I'll consider it! I'm remarkably unfamiliar with S3 style of services, only occasionally trying to dabble edit: The choosing of appropriate headers may come down to Mattermost/clients -- BackBlaze might be technically doing the right thing saying "I don't know what to do with these"
  - toomuchtodo 3y ago
    What can I say, I want to solve all the things :) I will report back.
- kpetermeni 3y ago
  +1 Please do. Backblaze lists x-amz-sdk-checksum-algorithm as unsupported [1]. Would be great to have it supported to be able to use it with Mattermost and other tools that use min.io for S3. [1] https://www.backblaze.com/b2/docs/s3_compatible_api.html https://www.backblaze.com/b2/docs/s3_compatible_api.html
jawngee 3y ago
Backblaze is not a good choice for most things unless you were using it strictly for backup. Their S3 compatibility, as is most S3 compatibility, is a cruel joke last I looked at it. I publish a mildly(?) popular offloading plugin for WordPress and adding Backblaze support was my biggest regret.
- bravetraveler 3y ago
  That's unfortunate to hear... I assumed the API to be more standardized in the strictest sense. I was truly surprised to see Amazon branded headers in something claiming to be 'standards compliant' They may be optional, but their presence simply complicates things; particularly when clients opt into them. It devalues what is common/reused! Either locking you into Amazon or another protocol entirely
- OnlyMortal 3y ago
  It’s also very, very slow. I switched to Wasabi and am using the C++ AWS SDK. I’m impressed by their performance Usage is domestic fibre gigabit symmetric from a 2013 MacBook Pro.

toomuchtodo 3y ago

Did you happen to submit this bug to Backblaze for triage? If not, I'll get it to the right people! > When you're using an SDK, you can set the value of the x-amz-sdk-checksum-algorithm parameter to the algorithm that you want Amazon S3 to use when calculating the checksum. Amazon S3 automatically calculates the checksum value. > When you're using the REST API, you don't use the x-amz-sdk-checksum-algorithm parameter. Instead, you use one of the algorithm-specific headers (for example, x-amz-checksum-crc32). https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/checki...

bravetraveler 3y ago
I hadn't considered it! Your username is appropriate, too much to do :D My requirements are pretty low so I was likely to just let it slide... ...but with your help (and perhaps from others on understanding the pieces), I'll consider it! I'm remarkably unfamiliar with S3 style of services, only occasionally trying to dabble edit: The choosing of appropriate headers may come down to Mattermost/clients -- BackBlaze might be technically doing the right thing saying "I don't know what to do with these"
- toomuchtodo 3y ago
  What can I say, I want to solve all the things :) I will report back.
kpetermeni 3y ago
+1 Please do. Backblaze lists x-amz-sdk-checksum-algorithm as unsupported [1]. Would be great to have it supported to be able to use it with Mattermost and other tools that use min.io for S3. [1] https://www.backblaze.com/b2/docs/s3_compatible_api.html https://www.backblaze.com/b2/docs/s3_compatible_api.html

jawngee 3y ago

Backblaze is not a good choice for most things unless you were using it strictly for backup. Their S3 compatibility, as is most S3 compatibility, is a cruel joke last I looked at it. I publish a mildly(?) popular offloading plugin for WordPress and adding Backblaze support was my biggest regret.

bravetraveler 3y ago
That's unfortunate to hear... I assumed the API to be more standardized in the strictest sense. I was truly surprised to see Amazon branded headers in something claiming to be 'standards compliant' They may be optional, but their presence simply complicates things; particularly when clients opt into them. It devalues what is common/reused! Either locking you into Amazon or another protocol entirely
OnlyMortal 3y ago
It’s also very, very slow. I switched to Wasabi and am using the C++ AWS SDK. I’m impressed by their performance Usage is domestic fibre gigabit symmetric from a 2013 MacBook Pro.