3 ms·

I’m personally thoroughly unimpressed by the bare metal S3 implementations I’ve tried. And there’s also an issue with client libraries and their compatibility

by amluto 2y ago

I’m personally thoroughly unimpressed by the bare metal S3 implementations I’ve tried. And there’s also an issue with client libraries and their compatibility with various implementations. I recently discovered this issue: https://github.com/boto/botocore/issues/3394 https://github.com/boto/botocore/issues/3394 This is GCS implementing the S3 API incorrectly in a way that really ought not to break clients, but it’s still odd because the particular bug on GCS’s end seems like it took active effort to get wrong. But it’s also boto (the main library used in Python to access S3 and compatible services) doing something silly, tripping over GCS’s bug, and failing. And it’s AWS, who owns boto, freely admitting that they don’t intend to fix it, because boto isn’t actually intended to work with services that are merely compatible with S3. As icing on the cake, to report this bug to Google, I apparently need to pay $29 plus a 3% surcharge on my GCS usage. Thanks. Time to check out OpenDAL, I suppose.

ajross 2y ago
> As icing on the cake, to report this bug to Google, I apparently need to pay $29 plus a 3% surcharge on my GCS usage. Thanks. That's the price of a support contract, not a "bug report". And it's not "plus", it's "or": support costs $29/month or 3% of your monthly billing, whichever is greater. It comes with SLA agreements for fixing or working around your reported problems. Though obviously in this case they'll probably just tell you to use their own python library and not boto.
- amluto 2y ago
  Oh, thanks Google, if my cloud spend is more than $967/mo, then I don’t get dinged by the $29 minimum. But this is the price of a bug report, because I can’t file a bug report without paying it. And this situation is bad business. Google advertises that GCS has S3 interoperability support. And they have customers who use it in its interoperable mode. Presumably those customers could use GCS’s biggest competitor, too. Shouldn’t Google try to make the S3 interop work correctly?
  - ajross 2y ago
    GCS and AWS are commercial products for which you pay, not open source projects you can expect to support you for free. I don't know what to say here, if you have something "serious" to do on these platforms then a 3% overhead for support seems like an obvious choice. Honestly it seems to me like you're excited to have found a bug and want to report it for glory; we've all been there. But No One Cares about that stuff in the world of commercial software. They fix bugs for real customers, not internet rock stars. If you aren't losing even $29 (one mid-tier meal!) from this bug, well... does it even rise to the level of "yell about it on HN?".
- notpushkin 2y ago
  > Though obviously in this case they'll probably just tell you to use their own python library and not boto. What’s the point of S3 compatibility then?
  - ajross 2y ago
    S3 isn't a spec. You'll never be 100% compatible with a third party product with ad hoc behavior. Companies do what they can as a migration aid, but at the edges there will always be bugs like this (which I'm gathering is really an issue with boto being inflexible about unexpected details and not so much "incompatible" behavior by GCS). Again, this isn't an open source product. It's not like there's a universal agreement that coordination and compatibility are ideals to be respected, nor a public place for discussion and development. Google is trying to make GCS as S3 compatible as possible, but obviously they prioritize their paying customers (who seem not to have reported this issue). Amazon does not want GCS to be boto-compatible at all, really.

by amluto 2y ago

ajross 2y ago

> As icing on the cake, to report this bug to Google, I apparently need to pay $29 plus a 3% surcharge on my GCS usage. Thanks. That's the price of a support contract, not a "bug report". And it's not "plus", it's "or": support costs $29/month or 3% of your monthly billing, whichever is greater. It comes with SLA agreements for fixing or working around your reported problems. Though obviously in this case they'll probably just tell you to use their own python library and not boto.

amluto 2y ago
Oh, thanks Google, if my cloud spend is more than $967/mo, then I don’t get dinged by the $29 minimum. But this is the price of a bug report, because I can’t file a bug report without paying it. And this situation is bad business. Google advertises that GCS has S3 interoperability support. And they have customers who use it in its interoperable mode. Presumably those customers could use GCS’s biggest competitor, too. Shouldn’t Google try to make the S3 interop work correctly?
- ajross 2y ago
  GCS and AWS are commercial products for which you pay, not open source projects you can expect to support you for free. I don't know what to say here, if you have something "serious" to do on these platforms then a 3% overhead for support seems like an obvious choice. Honestly it seems to me like you're excited to have found a bug and want to report it for glory; we've all been there. But No One Cares about that stuff in the world of commercial software. They fix bugs for real customers, not internet rock stars. If you aren't losing even $29 (one mid-tier meal!) from this bug, well... does it even rise to the level of "yell about it on HN?".
notpushkin 2y ago
> Though obviously in this case they'll probably just tell you to use their own python library and not boto. What’s the point of S3 compatibility then?
- ajross 2y ago
  S3 isn't a spec. You'll never be 100% compatible with a third party product with ad hoc behavior. Companies do what they can as a migration aid, but at the edges there will always be bugs like this (which I'm gathering is really an issue with boto being inflexible about unexpected details and not so much "incompatible" behavior by GCS). Again, this isn't an open source product. It's not like there's a universal agreement that coordination and compatibility are ideals to be respected, nor a public place for discussion and development. Google is trying to make GCS as S3 compatible as possible, but obviously they prioritize their paying customers (who seem not to have reported this issue). Amazon does not want GCS to be boto-compatible at all, really.