3 ms·

Just be careful when you use minIO, as it does not support all of the S3 API's. The maintainer of the project will close the tickets for errors when using S3 li

Just be careful when you use minIO, as it does not support all of the S3 API's. The maintainer of the project will close the tickets for errors when using S3 libraries and ask to write custom code specific for minIO which works well in amazon S3 [1]. So if your company is ready to invest in MinIO without S3 compatibility it's a nice software, and my kudos to the team who took the efforts to build it. It's just that it's not fully S3 compatible and MinIO buckets do not behave the same as S3 buckets. [1] https://github.com/minio/minio/issues/10160 https://github.com/minio/minio/issues/10160

polskibus 6y ago
That's really bad. Their marketing says upfront that they are designed for S3 API. If they don't strive for maximum compatibility, then their value prop goes out of the window. I was wondering if I should use minio as a compat layer to be able to use other tools that integrate with S3, what you pointed out probably saved me lots of problem investigation time on the future.
stingraycharles 6y ago
We’re using minio using Amazon’s own S3 SDK, and while they don’t support the latest S3 protocol version, it’s definitely not as bad as you say it is. We had more problems with Google Cloud Storage’s S3 API than with Minio’s (I believe there was something related to listing directories with their API as well, but it’s returning a flat out error instead). The key thing to note is that it’s probably not fully compatible, but it’s definitely compatible enough for many use cases. I can also see in the issue you reported that you changed the title of the issue from "Minio is not fully S3 compatible" to "Minio is not S3 compatible", which strikes me as you having some personal beef with them.
- vikrantrathore 6y ago
  There isn't any personal beef, I just take objections to the marekting line The defacto standard for Amazon S3 compatibility. In my view it's misleading as minIO is not a drop-in replacement for S3 API based applications. My team spend significant efforts to diagnose and find the error, as I believed in this line and asked them to retry different ways by changing code again and again, even though code was working fine with Amazon S3. I changed the title so that it can warn others, to not go through the same shooting the foot believing MinIO is a drop in replacement for Amazon S3 based applications. If you read the thread you can notice there isn't any plan to support posix like folder API in minIO which is supported in Amazon S3.
  - stingraycharles 6y ago
    Of course, but marketing is marketing; if you blindly believe marketing headlines before doing your due diligence, I'm sorry to say, but it's a bit naive. If you look at pretty much any protocol out there that's implemented by multiple vendors, there are all small nuances and incompatibilities. Even extremely detailed specified protocols have incompatibilities; S3 not having a formal specification should make you expect even worse. If you look at S3 implementations from the various clouds and NAS appliances and whatnot, you'll see that none of them is a 1:1 compatibilite with each other. If you wanted to warn others that, instead of Minio not providing "full" S3 compatibility, but Minio does not provide S3 compatibility at all (which is what the title edit suggests), then I believe you are mistaken. Minio definitely provides S3 compatibility, all the main tools and SDKs can talk perfectly with Minio, it's just that there are different implementation details around the edges.
    - vikrantrathore 6y ago
      > marketing is marketing I didn't believe it blindly and that's the reason there is a ticket, after thorough investigation of the error in code as well as self-hosted instance configuration and log file analysis. Also to make sure others do not need to go through the same long process of debugging, try to request the project maintainers to change this marketing claim as evident in the ticket. Please check the ticket and see for yourself. I did it to make sure others facing the same problem can see it before spending too much efforts to debug. I can see the amount of work minIO team has put in this project and again kudos to the team. Open source is hard and requires a lot of commitment. This is how open source works, you give back to community in the form of feedback, documentation, issues and if capable in code.
kdkeyser 6y ago
I have not yet seen a storage system that is actually 100% S3 compatible. Either functionality is missing, and/or there are corner cases where the behavior is just slightly different. It is also a complicated API, that often looks like it is a reflection of some internal engineering choices that were made (e.g. the object metadata structure AWS uses probably is the reason why the list call you mentioned behaves the way it does on AWS S3) However, in this case, MinIO makes the deliberate choice to deviate from AWS S3 behavior, on a very common call (listing objects). I do agree that this has the risk to break applications in non-obvious ways, e.g. the call succeeds, bu t your application behavior differs, compared to running against AWS S3, due to the missing directory entry. I also find their argumentation in the ticket a bit worrisome, calling the AWS S3 behavior a "blunder". I saw the same hubris when looking at their data path, where they choose to ignore battle-tested approaches like Paxos / Raft for distributed coordination, and instead build their own distributed lock algorithm and implementation: it does not seem to persist state, so resilience to power failure and server/process crashes might not be what you expect.
- cyberpunk 6y ago
  Check out leofs -- we're yet to hit any compatability issues with it, and it even has a nfs server which is quite nice.

polskibus 6y ago

That's really bad. Their marketing says upfront that they are designed for S3 API. If they don't strive for maximum compatibility, then their value prop goes out of the window. I was wondering if I should use minio as a compat layer to be able to use other tools that integrate with S3, what you pointed out probably saved me lots of problem investigation time on the future.

stingraycharles 6y ago

We’re using minio using Amazon’s own S3 SDK, and while they don’t support the latest S3 protocol version, it’s definitely not as bad as you say it is. We had more problems with Google Cloud Storage’s S3 API than with Minio’s (I believe there was something related to listing directories with their API as well, but it’s returning a flat out error instead). The key thing to note is that it’s probably not fully compatible, but it’s definitely compatible enough for many use cases. I can also see in the issue you reported that you changed the title of the issue from "Minio is not fully S3 compatible" to "Minio is not S3 compatible", which strikes me as you having some personal beef with them.

vikrantrathore 6y ago
There isn't any personal beef, I just take objections to the marekting line The defacto standard for Amazon S3 compatibility. In my view it's misleading as minIO is not a drop-in replacement for S3 API based applications. My team spend significant efforts to diagnose and find the error, as I believed in this line and asked them to retry different ways by changing code again and again, even though code was working fine with Amazon S3. I changed the title so that it can warn others, to not go through the same shooting the foot believing MinIO is a drop in replacement for Amazon S3 based applications. If you read the thread you can notice there isn't any plan to support posix like folder API in minIO which is supported in Amazon S3.
- stingraycharles 6y ago
  Of course, but marketing is marketing; if you blindly believe marketing headlines before doing your due diligence, I'm sorry to say, but it's a bit naive. If you look at pretty much any protocol out there that's implemented by multiple vendors, there are all small nuances and incompatibilities. Even extremely detailed specified protocols have incompatibilities; S3 not having a formal specification should make you expect even worse. If you look at S3 implementations from the various clouds and NAS appliances and whatnot, you'll see that none of them is a 1:1 compatibilite with each other. If you wanted to warn others that, instead of Minio not providing "full" S3 compatibility, but Minio does not provide S3 compatibility at all (which is what the title edit suggests), then I believe you are mistaken. Minio definitely provides S3 compatibility, all the main tools and SDKs can talk perfectly with Minio, it's just that there are different implementation details around the edges.
  - vikrantrathore 6y ago
    > marketing is marketing I didn't believe it blindly and that's the reason there is a ticket, after thorough investigation of the error in code as well as self-hosted instance configuration and log file analysis. Also to make sure others do not need to go through the same long process of debugging, try to request the project maintainers to change this marketing claim as evident in the ticket. Please check the ticket and see for yourself. I did it to make sure others facing the same problem can see it before spending too much efforts to debug. I can see the amount of work minIO team has put in this project and again kudos to the team. Open source is hard and requires a lot of commitment. This is how open source works, you give back to community in the form of feedback, documentation, issues and if capable in code.

kdkeyser 6y ago

I have not yet seen a storage system that is actually 100% S3 compatible. Either functionality is missing, and/or there are corner cases where the behavior is just slightly different. It is also a complicated API, that often looks like it is a reflection of some internal engineering choices that were made (e.g. the object metadata structure AWS uses probably is the reason why the list call you mentioned behaves the way it does on AWS S3) However, in this case, MinIO makes the deliberate choice to deviate from AWS S3 behavior, on a very common call (listing objects). I do agree that this has the risk to break applications in non-obvious ways, e.g. the call succeeds, bu t your application behavior differs, compared to running against AWS S3, due to the missing directory entry. I also find their argumentation in the ticket a bit worrisome, calling the AWS S3 behavior a "blunder". I saw the same hubris when looking at their data path, where they choose to ignore battle-tested approaches like Paxos / Raft for distributed coordination, and instead build their own distributed lock algorithm and implementation: it does not seem to persist state, so resilience to power failure and server/process crashes might not be what you expect.

cyberpunk 6y ago
Check out leofs -- we're yet to hit any compatability issues with it, and it even has a nfs server which is quite nice.