4 ms·

That's pretty impressive. Anyone using minio in production? What's the backup story?

71a54xd 5y ago
I've only heard horror stories from people using this on real apps. The last one was a bug where MinIO wouldn't actually assert a file had been updated or something. Maybe they've improved since then, but I wouldn't call this "production ready" for the time being.
- tyingq 5y ago
  I think you're thinking of this comment thread: https://news.ycombinator.com/item?id=28133902 https://news.ycombinator.com/item?id=28133902 I was also surprised by many of the comments. I had only played with minio a bit, and was considering using it. The durability comments were concerning, and there were other issues too. One example is this bug: https://github.com/minio/minio/issues/8873 https://github.com/minio/minio/issues/8873 Basically, it was treating object names like foo//bar in them the same as foo/bar, except for sharding, which thought they were different. Their fix was just to disallow '//' in an object name, even though other s3-like implementations allow it.
  - mastazi 5y ago
    Given that MinIO seems to be canonical way to add S3 API compatibility to Azure Blob Storage[1], this is not very encouraging. We have a lot of products that use S3 API and Azure was the only cloud that did not offer that out of the box - at first I could not believe this was the case, because even smaller providers like Linode or Digitalocean offer S3 API compatibility. But then I found the post linked below. [1] https://cloudblogs.microsoft.com/opensource/2017/11/09/s3cmd-amazon-s3-compatible-apps-azure-storage/ https://cloudblogs.microsoft.com/opensource/2017/11/09/s3cmd...
    - tyingq 5y ago
      There's s3proxy: https://github.com/gaul/s3proxy https://github.com/gaul/s3proxy https://ventral.digital/posts/2020/10/11/s3-api-compatibility-on-microsoft-azure https://ventral.digital/posts/2020/10/11/s3-api-compatibilit...
      - mastazi 5y ago
        It's good that there are more alternatives, this seems like it might be a lighter weight alternative compared to MinIO. Still, I wish Azure just added a built-in compatibility layer, like virtually every other cloud provider - I'm not a big fan of having to spin up a container just for this reason.
outworlder 5y ago
Of course it's used in production. We use it heavily. It doesn't have to use local disks. A big part of what it does is to provide a S3-compatible API, which you can use with any number of backends. Even other cloud providers. Say you want to be able to deploy on prem, on AWS, and GCP. You can add MinIO in front of all of these and you app won't care. Of course, if you are writing to an actual disk you'll have to figure out the backup part.
rjzzleep 5y ago
I've done Kubernetes consulting with some other people and for their on prem solutions I always recommend just buying a TrueNAS with it's s3 api. The other people I tend to end up with always try to sell people on minio+rook-ceph and then offer support along the way. So basically you buy their kubernetes deployment and then you have to pay them for the rest of your life for troubleshooting. The TrueNAS seems cheaper to me. I don't see a good backup story, but maybe I just don't know it.
- ochoseis 5y ago
  Truenas with S3 piqued my interest. Does that use minio under the hood to serve objects? This doc indicates it might at least use the browser component: https://www.truenas.com/docs/core/services/s3/ https://www.truenas.com/docs/core/services/s3/
  - Havoc 5y ago
    Pretty sure it’s minio for everything. Mainly use it for gitlab object storage current. Logs docker images etc
- hardwaresofton 5y ago
  Have you ever seen people using rook-ceph and the built in object gateway Ceph provides? AFAIK Ceph's object store powers some official cloud solutions out there like Digital Ocean spaces
  - rjzzleep 5y ago
    Yes, one of my clients uses it. It seems okay, but it's all pretty low volume and most people have no real concept of disaster recovery. I think backups and DR are things that most people take for granted and don't really think about.
    - hardwaresofton 5y ago
      Yep reasonable -- most people these days don't have to, since the clouds do the hard work for most people. The knowledge on how to run those kinds of systems becomes harder to find (in the community) by the year. Also by the time most people account for 2/3 copies of the data + 1 completely offsite backup of data I think their eyes might start to water at the cost if you want reasonable performance as well. I experiment with this kind of stuff a lot of the time and always am surprised at how much more than you'd expect it costs to get drive, node, and region level redundancy with backups. You need essentially at a minimum 3TB for every 1TB of usable storage assuming regular RAID1.

I've only heard horror stories from people using this on real apps. The last one was a bug where MinIO wouldn't actually assert a file had been updated or something. Maybe they've improved since then, but I wouldn't call this "production ready" for the time being.

tyingq 5y ago
I think you're thinking of this comment thread: https://news.ycombinator.com/item?id=28133902 https://news.ycombinator.com/item?id=28133902 I was also surprised by many of the comments. I had only played with minio a bit, and was considering using it. The durability comments were concerning, and there were other issues too. One example is this bug: https://github.com/minio/minio/issues/8873 https://github.com/minio/minio/issues/8873 Basically, it was treating object names like foo//bar in them the same as foo/bar, except for sharding, which thought they were different. Their fix was just to disallow '//' in an object name, even though other s3-like implementations allow it.
- mastazi 5y ago
  Given that MinIO seems to be canonical way to add S3 API compatibility to Azure Blob Storage[1], this is not very encouraging. We have a lot of products that use S3 API and Azure was the only cloud that did not offer that out of the box - at first I could not believe this was the case, because even smaller providers like Linode or Digitalocean offer S3 API compatibility. But then I found the post linked below. [1] https://cloudblogs.microsoft.com/opensource/2017/11/09/s3cmd-amazon-s3-compatible-apps-azure-storage/ https://cloudblogs.microsoft.com/opensource/2017/11/09/s3cmd...
  - tyingq 5y ago
    There's s3proxy: https://github.com/gaul/s3proxy https://github.com/gaul/s3proxy https://ventral.digital/posts/2020/10/11/s3-api-compatibility-on-microsoft-azure https://ventral.digital/posts/2020/10/11/s3-api-compatibilit...
    - mastazi 5y ago
      It's good that there are more alternatives, this seems like it might be a lighter weight alternative compared to MinIO. Still, I wish Azure just added a built-in compatibility layer, like virtually every other cloud provider - I'm not a big fan of having to spin up a container just for this reason.

outworlder 5y ago

Of course it's used in production. We use it heavily. It doesn't have to use local disks. A big part of what it does is to provide a S3-compatible API, which you can use with any number of backends. Even other cloud providers. Say you want to be able to deploy on prem, on AWS, and GCP. You can add MinIO in front of all of these and you app won't care. Of course, if you are writing to an actual disk you'll have to figure out the backup part.

rjzzleep 5y ago

I've done Kubernetes consulting with some other people and for their on prem solutions I always recommend just buying a TrueNAS with it's s3 api. The other people I tend to end up with always try to sell people on minio+rook-ceph and then offer support along the way. So basically you buy their kubernetes deployment and then you have to pay them for the rest of your life for troubleshooting. The TrueNAS seems cheaper to me. I don't see a good backup story, but maybe I just don't know it.

ochoseis 5y ago
Truenas with S3 piqued my interest. Does that use minio under the hood to serve objects? This doc indicates it might at least use the browser component: https://www.truenas.com/docs/core/services/s3/ https://www.truenas.com/docs/core/services/s3/
- Havoc 5y ago
  Pretty sure it’s minio for everything. Mainly use it for gitlab object storage current. Logs docker images etc
hardwaresofton 5y ago
Have you ever seen people using rook-ceph and the built in object gateway Ceph provides? AFAIK Ceph's object store powers some official cloud solutions out there like Digital Ocean spaces
- rjzzleep 5y ago
  Yes, one of my clients uses it. It seems okay, but it's all pretty low volume and most people have no real concept of disaster recovery. I think backups and DR are things that most people take for granted and don't really think about.
  - hardwaresofton 5y ago
    Yep reasonable -- most people these days don't have to, since the clouds do the hard work for most people. The knowledge on how to run those kinds of systems becomes harder to find (in the community) by the year. Also by the time most people account for 2/3 copies of the data + 1 completely offsite backup of data I think their eyes might start to water at the cost if you want reasonable performance as well. I experiment with this kind of stuff a lot of the time and always am surprised at how much more than you'd expect it costs to get drive, node, and region level redundancy with backups. You need essentially at a minimum 3TB for every 1TB of usable storage assuming regular RAID1.