14 ms·

A tangent - I love using minio in dev & test for their s3 simulators. Being able to throw away a bucket and start from scratch, and having everything self conta

A tangent - I love using minio in dev & test for their s3 simulators. Being able to throw away a bucket and start from scratch, and having everything self contained in my docker-compose command is a real blessing. Has anyone ever used min.io for production stuff? What are the pros & cons over vanilla s3?

mildbyte 6y ago
We use it in anger at Splitgraph [0] to let people upload/download datasets and Postgres table snapshots (instead of storing them directly in S3). Pros: * Less platform-dependent. By self-managing it, we can also run deploys to GCP / Azure / Scaleway / other providers without writing a separate adapter for e.g. Azure Blob Storage. * Python API [1] much more pleasant to use than boto3 (and can speak to normal S3). It doesn't do everything that boto3 does, but it supports everything we need (e.g. pre-signed URLs). * minio server itself supports a large chunk of S3's functionality (e.g. SELECT API / AssumeRole / bucket versioning) * Don't pay per request and for egress: this was a big deal since people might want to download large amounts of data from us (or make a bunch of small requests to download/upload a subset of data). Cons: * Have to manage own infrastructure. We run it on managed VMs so it's semi-managed, but we still have to provision block storage, set up backup policies etc. * In a similar vein, scaling and availability all have to be DIY [2]. We haven't run into situations yet where Minio would be the bottleneck, but it might be something to keep in mind. * Obviously not as seamless: you don't get things like Glacier or integration with other IAM. [0] https://www.splitgraph.com/ https://www.splitgraph.com/ [1] https://github.com/minio/minio-py https://github.com/minio/minio-py [2] https://docs.min.io/docs/distributed-minio-quickstart-guide.html https://docs.min.io/docs/distributed-minio-quickstart-guide....
ddorian43 6y ago
It doesn't work for a lot of small files scenario, there is too much overhead. You can't +1 new machines, you need to add whole clusters. If you ask too many questions you get invited for a premium subscription. You can't add +1 server, you need to add in clusters. There is no public big production scenario of 10PB+. There is no community of big production users (like there is with ceph).
sofixa 6y ago
> What are the pros & cons over vanilla s3 Mostly that you aren't bound to AWS IMHO. You can run it on-prem, or in a cloud provider with no object storage service.
mrweasel 6y ago
We opted against Minio, mostly because we had a storage solution that provide S3 api, but also because managing Minio seemed painful. It’s just not clearly versioned, we didn’t figure out how you where suppose to rotate cerificates and documentation isn’t great.
jnwatson 6y ago
My n=1 indirect experience as a consumer of someone else setting it up. The devops team had a lot of trouble keeping Minio up and running. Performance was not great. The main pro and con is that it is self-hosted.

mildbyte 6y ago

We use it in anger at Splitgraph [0] to let people upload/download datasets and Postgres table snapshots (instead of storing them directly in S3). Pros: * Less platform-dependent. By self-managing it, we can also run deploys to GCP / Azure / Scaleway / other providers without writing a separate adapter for e.g. Azure Blob Storage. * Python API [1] much more pleasant to use than boto3 (and can speak to normal S3). It doesn't do everything that boto3 does, but it supports everything we need (e.g. pre-signed URLs). * minio server itself supports a large chunk of S3's functionality (e.g. SELECT API / AssumeRole / bucket versioning) * Don't pay per request and for egress: this was a big deal since people might want to download large amounts of data from us (or make a bunch of small requests to download/upload a subset of data). Cons: * Have to manage own infrastructure. We run it on managed VMs so it's semi-managed, but we still have to provision block storage, set up backup policies etc. * In a similar vein, scaling and availability all have to be DIY [2]. We haven't run into situations yet where Minio would be the bottleneck, but it might be something to keep in mind. * Obviously not as seamless: you don't get things like Glacier or integration with other IAM. [0] https://www.splitgraph.com/ https://www.splitgraph.com/ [1] https://github.com/minio/minio-py https://github.com/minio/minio-py [2] https://docs.min.io/docs/distributed-minio-quickstart-guide.html https://docs.min.io/docs/distributed-minio-quickstart-guide....

ddorian43 6y ago

It doesn't work for a lot of small files scenario, there is too much overhead. You can't +1 new machines, you need to add whole clusters. If you ask too many questions you get invited for a premium subscription. You can't add +1 server, you need to add in clusters. There is no public big production scenario of 10PB+. There is no community of big production users (like there is with ceph).

sofixa 6y ago

> What are the pros & cons over vanilla s3 Mostly that you aren't bound to AWS IMHO. You can run it on-prem, or in a cloud provider with no object storage service.

mrweasel 6y ago

We opted against Minio, mostly because we had a storage solution that provide S3 api, but also because managing Minio seemed painful. It’s just not clearly versioned, we didn’t figure out how you where suppose to rotate cerificates and documentation isn’t great.

jnwatson 6y ago

My n=1 indirect experience as a consumer of someone else setting it up. The devops team had a lot of trouble keeping Minio up and running. Performance was not great. The main pro and con is that it is self-hosted.