5 ms·

Or, run Riak with their S3 compatibility layer. Riak is extremely stable and the work Basho has done to make a truly robust distributed database is significant.

by Ixiaus 10y ago

Or, run Riak with their S3 compatibility layer. Riak is extremely stable and the work Basho has done to make a truly robust distributed database is significant. http://docs.basho.com/riak/cs/2.1.1/ http://docs.basho.com/riak/cs/2.1.1/

merb 10y ago
guess it's possible, but riak is not designed to run on a single node. Guess even basho suggests using at least a 5 node cluster.
- unlocksmith 10y ago
  Minio is deliberately designed this way. Cloud native applications require strict multitenancy. Minio's approach is to build just enough to meet a single tenant's requirement. Deploy one minio server per tenant or user or customer .. whichever fits you the best. This will allow you to upgrade, customize or bug fix in isolation. To replicate for HA, use "mc mirror -watch SOURCE TARGET" command to pair them up. If you have multiple drives (JBOD), you can eliminate RAID or ZFS and use Minio's erasure code to pool them up. Distributed version is also in testing at the moment. It should be out in a month.
  - merb 10y ago
    I know and that. And that's why I find minio interesting. Start with a single node. Raise up to 16.
viraptor 10y ago
Other alternatives: ceph - http://docs.ceph.com/docs/master/radosgw/s3/ http://docs.ceph.com/docs/master/radosgw/s3/ swift - https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#Amazon_S3_REST_API_Compatability https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#A...
- theanalyst 10y ago
  Also ceph (& swift) are known to scale well in prod. clusters with over 30+ PB of data (at least looking at CERN's cluster) and the latest version of RGW does support geographic redundancy for S3 like apis
  - shinydevops 10y ago
    +1 for Ceph. We're running several ~3.5 PB clusters in production. We've not taken advantage of the new RGW features in Jewel, but it works well as an object storage solution.
- dekobon 10y ago
  Don't forget: manta - https://www.joyent.com/manta https://www.joyent.com/manta
- dc2447 10y ago
  CEPH is a volume service not an object storage service. SWIFT is indeed analogous to S3.
  - XorNot 10y ago
    Ceph is object storage first. The volume service is implemented on top of that.
  - viraptor 10y ago
    Come on, I literally linked to a website describing "CEPH OBJECT GATEWAY S3 API"
corobo 10y ago
There's also Skylable's SX Cluster if you use the libres3 daemon with it. Been using it for over a year with no problems. Set, forget, add more nodes when I need more disk. Everyone's got their s3 of choice, always good to have more options on the table. https://www.skylable.com/products/sx/ https://www.skylable.com/products/sx/ https://www.skylable.com/products/libres3/ https://www.skylable.com/products/libres3/
mprev 10y ago
There's also Pithos from Exoscale. Runs on top of Cassandra. Code is Clojure and open source. Http://Pithos.io
ranman 10y ago
Ran Riak CS in production and had constant issues. It's not terrible but it's also not ideal. I would caution against anyone depending on it for mission critical systems. Many of the failure modes are undocumented.
- 0xmohit 10y ago
  Could you elaborate on some of the specific issues you ran into?
Natales 10y ago
+1 on RiakCS. They now call it RiakS2 for kicks. The scalability and reliability of their server is insane. You just can't beat Erlang software in that regard. Unfortunately, Basho has been so successful with their TSDB and KV products that they have basically put S2 on maintenance mode. They are still "supporting" it, but no new features. I was hoping this Minio tool could do something similar, but with a single daemon is a single point of failure. Unacceptable for serious deployments.
- jdboyd 10y ago
  Considering how many serious deployments still use non-clustered NASs, a single node object store seems equally reasonable.
- owyn 10y ago
  That sounds pretty nice. If it works does it need new features? :)
- bramd 10y ago
  Another interesting project written in Erlang is LeoFS: http://leo-project.net/leofs/ http://leo-project.net/leofs/
  - bandris 10y ago
    Worth adding that LeoFS is being used in production by Rakuten for years now. Still not a widely known project for some reason. http://www.slideshare.net/rakutentech/scaling-and-high-performance-storage-system-leofs http://www.slideshare.net/rakutentech/scaling-and-high-perfo...
dc2447 10y ago
Dont you have to pay for an enterprise licence if you want multi region/datacentre/AZ?
- deleted 10y ago
  [deleted]

by Ixiaus 10y ago

merb 10y ago

guess it's possible, but riak is not designed to run on a single node. Guess even basho suggests using at least a 5 node cluster.

unlocksmith 10y ago
Minio is deliberately designed this way. Cloud native applications require strict multitenancy. Minio's approach is to build just enough to meet a single tenant's requirement. Deploy one minio server per tenant or user or customer .. whichever fits you the best. This will allow you to upgrade, customize or bug fix in isolation. To replicate for HA, use "mc mirror -watch SOURCE TARGET" command to pair them up. If you have multiple drives (JBOD), you can eliminate RAID or ZFS and use Minio's erasure code to pool them up. Distributed version is also in testing at the moment. It should be out in a month.
- merb 10y ago
  I know and that. And that's why I find minio interesting. Start with a single node. Raise up to 16.

viraptor 10y ago

Other alternatives: ceph - http://docs.ceph.com/docs/master/radosgw/s3/ http://docs.ceph.com/docs/master/radosgw/s3/ swift - https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#Amazon_S3_REST_API_Compatability https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#A...

theanalyst 10y ago
Also ceph (& swift) are known to scale well in prod. clusters with over 30+ PB of data (at least looking at CERN's cluster) and the latest version of RGW does support geographic redundancy for S3 like apis
- shinydevops 10y ago
  +1 for Ceph. We're running several ~3.5 PB clusters in production. We've not taken advantage of the new RGW features in Jewel, but it works well as an object storage solution.
dekobon 10y ago
Don't forget: manta - https://www.joyent.com/manta https://www.joyent.com/manta
dc2447 10y ago
CEPH is a volume service not an object storage service. SWIFT is indeed analogous to S3.
- XorNot 10y ago
  Ceph is object storage first. The volume service is implemented on top of that.
- viraptor 10y ago
  Come on, I literally linked to a website describing "CEPH OBJECT GATEWAY S3 API"

corobo 10y ago

There's also Skylable's SX Cluster if you use the libres3 daemon with it. Been using it for over a year with no problems. Set, forget, add more nodes when I need more disk. Everyone's got their s3 of choice, always good to have more options on the table. https://www.skylable.com/products/sx/ https://www.skylable.com/products/sx/ https://www.skylable.com/products/libres3/ https://www.skylable.com/products/libres3/

mprev 10y ago

There's also Pithos from Exoscale. Runs on top of Cassandra. Code is Clojure and open source. Http://Pithos.io

ranman 10y ago

Ran Riak CS in production and had constant issues. It's not terrible but it's also not ideal. I would caution against anyone depending on it for mission critical systems. Many of the failure modes are undocumented.

0xmohit 10y ago
Could you elaborate on some of the specific issues you ran into?

Natales 10y ago

+1 on RiakCS. They now call it RiakS2 for kicks. The scalability and reliability of their server is insane. You just can't beat Erlang software in that regard. Unfortunately, Basho has been so successful with their TSDB and KV products that they have basically put S2 on maintenance mode. They are still "supporting" it, but no new features. I was hoping this Minio tool could do something similar, but with a single daemon is a single point of failure. Unacceptable for serious deployments.

jdboyd 10y ago
Considering how many serious deployments still use non-clustered NASs, a single node object store seems equally reasonable.
owyn 10y ago
That sounds pretty nice. If it works does it need new features? :)
bramd 10y ago
Another interesting project written in Erlang is LeoFS: http://leo-project.net/leofs/ http://leo-project.net/leofs/
- bandris 10y ago
  Worth adding that LeoFS is being used in production by Rakuten for years now. Still not a widely known project for some reason. http://www.slideshare.net/rakutentech/scaling-and-high-performance-storage-system-leofs http://www.slideshare.net/rakutentech/scaling-and-high-perfo...

dc2447 10y ago

Dont you have to pay for an enterprise licence if you want multi region/datacentre/AZ?

deleted 10y ago
[deleted]