5 ms·

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incident

by jorams 6y ago

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Storage has been very reliable and fast.

saddlerustle 6y ago
This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky. Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.
- speedgoose 6y ago
  That's not a very common use case though. Most companies don't need to DDOS their cloud provider.
  - jrockway 6y ago
    25Gbps is 250 cable Internet users downloading the release of your new software. What you call a DDOS is actually a woefully underwhelming instantaneous transfer rate.
    - speedgoose 6y ago
      You should put your objects storage behind a CDN if you expect that many downloads. That's much cheaper and faster.
    - Epixors 6y ago
      If you're distributing releases (or anything, really) users should not have to go directly to your object store...
- dijit 6y ago
  AWS S3 buckets have a limit of 5gb/s access to a single bucket. Unless that limitation has changed in the last couple of years; I can easily make a system beat that, if that's the requirement.. and ultimately it does come down to understanding requirements. :\ I think generally people forget that cloud is just computers too, it's really not anything special, and amazon/google/microsoft are solving the general case (and, doing so well, actually) but this comes at a high premium.
  - duggan 6y ago
    It's 25Gbps now[1]. I have a rule of thumb: if I haven't refreshed something I think I know about AWS for more than half a year, it's probably out of date. [1]: https://aws.amazon.com/premiumsupport/knowledge-center/s3-maximum-transfer-speed-ec2/ https://aws.amazon.com/premiumsupport/knowledge-center/s3-ma...
    - saddlerustle 6y ago
      That the throughput available to a single EC2 instance. S3 as a whole will do hundreds of Gbps no problem.
- fulafel 6y ago
  There's options outside SaaS object storage though, eg running your own MinIO or Ceph, + DAS (or SAN) at your colo. Yes it's some hassle but if you have significant expenses from SaaS it might be worth checking out.
csunbird 6y ago
is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.
- jonatron 6y ago
  I tried it for a small personal project, just for somewhere to put some small images. On pages of about 30 images, usually one of them just wouldn't load. It didn't look reliable to me either.
- barake 6y ago
  Provisioning of new Spaces was recently disabled in some regions while they add capacity. Vaguely concerning IMO and has caused me to seriously reconsider using DO Spaces for a new product I'm building. https://www.digitalocean.com/docs/release-notes/upcoming/spaces-fra1-nyc3/ https://www.digitalocean.com/docs/release-notes/upcoming/spa...
  - csunbird 6y ago
    I do know that they have limited the creation and looking at the docs for limitation, it seems like they have a 750 requests per second limit per space, which might not scale well. My idea was to create multiple spaces to distribute a load, e.g. sharding per customer, but if the availability is not that good, I might have to re-consider our choice. The main reason we wanted to work with DO is that the egress traffic cost is reasonable, unlike AWS or Google Cloud where it is questionably high.
- jorams 6y ago
  It was that bad 2 years ago, in the AMS3 region. I don't know how it is these days, but I expect after 2 years it has matured at least a little bit. They also now include a CDN, which might improve things a bit if it reduces the load on the buckets behind it. Honestly it soured my opinion on DO a bit. I'd previously had a droplet there for several years, which never seemed to experience any issues at all.
  - 6y ago
jrockway 6y ago
I run my own Docker registry for my personal projects backed by DO Spaces. It was so slow that I couldn't actually believe it, and had to instrument the registry code extensively to account for all the time spent in the storage API, to see what the actual problem was. Turns out Spaces is really, really slow.

by jorams 6y ago

saddlerustle 6y ago

This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky. Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.

speedgoose 6y ago
That's not a very common use case though. Most companies don't need to DDOS their cloud provider.
- jrockway 6y ago
  25Gbps is 250 cable Internet users downloading the release of your new software. What you call a DDOS is actually a woefully underwhelming instantaneous transfer rate.
  - speedgoose 6y ago
    You should put your objects storage behind a CDN if you expect that many downloads. That's much cheaper and faster.
  - Epixors 6y ago
    If you're distributing releases (or anything, really) users should not have to go directly to your object store...
dijit 6y ago
AWS S3 buckets have a limit of 5gb/s access to a single bucket. Unless that limitation has changed in the last couple of years; I can easily make a system beat that, if that's the requirement.. and ultimately it does come down to understanding requirements. :\ I think generally people forget that cloud is just computers too, it's really not anything special, and amazon/google/microsoft are solving the general case (and, doing so well, actually) but this comes at a high premium.
- duggan 6y ago
  It's 25Gbps now[1]. I have a rule of thumb: if I haven't refreshed something I think I know about AWS for more than half a year, it's probably out of date. [1]: https://aws.amazon.com/premiumsupport/knowledge-center/s3-maximum-transfer-speed-ec2/ https://aws.amazon.com/premiumsupport/knowledge-center/s3-ma...
  - saddlerustle 6y ago
    That the throughput available to a single EC2 instance. S3 as a whole will do hundreds of Gbps no problem.
fulafel 6y ago
There's options outside SaaS object storage though, eg running your own MinIO or Ceph, + DAS (or SAN) at your colo. Yes it's some hassle but if you have significant expenses from SaaS it might be worth checking out.

csunbird 6y ago

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

jonatron 6y ago
I tried it for a small personal project, just for somewhere to put some small images. On pages of about 30 images, usually one of them just wouldn't load. It didn't look reliable to me either.
barake 6y ago
Provisioning of new Spaces was recently disabled in some regions while they add capacity. Vaguely concerning IMO and has caused me to seriously reconsider using DO Spaces for a new product I'm building. https://www.digitalocean.com/docs/release-notes/upcoming/spaces-fra1-nyc3/ https://www.digitalocean.com/docs/release-notes/upcoming/spa...
- csunbird 6y ago
  I do know that they have limited the creation and looking at the docs for limitation, it seems like they have a 750 requests per second limit per space, which might not scale well. My idea was to create multiple spaces to distribute a load, e.g. sharding per customer, but if the availability is not that good, I might have to re-consider our choice. The main reason we wanted to work with DO is that the egress traffic cost is reasonable, unlike AWS or Google Cloud where it is questionably high.
jorams 6y ago
It was that bad 2 years ago, in the AMS3 region. I don't know how it is these days, but I expect after 2 years it has matured at least a little bit. They also now include a CDN, which might improve things a bit if it reduces the load on the buckets behind it. Honestly it soured my opinion on DO a bit. I'd previously had a droplet there for several years, which never seemed to experience any issues at all.
- 6y ago

jrockway 6y ago

I run my own Docker registry for my personal projects backed by DO Spaces. It was so slow that I couldn't actually believe it, and had to instrument the registry code extensively to account for all the time spent in the storage API, to see what the actual problem was. Turns out Spaces is really, really slow.