6 ms·
Spaces: Scalable Object Storage on DigitalOcean
- stephenr 9y agoAnyone know what this uses under the hood? The reference to erasure coding specifically reminds me of Minio..
- misframer 9y agoMaybe Ceph.
- DonbunEf7 9y agoOr Tahoe-LAFS, although Tahoe's encoding doesn't leave any keys with the provider, but only with the uploader.
- stephenr 9y agoRight, I guess I assumed they would use something that already natively supports the S3 API, which Tahoe doesn't AFAIK?
- DonbunEf7 9y agoActually! (Full disclosure: I sell access to a Tahoe-LAFS grid.) The Tahoe team has recently been working to merge the "cloud-backend" branch, which would open the door to storing shares on grids by interfacing directly with S3, GSE, etc. This technology's been battle-tested by Least Authority Enterprises, and at least Matador Cloud has indicated a desire to adopt it as well. Other Tahoe vendors are probably sitting up and taking notice too.
- stephenr 9y agoUm, you're talking about using s3 as a storage backend (I think?) whereas im talking about exposing data over an s3 compatible API (like minio does)
- heavymark 9y agoCurious why they would publish tutorial for Transmit 4 and not Transmit 5. It says Transmit 5 tutorial will come later but not when. One would think they would have the tutorial for 5, and then later release one for people still on Transmit 4. Unless Transmit 5 doesn't support it yet? Also a bit weird you have to use S3 to connect to it even though it isn't S3, unless this new feature is S3 but wrapped in Digital Ocean interface?
- jawngee 9y agoA lot of storage services have S3 compatible APIs. Minio, Google Cloud Storage (when in interop mode) also can be accessed with S3 libraries/SDKs (you have to change the endpoint for it to work). And they aren't 100% compatible. I believe that DO doesn't support headObject, bucketExists and creating presigned upload URLs.
- stephenr 9y agoUnfortunately the s3 API has become a common thing with blob storage services/software. It means they can support any apps/services that currently rely on s3 without little more than a s3 host/URL change.
- beardicus 9y ago(DO employee here) It's S3-compatible. Agreed that it's a bit strange in Transmit and other clients to have to first select "S3" when connecting to something else entirely... but that's the world we live in. Spaces isn't working with Transmit 5 yet. I don't have the technical details, but it's a known issue that's being worked on. I'll write a Transmit 5 tutorial as soon as it works.
- tombrossman 9y agoUsing AWS's simple calculator[0] for S3, I get a figure of $97.82 for 250GB storage and 1TB data transfer out per month. DO's price is $5. I know AWS is considered more expensive but that's a huge difference - is my calculation wrong here? [0]http://calculator.s3.amazonaws.com/index.html http://calculator.s3.amazonaws.com/index.html
- purvis 9y agoDo you have something else in your estimate? It shows as $5.63 for me.
- slyzmud 9y agoI was chosen to use the preview and the pricing was originally was $0.02 storage and $0.01 transfer without the first $5. I guess that most of the people spent a lot less than $5. Increasing the minimum spend to 5 should give them a great profit for the people that don't use it that much. I don't know what will happen if a lot of people start using it a lot.
- rkangel 9y ago> I don't know what will happen if a lot of people start using it a lot. That's almost always a nice problem to have. If it was profitable in small scale usage, then it's usually profitable in widespread usage once economies of scale have been applied.
- joshribakoff 9y agoYes Digital Ocean / Linode bandwidth is 40x cheaper than AWS. Amazon also charges extra for a support plan so you can open a ticket, and inform them of their bugs. And we have found bugs in AWS infrastructure that still remains unfixed, because we have a disproportionate amount of IE users. They essentially forced us to stand up a linode proxy infront of s3 & create our own CDN, since theirs did not work for our users. In the process we saved 4,000% on our bandwidth.
- minibob 9y agowhat is differance beetwen cdn and object storage? i thiink cdn is more cheap. am i wrong?
- riobard 9y agoInteresting pricing: Base $5/month for 250GB storage and 1TB outbound traffic. No cost for requests. Additional storage is $0.02/GB/month and additional outbound traffic is $0.01/GB. Compared to BackBlaze B2: $0.005/GB/month for storage and $0.02/GB for traffic, plus tiered API call fees (see https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html) So Spaces is better for serving assets (lower traffic cost) and B2 is better for archiving data (lower storage cost).
- minxomat 9y agoActually I'd say that managed OpenStack Swift is a better solution than B2, if only marginally more expensive. Take e.g. OVH's Swift pricing[1], which is $0.011 per GB (stored and outgoing). I found Swifts latency (at least on the OVH cluster, with the OVH CDN) to be much better than B2. Swift also satisfies my paranoia about having a remote storage system I could setup on-prem. Also, more storage regions, much better network infra, actual access to software running on the cluster (Swift, not some proprietary API). YMMV [1] - https://www.ovh.com/us/public-cloud/storage/object-storage/ https://www.ovh.com/us/public-cloud/storage/object-storage/
- filleokus 9y agoSomewhat off-topic but: What is the cheapest cloud storage out there? I don't care about multi-region or extreme durability, I would just like to store encrypted blobs containing easily re-creatable data. I need reasonable download speeds however.
- slyzmud 9y agoI currently use serverhub.com I doesn't have a S3 like service but they offer a VPS with 500GB and 1TB of transfer for $5. I have an orphaned plan of 125Gb of storage for $15/year. Super recommended.
- mastax 9y agoBackblaze B2 claims to be the cheapest. I don't know if they are, but they're really cheap and it looks like a great service.
- cheapsteak 9y agoB2 charges $0.02/GB downloaded, 0.005/GB stored DO charges $0.01/GB downloaded, 0.020/GB stored B2 also has a free tier for the first 10 GB of storage, 1 GB of daily downloads. DO charges $5/month minimum
- tyingq 9y agoOVH's object storage offering is probably worth looking at: https://www.ovh.com/us/public-cloud/storage/object-storage/ https://www.ovh.com/us/public-cloud/storage/object-storage/ Storage: $0.0112/month/GB, Outgoing traffic: $0.011/GB
- tyingq 9y agoI wonder what the next add on piece will be. There seems to be a progression here to position themselves as an AWS alternative for budget minded customers. They already have a sort of "cloud formation" like API for provisioning nodes, a load balancer, and now object storage. Maybe a managed database offering? Or managed K8S? Or a more clear "region / availability zone" strategy?
- wastedhours 9y agoI'm hoping for managed database offering next. Not being a sysadmin/devops, always scared launching any kind of side project because of security concerns, if DO adds some managed/context specific nodes to their lineup, will make it much easier to wind things up.
- lux 9y agoCongrats on the launch! Love using DO for VMs and this really rounds out the offering. Since Google started offering strong consistency that's been a really tempting feature. We often want to upload something for another user to immediately access, and having that extra delay or potential 404 with other services sucks. But Google's pricing is nowhere near B2 or now DO's Spaces. Any thoughts on eventually adding strong consistency to Spaces?
- kyledrake 9y agoHas anyone ran any benchmarks on this yet? Latency is always an issue with using these for production static file serving over HTTP (even if it's just feeding a proxy network). Another feature it would be nice to see is accidental deletion protection of some kind. S3 does this with versioning, and it's an important feature lacking on most of the alternatives: > Versioning allows you to preserve, retrieve, and restore every version of every object stored in an Amazon S3 bucket. Once you enable Versioning for a bucket, Amazon S3 preserves existing objects anytime you perform a PUT, POST, COPY, or DELETE operation on them. By default, GET requests will retrieve the most recently written version. Older versions of an overwritten or deleted object can be retrieved by specifying a version in the request.
- JosephLark 9y agoInteresting offering. I'm wondering how people feel about the $5 minimum? At 250GB storage and 1TB traffic, plenty of people using much less than 250GB and 1TB traffic would be better served with a pay for what you use model.
- jlgaddis 9y agoAs an individual, $5 is easily worth it so I don't have to pay attention to how much I have in S3 and how much I download. I don't have to worry about getting a huge bill at the beginning of the month.
- kitotik 9y agoI love this company and this new offering. Such a pleasure to work with - from both a developer and customer perspective. I only wish the other regions would roll out quicker(sfo please!)
- mstade 9y agoWith no costs for uploads and $5 per every 250GB it sounds like this is actually probably a pretty nice deal for personal archiving. Of course if you're also downloading a bunch of that stuff you'll hit that 1TB transfer ceiling pretty quickly, but it sounds to me like it could be a reasonably cheap backend for a backup solution – or am I missing something?
- unvs 9y agoFrom the introduction document: [1] > Spaces currently supports v2 of pre-signed URL functionality. Tools and libraries that only support v4 of pre-signed URL functionality will not work. I had really hoped this would be in place for the launch, but I guess this wasn't a priority. Looking forward to this + EU locations! [1] https://www.digitalocean.com/community/tutorials/an-introduction-to-digitalocean-spaces https://www.digitalocean.com/community/tutorials/an-introduc...
- jlgaddis 9y agoThis looks like a much cheaper way (than AWS S3) to use Arq to backup my MBP.
- ernsheong 9y agoAny experience/data on rate limits? I heard B2 rate limits a lot. I need to pour data in at will at a moment's notice without limits.