11 ms·

Highly Available Block Storage

262 points by dineshp2 10y ago

lamarkia 10y ago
I benchmarked the new block storage and it is not that faster than the virtio disk.
- Spooks 10y ago
  can we see some more info on your benchmarks? what was the performance difference? Kind of need to backup your statement
- daveguy 10y ago
  Um. Can you provide context? What virtio setup? How much faster? How are you accessing each? This is the reason benchmarks are often worse than useless.
skrowl 10y ago
I like their straight forward pricing. $0.10 USD per GB per month. No IOPS limits. That said, how do you prevent a rogue droplet from going crazy and hogging up all of the SSD I/O?
- askmike 10y ago
  They might limit IO speed (based on usage of previous 24h for example). Like how most cheaper AWS EC2 instances get their CPUs throttled after you're running CPU heavy stuff.
- zbjornson 10y ago
  Unless I missed it, I saw "no need for complicated formulas to determine the overall cost for transactions or IOPs limit," which I do not read as "no IOPS limit." I was looking but could not find any word on performance.
  - brianwawok 10y ago
    I read it as they either silently limit you, or have a free for all and let the traffic duke it out for iops.
- mdasen 10y ago
  Google doesn't charge for IOPS either, but they do explain how things are limited. For example, speed in MB/s is limited by the instance size so that you don't use the entire network link for your connection to the disk. IOPS are limited by the size of the disk. If you're sharing the disk with other people, you're going to be limited in your IOPS. For example, if you get 10GB of SSD storage, you'll get 300 random IOPS, but another user with 50GB of storage will get 1,500 IOPS because they're renting a larger portion of the disk. If DigitalOcean isn't doing something limiting, they'll have to in order to prevent the situation that you're describing. You don't have to charge for it in order to prevent bad neighbors. https://cloud.google.com/compute/docs/disks/performance https://cloud.google.com/compute/docs/disks/performance
- marcstreeter 10y ago
  There are limits - they are marketing their simplicity by avoiding complicated pricing schemes. This is typical DO, and I see that as a good thing especially if you're just dipping your feet into the concept. The past year or two that we've had block storage at Codero we've been marketing the ability to set your own IOPS with even our smallest blocks (1000/2000/3000 iops). That said I'd be interested to know exactly what performance they offer because they are definitely cheaper.
daveguy 10y ago
Edit3: Remountable/movable flexible storage for DO instances is what this gives you and it's kind of pricey. The comparison to B2 is not valid. Leaving the original mess for posterity. --- TWENTY times (edit) the price as B2 from Backblaze ($.10 vs $.005 per GB per month). It is one of the more expensive ones. But that gets you two things: * (moot See edit 2) SSD! (significant iops improvement) * (moot See edit 2) No transactional costs! (not sure if just between digital ocean instances, but they say none) Improved performance and no transaction costs MAY (edit) be worth it for some applications. Edit1: made it an order of magnitude cheaper in my head after looking at backblaze. It is no where close to the same price. Thank you for the catch, scq! Edit2: and I'm just all kinds of off on this. Block not object storage. Essentially storage you can mount and move between digitalocean instances. That makes no transaction costs moot. You still have to get data out of the instance. Thank you all for quickly catching how backwards this post was! I need coffee.
- koolba 10y ago
  > Twice the price as B2 from Backblaze ($.010 vs $.005 per GB per month). I believe B2 is an object store, not a block store (like this). The AWS analogy is block store is EBS, object store is S3.
- AYBABTME 10y ago
  B2 is not block storage tho, it's object storage. Different products. =]
- crb 10y ago
  B2 is blob storage, like Google Cloud Storage or Amazon S3. What they have launched is SSD block storage, like Google persistent disks or Amazon EBS. (The TechCrunch article also got this confused.)
  - askmike 10y ago
    > Amazon EBS. Not more like amazon EFS?
    - 10y ago
dastbe 10y ago
Don't be confused: the article makes the mistake of comparing DOs new block storage service with other companies object stores. EBS is the competitor to this, not S3. Same for gce persistent disks and azure drives. Unfortunately this means the pricing comparison is just wrong.
- BrandonBradley 10y ago
  Whoops
- priteshjain 10y ago
  Ohh
koolba 10y ago
This has been a much requested feature and I'm sure it will be very popular. I'm still reminded of this quote though: "He was a bold man that first ran a production database on a brand new block storage service!"
drtse4 10y ago
A bit pricey for the long term but great if you just need to add some disk space to your vm and don't need the other improvements more expensive vms give you. I use DO mostly to compile stuff on Linux when i don't have access to a physical server, and storage size is always a problem.
- marcstreeter 10y ago
  The purpose of the block storage in this instance isn't about giving your vm/droplet more space. It's separation. That way any data that's on that device can be attached to another vm/droplet. It probably would be more cost effective just to upgrade the vm/droplet if space were a concern. It's at least how we've marketed the same feature for the past year or two via Codero's portal. Not to say I don't like how DO has entered the space: keeping it simple.
  - walkertraylor 10y ago
    Exactly! It's about having an easy upgrade path, generally reducing the amount of work for common operations, and for more flexibility engineering your cloud architecture. It's nice that it happens to be SSD, but highest performance or lowest cost per GB isn't necessarily the only cost saving factor. This a main reason I still use Amazon AWS: I can create an instance and if it doesn't perform, upgrade it until it does. Then when I'm finished, kill the instance and save the volume. Next time I need it, just create the instance for the job, perhaps at spot pricing, then kill it again.
    - drtse4 10y ago
      What you describe was already possible on DO using snapshots (that can be stored for free), and that's what I usually do too.
andybak 10y ago
This helps me with a nicer deployment setup. I was always keen on 'rebuild from scratch' rather than 'update stuff and hope you're idempotent and have captured all changes' but transitory data was always the problem. Now I can start building a new updated droplet and the only downtime will be that needed to detach and reattach the block storage containing the db etc. Anyone see a flaw in this? (I know there are other ways to achieve similar benefits - my files could be on S3 and the database could be a separate droplet etc but these introduced various drawbacks and added complexity)
- Jedd 10y ago
  > Anyone see a flaw in this? Perhaps not a flaw, but some issues with your setup are implied. If you're rebuilding from scratch because you're not sure that you can update things, then you're probably in need of a configuration management tool (I'm a big fan of saltstack[1], mostly because I don't like Ruby or DSL's, but there's lots of options out there[2]) If you're worried you're going to lose transitory data, it sounds like you don't have a trusted and tested backup/archival/recovery process in place. So having it stored on a single EBS / DO BS / etc means you're still exposed. If you're rebuilding and rolling data over, in this scenario, I'd be copying, rather than relocating, any precarious data repositories. [1] https://saltstack.com/ https://saltstack.com/ [2] https://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software https://en.wikipedia.org/wiki/Comparison_of_open-source_conf...
  - vidarh 10y ago
    I tend to avoid those config management tools other than for basic bootstrapping exactly because while you can use those too to recreate from scratch, when you don't do that, you leave the door open for undocumented, unknown state, since most of them basically take a system in an unknown-but-hopefully-mostly-consistent state and try to bring them to a known state. But they'll only be in a known state in that case if your setup is extremely comprehensive. In practice I've seen too many config management tools where long running servers have ended up in unknown states because changes have been applied, and subsequently changes have either been made outside of the toolchain, or changes have been done to the config in ways that doesn't let the tool know what has changed, or the tools simply doesn't have a way of comparing machine states without comprehensively enumerating everything on the server (e.g. people running Ansible playbooks that adds X, subsequently removing the requirement X from the playbook, and going about without considering whether or not X will interact with Y which they've added later). As a result, I see rebuilding from scratch as largely orthogonal to whether or not you use a configuration management tool or e.g. build VM images that you replace wholesale, or whatever you do: You should rebuild from scratch regularly, as coupled with a test-suite it's the only realistic way of knowing whether or not you've left anything out of your build process. My biggest caveat with config management systems is that they tend to end up encouraging live changes to a setup, instead of a build-test-deploy cycle. Sometimes that's necessary, but to me that's a last resort.
happyslobro 10y ago
Setup: find it's name, format and mount it, as if you were adding an SSD to a desktop. https://www.digitalocean.com/community/tutorials/how-to-use-block-storage-on-digitalocean https://www.digitalocean.com/community/tutorials/how-to-use-...
happyslobro 10y ago
I don't suppose DO or anyone else is already working on an Ansible addon for this?
wiremine 10y ago
Spun one up and ran some quick numbers on a 100GB volume: root@ubuntu-1gb-nyc1-01:~# time dd if=/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 of=test.dat bs=1024 count=10000000 10000000+0 records in 10000000+0 records out 10240000000 bytes (10 GB) copied, 58.0655 s, 176 MB/s real 0m58.248s user 0m2.608s sys 0m41.604s Some quick observations: * Easy to add one when creating a droplet; by default they let you create volumes with these sizes: 100GB, 250GB, 500GB, 1000GB, 1.95TB; it's also really easy to create your own size. * You can resize in any increments; took about 4 seconds to go from 100GB to 110GB with no downtime; you obviously need to resize/manage the mounted volume yourself. * [Edit 1] Deleting the droplet does NOT destroy the volume. Worth keeping in mind when you spin them up/down. * [Edit 2] Remounting an existing volume to a new droplet was quick and painless.
- olavgg 10y ago
  Thank you for telling us these details. I can't help but I note that you benchmark the drive with a block size of 1024. Minimum block size of a modern SSD is 4096, anything lower would just cause unnecessary load. Also if you want to benchmark a drive/block storage I highly recommend using fio.
  - wiremine 10y ago
    Thx for the tip, I'll give that a try!
    - ymse 10y ago
      fio --direct=1 --rw=write --ioengine=libaio --runtime=300 --bs=4k --numjobs=4 --iodepth=32 --size=2G --name=4096-direct --filename=/dev/vdb --group_reporting This is my "dd" test for any storage system. Filename can be a regular file; you can also pass --directory instead. I would love to see numbers for both this test as well as with --bs=4M. Also experiment with --rw=rw|randrw|read :) (and make sure your device is large enough to hold numjobs*size)
- Thaxll 10y ago
  Don't use dd for those tests it's really bad, especially on VMs.
deleted 10y ago
[deleted]
simos 10y ago
Some early benchmarks about the new block storage, https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-on-digitalocean-with-block-storage/ https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-... I did not get good speeds and I am wondering why that may be...
- zbjornson 10y ago
  > The immediate benefits are that the latency is much lower with the new block storage I think you might have misread the units: locally attached 50105 us (50 ms) vs block 546 ms. The throughput numbers are on par with AWS and GCP block storage. This seems reasonable aside from the high latency.
scurvy 10y ago
What's the backend? Ceph?
- marcstreeter 10y ago
  Ceph is not block storage -- ceph is object storage
  - vruiz 10y ago
    It's actually both, and a file system.
    - marcstreeter 10y ago
      Oh I guess I thought ceph was all about eventual consistency. I didn't know it was strongly consistent like what I expect from block storage.
    - frabbit 10y ago
      Not really. It's object storage with other stuff layered on top of it with varying degrees of success.
  - frabbit 10y ago
    Ceph can use RBD (rados block device) to provide block storage.
  - notacoward 10y ago
    It's fundamentally objects, but not S3-style objects. It's more NASD/T10 objects, with rich semantics including partial writes and strong consistency. Implementing block storage on top of this layer (called RADOS) is trivial. Implementing S3-style objects is a bit more work, and implementing a filesystem was downright hard, but both have been done. Disclaimer: I'm a Gluster developer, which isn't quite the same as being a Ceph developer but we do talk to one another. ;)
    - scoot 10y ago
johnwheeler 10y ago
I love how DO focuses on what matters the most: Inexpensive VMs and scalable block storage. If I had to pick two, those would be them!
- brianwawok 10y ago
  Well except being 4 years late to the block storage game? Seems they have an uphill fight. Aws and gce match them on low end droplet price and offer much more. No local ssd but not sure what % of apps really need local ssd. DO effectively forces you to pay for local ssd for all of your servers.
  - velodrome 10y ago
    Well, they beat Linode.. With DO, at least there is an additional option available for users. With GCE and AWS, the outbound bandwidth is expensive. 1TB = $90 (AWS, GCE) vs 1TB included (Linode, DO).
    - e12e 10y ago
      This. I'm always a little confused by how people even evaluate the big three clouds, with prices per request and all bandwidth "not included". It's so strange coming from cheap dedicated servers with typically 10tb bandwidth/month "included". I mean, what do people do in the cloud that requires 100gb of storage, and insignificant transfer?
    - brianwawok 10y ago
      Sure, and certain classes of problems are cheaper on DO (webscraper, botnet, etc). When they launched they just felt so far ahead of everyone else (1/4 the price of linode for SSD!). But price has been static for 5? years, so feeling lessing exciting these days.
bjacobel 10y ago
Reminder that just a few weeks ago DigitalOcean rolled over on one of their customers and took down 38,000 websites after receiving a claim of infringement from the NRA against a parody site hosted on surge.sh: http://motherboard.vice.com/read/nra-complaint-takes-down-38000-websites http://motherboard.vice.com/read/nra-complaint-takes-down-38...
- treehau5 10y ago
  Just a reminder that most businesses would do this exact same thing in this position and the problem is with the policy and enforcement. Get out and vote for the right people this November.
  - cmdrfred 10y ago
    Find me a viable candidate who will work for the people and I will.
    - unethical_ban 10y ago
      Gary Johnson isn't as insane as the other two.
      - fweespeech 10y ago
        Yeah, tbh, just throwing your vote that way to pollsters may be enough to get him on the debate stage and that is the best you can really do.
      - btilly 10y ago
        Remember the old joke that I can't be a Democrat because I want to spend my own money, and I can't be a Republican because of what I want to spend my money on? If you chuckled at it, you should check Gary Johnson out. For real.
        acehyzer 10y ago
Mister_Snuggles 10y ago
This is EXACTLY the thing that I need for one of my droplets! I love how there is nothing "special" about it - it's just a disk that you can attach to a droplet. I'm sure that under the hood there's some kind of magic going on, but it looks like it's nicely abstracted away. This is what I hoped block storage would turn out to be - here's a block device, use it like one. As soon as this rolls out to the region I've got that droplet in, I'm going to pull the trigger on it. I might even spend the effort to migrate my droplet to a supported region just to get this.
- mikeash 10y ago
  Same here. My $5/month droplet is sufficient for all my needs, except it's a bit limiting on storage. $2/month for an extra 20GB, doubling my storage, would be great. I don't want to migrate anything and I don't care about more CPU or RAM, I just want more space!
  - hashmp 10y ago
    Totally agree, this was their main limitation. Glad they have now solved it.
misframer 10y ago
It's the same price as AWS's General Purpose SSD EBS volumes. https://aws.amazon.com/ebs/pricing/ https://aws.amazon.com/ebs/pricing/
- hashmp 10y ago
  Yep.... Digital Ocean is much more competitive for instances though and out going bandwidth.
  - eropple 10y ago
    At the cost of significantly worse networking (VPC is fantastic), no object storage mechanism, no equivalent to IAM for instance permissioning, and no autoscaling (seriously? in 2016? I had to go look to make sure I wasn't misremembering--that they recommend something like DOProxy should make the lot of them faintly embarrassed). DigitalOcean may have some value at the basement level of compute, but as a professional in this area there is literally no situation where I would use DigitalOcean right now because I value my time. AWS is already laughably cheap, the tooling is overwhelmingly superior, the resources available are better, and you aren't duct-taping together half-solutions and reinventing every wheel. (This is less an endorsement of AWS and more an endorsement of Not DigitalOcean; GCE is more than fine, Azure has some ugly bits around autoscaling that I don't like but you can get by.)
fweespeech 10y ago
$.10/GB block storage is too expensive. https://www.online.net/en/dedicated-server/rpn-san https://www.online.net/en/dedicated-server/rpn-san There are several places you can get ~5 TB for +/-10% of the 1TB price at DO. DO is offering a SAN at Object Store prices. :/
- tomschlick 10y ago
  So is that service. You linked to the non SSD version. Their SSD version is $0.12/GB
3pt14159 10y ago
I have been asking for non-SSD on DO for a long time now. My heart jumped when I saw the HN title, only to be dashed on the rocks. What are us data nerds supposed to do? We want to take 10 terabytes, run a batch process on it, keep the 20TB, then continue with about 5GB of working data until the next month's terabyte comes in, then we want to batch through the 21TB. Right now the price slider doesn't even go up to 21TB, and clicking on the "need more storage button" doesn't go anywhere, but I'm assuming it would be $2100 / month which is more than 3x as expensive than vanilla S3.
- mattbee 10y ago
  At Bytemark we've done network as standard since 2012, so we can do 100GB of higher-capacity storage for £2/month, and all discs are expandible / migrateable online: bigv.io/prices
- boulos 10y ago
  Not being snarky, but at that scale why would you still be using Digital Ocean? Are you processing this dataset with just a single droplet? Are you going to keep each months 10 TiB forever? Something like Nearline is going to be a much better fit even if you process the data and have to pay the $.01/GB retrieval fee. Disclaimer: I work on compute engine (so of course I want your money), but I'm honestly curious.
- mikeash 10y ago
  "All SSD all the time" is kind of DO's thing. There are lots of other VPS providers with round storage if that's what you're after.
  - raiyu 10y ago
    All SSD all the time, we may make that our new tagline. But yeah, we basically look to SSD for everything as that continues to take over market share from spinning disk and when you have to support a product 5-10 years, it's necessary to look ahead, so as SSD prices continue to drop we will eventually switch to SSD for everything and spinning disk will become the equivalent of tape drives in the old days.
Mister_Snuggles 10y ago
I can't wait for this to roll out to more regions. This is EXACTLY the thing I need for some stuff I'm working on!
mwcampbell 10y ago
I think this might be a mistake. Ever since Joyent's commentary on one of the big Amazon EBS failures in 2011 [1] [2] [3], I've been suspicious of all network-attached block storage. Then again, I haven't heard of any big EBS failures recently; I wonder what changed. [1]: https://www.joyent.com/blog/on-cascading-failures-and-amazons-elastic-block-store https://www.joyent.com/blog/on-cascading-failures-and-amazon... [2]: https://www.joyent.com/blog/magical-block-store-when-abstractions-fail-us https://www.joyent.com/blog/magical-block-store-when-abstrac... [3]: https://www.joyent.com/blog/network-storage-in-the-cloud-delicious-but-deadly https://www.joyent.com/blog/network-storage-in-the-cloud-del...
- boulos 10y ago
  Network block storage isn't inherently broken, the initial EBS implementation was frankly just unreliable. We've not had anything like those dark days with Persistent Disk. It's still true that having your storage across the network opens you to networking failures taking out your storage, but the gain in durability and maintenance pays for it (in our case, live migration would just be crazy with local spinning disks, we tried it didn't work). Disclaimer: I work on GCE, and we want your business ;)
  - ngrilly 10y ago
    > in our case, live migration would just be crazy with local spinning disks, we tried it didn't work It looks like Exoscale does live migrations with locally attached SSDs.
    - regularfry 10y ago
      Then they'll run into the same problem anyone does doing that: migrating reasonably-sized block devices across a reasonable network takes an unreasonable length of time. Been there, done that, don't want to sit staring at consoles waiting for live disc migrations ever again.
      - ngrilly 10y ago
cgag 10y ago
Sweet. This makes digitalocean much more appealing as a potential substrate for a kubernetes cluster.
- pstadler 10y ago
  I just migrated my Kubernetes/Rancher stack from NYC2 to NYC1 in order to use block storage. Eager to see whether this plays well with GlusterFS.
ozy23378 10y ago
Going to perform some very basic DD i/o benchmarks using: https://haydenjames.io/web-host-doesnt-want-read-benchmark-vps/ https://haydenjames.io/web-host-doesnt-want-read-benchmark-v... Will post results.
iofj 10y ago
Why does anyone want block storage ? Why not just have an unlimited filesystem ?
aibottle 10y ago
Thank god! Highly Available Block Storage. From Digital Ocean. Great! Now I can finally store all the 300mb/s streaming in on my server. Oh wait. I cannot, because DO cancelled the service again. Bummer.
- cheapsteak 10y ago
  Cancelled which service?
aibottle 10y ago
Thank god! Highly Available Block Storage. From Digital Ocean. Great! Now I can finally store all the 300mb/s streaming in on my server. Oh wait. I cannot, because DO cancelled the service again. Bummer.
mrmondo 10y ago
How is it taking huge cloud providers so long to catch up with things we do self hosted every day? It obviously has to be well engineered, yet it's relatively simple. Woefully poor performance too.

262 points by dineshp2 10y ago

lamarkia 10y ago

I benchmarked the new block storage and it is not that faster than the virtio disk.

Spooks 10y ago
can we see some more info on your benchmarks? what was the performance difference? Kind of need to backup your statement
daveguy 10y ago
Um. Can you provide context? What virtio setup? How much faster? How are you accessing each? This is the reason benchmarks are often worse than useless.

skrowl 10y ago

I like their straight forward pricing. $0.10 USD per GB per month. No IOPS limits. That said, how do you prevent a rogue droplet from going crazy and hogging up all of the SSD I/O?

askmike 10y ago
They might limit IO speed (based on usage of previous 24h for example). Like how most cheaper AWS EC2 instances get their CPUs throttled after you're running CPU heavy stuff.
zbjornson 10y ago
Unless I missed it, I saw "no need for complicated formulas to determine the overall cost for transactions or IOPs limit," which I do not read as "no IOPS limit." I was looking but could not find any word on performance.
- brianwawok 10y ago
  I read it as they either silently limit you, or have a free for all and let the traffic duke it out for iops.
mdasen 10y ago
Google doesn't charge for IOPS either, but they do explain how things are limited. For example, speed in MB/s is limited by the instance size so that you don't use the entire network link for your connection to the disk. IOPS are limited by the size of the disk. If you're sharing the disk with other people, you're going to be limited in your IOPS. For example, if you get 10GB of SSD storage, you'll get 300 random IOPS, but another user with 50GB of storage will get 1,500 IOPS because they're renting a larger portion of the disk. If DigitalOcean isn't doing something limiting, they'll have to in order to prevent the situation that you're describing. You don't have to charge for it in order to prevent bad neighbors. https://cloud.google.com/compute/docs/disks/performance https://cloud.google.com/compute/docs/disks/performance
marcstreeter 10y ago
There are limits - they are marketing their simplicity by avoiding complicated pricing schemes. This is typical DO, and I see that as a good thing especially if you're just dipping your feet into the concept. The past year or two that we've had block storage at Codero we've been marketing the ability to set your own IOPS with even our smallest blocks (1000/2000/3000 iops). That said I'd be interested to know exactly what performance they offer because they are definitely cheaper.

daveguy 10y ago

Edit3: Remountable/movable flexible storage for DO instances is what this gives you and it's kind of pricey. The comparison to B2 is not valid. Leaving the original mess for posterity. --- TWENTY times (edit) the price as B2 from Backblaze ($.10 vs $.005 per GB per month). It is one of the more expensive ones. But that gets you two things: * (moot See edit 2) SSD! (significant iops improvement) * (moot See edit 2) No transactional costs! (not sure if just between digital ocean instances, but they say none) Improved performance and no transaction costs MAY (edit) be worth it for some applications. Edit1: made it an order of magnitude cheaper in my head after looking at backblaze. It is no where close to the same price. Thank you for the catch, scq! Edit2: and I'm just all kinds of off on this. Block not object storage. Essentially storage you can mount and move between digitalocean instances. That makes no transaction costs moot. You still have to get data out of the instance. Thank you all for quickly catching how backwards this post was! I need coffee.

koolba 10y ago
> Twice the price as B2 from Backblaze ($.010 vs $.005 per GB per month). I believe B2 is an object store, not a block store (like this). The AWS analogy is block store is EBS, object store is S3.
AYBABTME 10y ago
B2 is not block storage tho, it's object storage. Different products. =]
crb 10y ago
B2 is blob storage, like Google Cloud Storage or Amazon S3. What they have launched is SSD block storage, like Google persistent disks or Amazon EBS. (The TechCrunch article also got this confused.)
- askmike 10y ago
  > Amazon EBS. Not more like amazon EFS?
  - 10y ago

dastbe 10y ago

Don't be confused: the article makes the mistake of comparing DOs new block storage service with other companies object stores. EBS is the competitor to this, not S3. Same for gce persistent disks and azure drives. Unfortunately this means the pricing comparison is just wrong.

BrandonBradley 10y ago
Whoops
priteshjain 10y ago
Ohh

koolba 10y ago

This has been a much requested feature and I'm sure it will be very popular. I'm still reminded of this quote though: "He was a bold man that first ran a production database on a brand new block storage service!"

drtse4 10y ago

A bit pricey for the long term but great if you just need to add some disk space to your vm and don't need the other improvements more expensive vms give you. I use DO mostly to compile stuff on Linux when i don't have access to a physical server, and storage size is always a problem.

marcstreeter 10y ago
The purpose of the block storage in this instance isn't about giving your vm/droplet more space. It's separation. That way any data that's on that device can be attached to another vm/droplet. It probably would be more cost effective just to upgrade the vm/droplet if space were a concern. It's at least how we've marketed the same feature for the past year or two via Codero's portal. Not to say I don't like how DO has entered the space: keeping it simple.
- walkertraylor 10y ago
  Exactly! It's about having an easy upgrade path, generally reducing the amount of work for common operations, and for more flexibility engineering your cloud architecture. It's nice that it happens to be SSD, but highest performance or lowest cost per GB isn't necessarily the only cost saving factor. This a main reason I still use Amazon AWS: I can create an instance and if it doesn't perform, upgrade it until it does. Then when I'm finished, kill the instance and save the volume. Next time I need it, just create the instance for the job, perhaps at spot pricing, then kill it again.
  - drtse4 10y ago
    What you describe was already possible on DO using snapshots (that can be stored for free), and that's what I usually do too.

andybak 10y ago

This helps me with a nicer deployment setup. I was always keen on 'rebuild from scratch' rather than 'update stuff and hope you're idempotent and have captured all changes' but transitory data was always the problem. Now I can start building a new updated droplet and the only downtime will be that needed to detach and reattach the block storage containing the db etc. Anyone see a flaw in this? (I know there are other ways to achieve similar benefits - my files could be on S3 and the database could be a separate droplet etc but these introduced various drawbacks and added complexity)

Jedd 10y ago
> Anyone see a flaw in this? Perhaps not a flaw, but some issues with your setup are implied. If you're rebuilding from scratch because you're not sure that you can update things, then you're probably in need of a configuration management tool (I'm a big fan of saltstack[1], mostly because I don't like Ruby or DSL's, but there's lots of options out there[2]) If you're worried you're going to lose transitory data, it sounds like you don't have a trusted and tested backup/archival/recovery process in place. So having it stored on a single EBS / DO BS / etc means you're still exposed. If you're rebuilding and rolling data over, in this scenario, I'd be copying, rather than relocating, any precarious data repositories. [1] https://saltstack.com/ https://saltstack.com/ [2] https://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software https://en.wikipedia.org/wiki/Comparison_of_open-source_conf...
- vidarh 10y ago
  I tend to avoid those config management tools other than for basic bootstrapping exactly because while you can use those too to recreate from scratch, when you don't do that, you leave the door open for undocumented, unknown state, since most of them basically take a system in an unknown-but-hopefully-mostly-consistent state and try to bring them to a known state. But they'll only be in a known state in that case if your setup is extremely comprehensive. In practice I've seen too many config management tools where long running servers have ended up in unknown states because changes have been applied, and subsequently changes have either been made outside of the toolchain, or changes have been done to the config in ways that doesn't let the tool know what has changed, or the tools simply doesn't have a way of comparing machine states without comprehensively enumerating everything on the server (e.g. people running Ansible playbooks that adds X, subsequently removing the requirement X from the playbook, and going about without considering whether or not X will interact with Y which they've added later). As a result, I see rebuilding from scratch as largely orthogonal to whether or not you use a configuration management tool or e.g. build VM images that you replace wholesale, or whatever you do: You should rebuild from scratch regularly, as coupled with a test-suite it's the only realistic way of knowing whether or not you've left anything out of your build process. My biggest caveat with config management systems is that they tend to end up encouraging live changes to a setup, instead of a build-test-deploy cycle. Sometimes that's necessary, but to me that's a last resort.

happyslobro 10y ago

Setup: find it's name, format and mount it, as if you were adding an SSD to a desktop. https://www.digitalocean.com/community/tutorials/how-to-use-block-storage-on-digitalocean https://www.digitalocean.com/community/tutorials/how-to-use-...

happyslobro 10y ago

I don't suppose DO or anyone else is already working on an Ansible addon for this?

wiremine 10y ago

Spun one up and ran some quick numbers on a 100GB volume: root@ubuntu-1gb-nyc1-01:~# time dd if=/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 of=test.dat bs=1024 count=10000000 10000000+0 records in 10000000+0 records out 10240000000 bytes (10 GB) copied, 58.0655 s, 176 MB/s real 0m58.248s user 0m2.608s sys 0m41.604s Some quick observations: * Easy to add one when creating a droplet; by default they let you create volumes with these sizes: 100GB, 250GB, 500GB, 1000GB, 1.95TB; it's also really easy to create your own size. * You can resize in any increments; took about 4 seconds to go from 100GB to 110GB with no downtime; you obviously need to resize/manage the mounted volume yourself. * [Edit 1] Deleting the droplet does NOT destroy the volume. Worth keeping in mind when you spin them up/down. * [Edit 2] Remounting an existing volume to a new droplet was quick and painless.

olavgg 10y ago
Thank you for telling us these details. I can't help but I note that you benchmark the drive with a block size of 1024. Minimum block size of a modern SSD is 4096, anything lower would just cause unnecessary load. Also if you want to benchmark a drive/block storage I highly recommend using fio.
- wiremine 10y ago
  Thx for the tip, I'll give that a try!
  - ymse 10y ago
    fio --direct=1 --rw=write --ioengine=libaio --runtime=300 --bs=4k --numjobs=4 --iodepth=32 --size=2G --name=4096-direct --filename=/dev/vdb --group_reporting This is my "dd" test for any storage system. Filename can be a regular file; you can also pass --directory instead. I would love to see numbers for both this test as well as with --bs=4M. Also experiment with --rw=rw|randrw|read :) (and make sure your device is large enough to hold numjobs*size)
Thaxll 10y ago
Don't use dd for those tests it's really bad, especially on VMs.

deleted 10y ago

[deleted]

simos 10y ago

Some early benchmarks about the new block storage, https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-on-digitalocean-with-block-storage/ https://simos.info/blog/trying-out-lxd-containers-on-ubuntu-... I did not get good speeds and I am wondering why that may be...

zbjornson 10y ago
> The immediate benefits are that the latency is much lower with the new block storage I think you might have misread the units: locally attached 50105 us (50 ms) vs block 546 ms. The throughput numbers are on par with AWS and GCP block storage. This seems reasonable aside from the high latency.

scurvy 10y ago

What's the backend? Ceph?

marcstreeter 10y ago
Ceph is not block storage -- ceph is object storage
- vruiz 10y ago
  It's actually both, and a file system.
  - marcstreeter 10y ago
    Oh I guess I thought ceph was all about eventual consistency. I didn't know it was strongly consistent like what I expect from block storage.
  - frabbit 10y ago
    Not really. It's object storage with other stuff layered on top of it with varying degrees of success.
- frabbit 10y ago
  Ceph can use RBD (rados block device) to provide block storage.
- notacoward 10y ago
  It's fundamentally objects, but not S3-style objects. It's more NASD/T10 objects, with rich semantics including partial writes and strong consistency. Implementing block storage on top of this layer (called RADOS) is trivial. Implementing S3-style objects is a bit more work, and implementing a filesystem was downright hard, but both have been done. Disclaimer: I'm a Gluster developer, which isn't quite the same as being a Ceph developer but we do talk to one another. ;)
  - scoot 10y ago

johnwheeler 10y ago

I love how DO focuses on what matters the most: Inexpensive VMs and scalable block storage. If I had to pick two, those would be them!

brianwawok 10y ago
Well except being 4 years late to the block storage game? Seems they have an uphill fight. Aws and gce match them on low end droplet price and offer much more. No local ssd but not sure what % of apps really need local ssd. DO effectively forces you to pay for local ssd for all of your servers.
- velodrome 10y ago
  Well, they beat Linode.. With DO, at least there is an additional option available for users. With GCE and AWS, the outbound bandwidth is expensive. 1TB = $90 (AWS, GCE) vs 1TB included (Linode, DO).
  - e12e 10y ago
    This. I'm always a little confused by how people even evaluate the big three clouds, with prices per request and all bandwidth "not included". It's so strange coming from cheap dedicated servers with typically 10tb bandwidth/month "included". I mean, what do people do in the cloud that requires 100gb of storage, and insignificant transfer?
  - brianwawok 10y ago
    Sure, and certain classes of problems are cheaper on DO (webscraper, botnet, etc). When they launched they just felt so far ahead of everyone else (1/4 the price of linode for SSD!). But price has been static for 5? years, so feeling lessing exciting these days.

bjacobel 10y ago

Reminder that just a few weeks ago DigitalOcean rolled over on one of their customers and took down 38,000 websites after receiving a claim of infringement from the NRA against a parody site hosted on surge.sh: http://motherboard.vice.com/read/nra-complaint-takes-down-38000-websites http://motherboard.vice.com/read/nra-complaint-takes-down-38...

treehau5 10y ago
Just a reminder that most businesses would do this exact same thing in this position and the problem is with the policy and enforcement. Get out and vote for the right people this November.
- cmdrfred 10y ago
  Find me a viable candidate who will work for the people and I will.
  - unethical_ban 10y ago
    Gary Johnson isn't as insane as the other two.
    - fweespeech 10y ago
      Yeah, tbh, just throwing your vote that way to pollsters may be enough to get him on the debate stage and that is the best you can really do.
    - btilly 10y ago
      Remember the old joke that I can't be a Democrat because I want to spend my own money, and I can't be a Republican because of what I want to spend my money on? If you chuckled at it, you should check Gary Johnson out. For real.
      - acehyzer 10y ago

Mister_Snuggles 10y ago

This is EXACTLY the thing that I need for one of my droplets! I love how there is nothing "special" about it - it's just a disk that you can attach to a droplet. I'm sure that under the hood there's some kind of magic going on, but it looks like it's nicely abstracted away. This is what I hoped block storage would turn out to be - here's a block device, use it like one. As soon as this rolls out to the region I've got that droplet in, I'm going to pull the trigger on it. I might even spend the effort to migrate my droplet to a supported region just to get this.

mikeash 10y ago
Same here. My $5/month droplet is sufficient for all my needs, except it's a bit limiting on storage. $2/month for an extra 20GB, doubling my storage, would be great. I don't want to migrate anything and I don't care about more CPU or RAM, I just want more space!
- hashmp 10y ago
  Totally agree, this was their main limitation. Glad they have now solved it.

misframer 10y ago

It's the same price as AWS's General Purpose SSD EBS volumes. https://aws.amazon.com/ebs/pricing/ https://aws.amazon.com/ebs/pricing/

hashmp 10y ago
Yep.... Digital Ocean is much more competitive for instances though and out going bandwidth.
- eropple 10y ago
  At the cost of significantly worse networking (VPC is fantastic), no object storage mechanism, no equivalent to IAM for instance permissioning, and no autoscaling (seriously? in 2016? I had to go look to make sure I wasn't misremembering--that they recommend something like DOProxy should make the lot of them faintly embarrassed). DigitalOcean may have some value at the basement level of compute, but as a professional in this area there is literally no situation where I would use DigitalOcean right now because I value my time. AWS is already laughably cheap, the tooling is overwhelmingly superior, the resources available are better, and you aren't duct-taping together half-solutions and reinventing every wheel. (This is less an endorsement of AWS and more an endorsement of Not DigitalOcean; GCE is more than fine, Azure has some ugly bits around autoscaling that I don't like but you can get by.)

fweespeech 10y ago

$.10/GB block storage is too expensive. https://www.online.net/en/dedicated-server/rpn-san https://www.online.net/en/dedicated-server/rpn-san There are several places you can get ~5 TB for +/-10% of the 1TB price at DO. DO is offering a SAN at Object Store prices. :/

tomschlick 10y ago
So is that service. You linked to the non SSD version. Their SSD version is $0.12/GB

3pt14159 10y ago

I have been asking for non-SSD on DO for a long time now. My heart jumped when I saw the HN title, only to be dashed on the rocks. What are us data nerds supposed to do? We want to take 10 terabytes, run a batch process on it, keep the 20TB, then continue with about 5GB of working data until the next month's terabyte comes in, then we want to batch through the 21TB. Right now the price slider doesn't even go up to 21TB, and clicking on the "need more storage button" doesn't go anywhere, but I'm assuming it would be $2100 / month which is more than 3x as expensive than vanilla S3.

mattbee 10y ago
At Bytemark we've done network as standard since 2012, so we can do 100GB of higher-capacity storage for £2/month, and all discs are expandible / migrateable online: bigv.io/prices
boulos 10y ago
Not being snarky, but at that scale why would you still be using Digital Ocean? Are you processing this dataset with just a single droplet? Are you going to keep each months 10 TiB forever? Something like Nearline is going to be a much better fit even if you process the data and have to pay the $.01/GB retrieval fee. Disclaimer: I work on compute engine (so of course I want your money), but I'm honestly curious.
mikeash 10y ago
"All SSD all the time" is kind of DO's thing. There are lots of other VPS providers with round storage if that's what you're after.
- raiyu 10y ago
  All SSD all the time, we may make that our new tagline. But yeah, we basically look to SSD for everything as that continues to take over market share from spinning disk and when you have to support a product 5-10 years, it's necessary to look ahead, so as SSD prices continue to drop we will eventually switch to SSD for everything and spinning disk will become the equivalent of tape drives in the old days.

Mister_Snuggles 10y ago

I can't wait for this to roll out to more regions. This is EXACTLY the thing I need for some stuff I'm working on!

mwcampbell 10y ago

I think this might be a mistake. Ever since Joyent's commentary on one of the big Amazon EBS failures in 2011 [1] [2] [3], I've been suspicious of all network-attached block storage. Then again, I haven't heard of any big EBS failures recently; I wonder what changed. [1]: https://www.joyent.com/blog/on-cascading-failures-and-amazons-elastic-block-store https://www.joyent.com/blog/on-cascading-failures-and-amazon... [2]: https://www.joyent.com/blog/magical-block-store-when-abstractions-fail-us https://www.joyent.com/blog/magical-block-store-when-abstrac... [3]: https://www.joyent.com/blog/network-storage-in-the-cloud-delicious-but-deadly https://www.joyent.com/blog/network-storage-in-the-cloud-del...

boulos 10y ago
Network block storage isn't inherently broken, the initial EBS implementation was frankly just unreliable. We've not had anything like those dark days with Persistent Disk. It's still true that having your storage across the network opens you to networking failures taking out your storage, but the gain in durability and maintenance pays for it (in our case, live migration would just be crazy with local spinning disks, we tried it didn't work). Disclaimer: I work on GCE, and we want your business ;)
- ngrilly 10y ago
  > in our case, live migration would just be crazy with local spinning disks, we tried it didn't work It looks like Exoscale does live migrations with locally attached SSDs.
  - regularfry 10y ago
    Then they'll run into the same problem anyone does doing that: migrating reasonably-sized block devices across a reasonable network takes an unreasonable length of time. Been there, done that, don't want to sit staring at consoles waiting for live disc migrations ever again.
    - ngrilly 10y ago

cgag 10y ago

Sweet. This makes digitalocean much more appealing as a potential substrate for a kubernetes cluster.

pstadler 10y ago
I just migrated my Kubernetes/Rancher stack from NYC2 to NYC1 in order to use block storage. Eager to see whether this plays well with GlusterFS.

ozy23378 10y ago

Going to perform some very basic DD i/o benchmarks using: https://haydenjames.io/web-host-doesnt-want-read-benchmark-vps/ https://haydenjames.io/web-host-doesnt-want-read-benchmark-v... Will post results.

iofj 10y ago

Why does anyone want block storage ? Why not just have an unlimited filesystem ?

aibottle 10y ago

Thank god! Highly Available Block Storage. From Digital Ocean. Great! Now I can finally store all the 300mb/s streaming in on my server. Oh wait. I cannot, because DO cancelled the service again. Bummer.

cheapsteak 10y ago
Cancelled which service?

aibottle 10y ago

mrmondo 10y ago

How is it taking huge cloud providers so long to catch up with things we do self hosted every day? It obviously has to be well engineered, yet it's relatively simple. Woefully poor performance too.