14 ms·

Amazon EBS Update – New Elastic Volumes Change Everything

133 points by chrisbolt 10y ago

addisonj 10y ago
This is a pretty swell announcement. While I think I still might do an LVM for adding capacity, the biggest change here would appear the ability to scale provisioned IOPS. While gp2 disks IOPS are primarily tied to storage capacity, which often leads to overprovisvioning for peak loads, the alternatively high cost of smaller and faster storage with PIOPS often times isn't worth it when peak loads only happen occasionally. That changes with this and I am sure this will make a lot of DBAs happy. AWS in the last year has really impressed with giving you even more knobs to turn to get exactly what you need. While it's still not quite as flexible as GCEs custom instance sizes, the GPU and now storage elasticity sort of make up for it. While it's still pretty expensive, AWS feels more and more like I can use an API to build a custom spec'ed server instead of trying to shove my workloads into only a handful of options.
techdragon 10y ago
This is an amazing new feature... other than the lack of "shrinking a volume" support... I'm definitely going to be using this from now on and this feature is going to make a big difference in my day to day work. That said... I'd still rather have access to Elastic File System in more regions, and in a perfect world, be able to talk to EFS from Lambda.
- craigching 10y ago
  > other than the lack of "shrinking a volume" support It looks like you can shrink a volume, the edit box to change this just a "set a size" box. We have tested this and downsized one volume now and appears to have worked.
mrmondo 10y ago
That's still terrible performance, your average MacBook Pro these days will provide between 100,000 and 300,000 IOP/s on 4k random reads and the average VM we deploy from our storage servers (commodity SSD based linux clusters that are cheap and ultra reliable) provide between 70,000-200,000 random 4k read (and write) IOPs per VM without a sweat and their TCO is significantly lower than amazons SSD storage that's 3 ½ - 10x slower per volume. Even if you go and provision storage on Vultr, Ramnode or Sitehost all of which are a lot cheaper options and just as if not more reliable provide considerably faster storage with a minimum of 20/30,000 IOPs up to 150,000 IOPs per volume and you don't have the vendor lock-in or the hidden costs such as inter-zone data transfer etc... Amazon really needs to pull finger on their storage performance, but it's not in their best interest. What they want you to do is scale horizontally which means not only do you need to invest in your system / product design to do this but you also have to put more money into Amazons pockets for additional compute / storage nodes and again the associated hidden costs. I'm not digging on Amazon / outsourced hosting providers but they truly are pulling the wool over people's eyes while still trying to make everyone think that they're not only the best option they're the only option.
- misframer 10y ago
  EBS is network-attached storage and has its benefits and downsides. I don't think it's fair to compare it to local storage.
  - sciurus 10y ago
    Amazon's options for local, high speed SSDs aren't very compelling either. You're limited to the I2 instances and have to increase the instance size in order to increase storage performance https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-optimized-instances.html#i2-instances-diskperf https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-... Compare this to Google Cloud, where you can attach a local SSD to any instance type and you can increase the storage performance by increasing the volume size. https://cloud.google.com/compute/docs/disks/performance https://cloud.google.com/compute/docs/disks/performance
  - mrmondo 10y ago
    I didn't compare it to local storage, I compared it to highly available network storage.
mdekkers 10y ago
What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use.
- koolba 10y ago
  > What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use. Have you tried EFS? https://aws.amazon.com/efs/ https://aws.amazon.com/efs/
  - mdekkers 10y ago
    No, since it is stupidly expensive in itself. Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks.
    - koolba 10y ago
      > No, since it is stupidly expensive in itself. EFS is ridiculous cheap if you use it the way it's intended. It's billed per-byte so you only store things that you actually need shared on there. If you're looking for an OCFS2 replacement for storing large volumes of data or to run something like Oracle RAC atop then you're probably better off either doing it yourself. > Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks. Besides the pricing argument, I'd disagree with all of those points. It's a joy to set up once you know what you're doing and performance is more than adequate for all the services I've used (with plenty of knobs to fiddle). Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise.
      - 10y ago
aphextron 10y ago
This is really fantastic. Volume management was always my biggest pain point with EC2. I never understood why they had to make it so complicated.
koolba 10y ago
For those of us running on EBS, this is welcome news. Now you can put off swapping out EBS volumes until you hit the cap of 16TB (or thereabouts). If you're really clever, you'd set up your AMIs to use LVM and keep adding additional EBS volumes before you reach the 16 TB per volume limit. Though I'm guessing if you have the issue where 16 TB of storage isn't enough for you then you probably can afford to find a real distributed solution to this.
- matt_wulfeck 10y ago
  There's an entire class of engineers who cooked up clever lvm/zfs partionining schemes to work around this. It's bittersweet to throw away a working, custom solution when something more automated and standard comes around. c'est la vie!
jpatokal 10y ago
"Change everything"? "Game-changing"? I'll just leave this here: https://cloudplatform.googleblog.com/2016/03/introducing-Google-Cloud-Persistent-Disks-with-non-disruptive-online-resizing.html https://cloudplatform.googleblog.com/2016/03/introducing-Goo... (March 2016) (Disclaimer: Yes, I work on Google Cloud.)
- manojlds 10y ago
  Change everything is in context of this feature. Game changing is mentioned in the end in context of EBS and not this feature. And the entire article is about AWS customers and it really is game changing for a customer like me. Your comment is in poor taste.
- inopinatus 10y ago
  Pretty sure it's a pun, since previously you couldn't change anything.
ryanmarsh 10y ago
LOL somebody's going to set up an alarm to automatically increase volume size on low space and one day end up with max volume size on accident.
wincy 10y ago
This is excellent, too bad I had to increase the volume size on our prod stack two weeks ago.
caw 10y ago
I had a volume that needed resizing, so went ahead and tried moving from a 50GB gp2 to a 100GB gp2 disk online. It took 1 hour 40 minutes to do the resize. It's not a quick operation, and from looking at the docs you're limited to 1 modification per 6 hours on any given volume.
- gigatexal 10y ago
  Whoa I guess that's more or less the same as downtime. Imagine if you were out of space but couldn't do anything for 100 minutes. I am watching EFS and hoping it falls in price or that EBS that backs EC2 instances becomes more automatically elastic.
  - gigatexal 10y ago
    To further my point: what I'm getting at is that EFS has the chance to be the answer to persistent storage for docker swarm and just persistent storage for docker in general. But at .30 cents a gig it's just too much right now.
- gbrits 10y ago
  was it unusable during resize?
  - caw 10y ago
    I didn't explicitly test that, though I expect it's usable as the docs suggest that after you resize a live root partition you should reboot the box after modifying the filesystem extents (resize2fs). I was modifying a scratch partition -- nothing blew up while the resize happened.
matt_wulfeck 10y ago
> Spiking Demand – You are running a relational database on a Provisioned IOPS volume that is set to handle a moderate amount of traffic during the month, with a 10x spike in traffic during the final three days of each month due to month-end processing. You can use Elastic Volumes to dial up the provisioning in order to handle the spike, and then dial it down afterward. At first I was a little dubious of the usefulness of this update, but that features is actually really useful. I'm not sure what the granularity is, but it would be nice to crank up he iops on a DB when doing some large batch operations.
- mark242 10y ago
  If this is possible with RDS, that's huge. It's extremely frustrating to deal with spikes of transitory data into RDS and have to have the RDS disk space be sized to the maximum amount of space always-- it's a huge cost waste.
  - JamesSwift 10y ago
    Reading through the caveat page [1], it seems the marketing copy is not quite in line with reality in terms of ability to scale down directly: > Decreasing the size of an EBS volume is not supported. However, you can create a smaller volume and then migrate your data to it using application-level tools such as robocopy. [1] - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/considerations.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/consider...
    - inopinatus 10y ago
      The blurb is not misleading. PIOPS are a performance measure distinct from volume size.
agwa 10y ago
Do you still have to stripe over multiple volumes to get good performance?
rymohr 10y ago
> M3.medium instances are treated as current generation. M3.large, m3.xlarge, and m3.2xl instances are treated as previous generation. Why? These other m3 instances are listed as current generation here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#current-gen-instances http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-...
compsciphd 10y ago
There's one thing that really annoys me about EBS volumes. You can't attach existing (unattached) persistent volumes to an instance at instance creation time. So for instance, I can't have a machine boot and expect to see ebs vol-xyz on device /dev/xvdg at boot. I have to boot the instance, use the API to attach it and then have my instance mount it.

133 points by chrisbolt 10y ago

This is a pretty swell announcement. While I think I still might do an LVM for adding capacity, the biggest change here would appear the ability to scale provisioned IOPS. While gp2 disks IOPS are primarily tied to storage capacity, which often leads to overprovisvioning for peak loads, the alternatively high cost of smaller and faster storage with PIOPS often times isn't worth it when peak loads only happen occasionally. That changes with this and I am sure this will make a lot of DBAs happy. AWS in the last year has really impressed with giving you even more knobs to turn to get exactly what you need. While it's still not quite as flexible as GCEs custom instance sizes, the GPU and now storage elasticity sort of make up for it. While it's still pretty expensive, AWS feels more and more like I can use an API to build a custom spec'ed server instead of trying to shove my workloads into only a handful of options.

techdragon 10y ago

This is an amazing new feature... other than the lack of "shrinking a volume" support... I'm definitely going to be using this from now on and this feature is going to make a big difference in my day to day work. That said... I'd still rather have access to Elastic File System in more regions, and in a perfect world, be able to talk to EFS from Lambda.

craigching 10y ago
> other than the lack of "shrinking a volume" support It looks like you can shrink a volume, the edit box to change this just a "set a size" box. We have tested this and downsized one volume now and appears to have worked.

mrmondo 10y ago

That's still terrible performance, your average MacBook Pro these days will provide between 100,000 and 300,000 IOP/s on 4k random reads and the average VM we deploy from our storage servers (commodity SSD based linux clusters that are cheap and ultra reliable) provide between 70,000-200,000 random 4k read (and write) IOPs per VM without a sweat and their TCO is significantly lower than amazons SSD storage that's 3 ½ - 10x slower per volume. Even if you go and provision storage on Vultr, Ramnode or Sitehost all of which are a lot cheaper options and just as if not more reliable provide considerably faster storage with a minimum of 20/30,000 IOPs up to 150,000 IOPs per volume and you don't have the vendor lock-in or the hidden costs such as inter-zone data transfer etc... Amazon really needs to pull finger on their storage performance, but it's not in their best interest. What they want you to do is scale horizontally which means not only do you need to invest in your system / product design to do this but you also have to put more money into Amazons pockets for additional compute / storage nodes and again the associated hidden costs. I'm not digging on Amazon / outsourced hosting providers but they truly are pulling the wool over people's eyes while still trying to make everyone think that they're not only the best option they're the only option.

misframer 10y ago
EBS is network-attached storage and has its benefits and downsides. I don't think it's fair to compare it to local storage.
- sciurus 10y ago
  Amazon's options for local, high speed SSDs aren't very compelling either. You're limited to the I2 instances and have to increase the instance size in order to increase storage performance https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-optimized-instances.html#i2-instances-diskperf https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage-... Compare this to Google Cloud, where you can attach a local SSD to any instance type and you can increase the storage performance by increasing the volume size. https://cloud.google.com/compute/docs/disks/performance https://cloud.google.com/compute/docs/disks/performance
- mrmondo 10y ago
  I didn't compare it to local storage, I compared it to highly available network storage.

mdekkers 10y ago

What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use.

koolba 10y ago
> What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use. Have you tried EFS? https://aws.amazon.com/efs/ https://aws.amazon.com/efs/
- mdekkers 10y ago
  No, since it is stupidly expensive in itself. Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks.
  - koolba 10y ago
    > No, since it is stupidly expensive in itself. EFS is ridiculous cheap if you use it the way it's intended. It's billed per-byte so you only store things that you actually need shared on there. If you're looking for an OCFS2 replacement for storing large volumes of data or to run something like Oracle RAC atop then you're probably better off either doing it yourself. > Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks. Besides the pricing argument, I'd disagree with all of those points. It's a joy to set up once you know what you're doing and performance is more than adequate for all the services I've used (with plenty of knobs to fiddle). Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise.
    - 10y ago

aphextron 10y ago

This is really fantastic. Volume management was always my biggest pain point with EC2. I never understood why they had to make it so complicated.

koolba 10y ago

For those of us running on EBS, this is welcome news. Now you can put off swapping out EBS volumes until you hit the cap of 16TB (or thereabouts). If you're really clever, you'd set up your AMIs to use LVM and keep adding additional EBS volumes before you reach the 16 TB per volume limit. Though I'm guessing if you have the issue where 16 TB of storage isn't enough for you then you probably can afford to find a real distributed solution to this.

matt_wulfeck 10y ago
There's an entire class of engineers who cooked up clever lvm/zfs partionining schemes to work around this. It's bittersweet to throw away a working, custom solution when something more automated and standard comes around. c'est la vie!

jpatokal 10y ago

"Change everything"? "Game-changing"? I'll just leave this here: https://cloudplatform.googleblog.com/2016/03/introducing-Google-Cloud-Persistent-Disks-with-non-disruptive-online-resizing.html https://cloudplatform.googleblog.com/2016/03/introducing-Goo... (March 2016) (Disclaimer: Yes, I work on Google Cloud.)

manojlds 10y ago
Change everything is in context of this feature. Game changing is mentioned in the end in context of EBS and not this feature. And the entire article is about AWS customers and it really is game changing for a customer like me. Your comment is in poor taste.
inopinatus 10y ago
Pretty sure it's a pun, since previously you couldn't change anything.

ryanmarsh 10y ago

LOL somebody's going to set up an alarm to automatically increase volume size on low space and one day end up with max volume size on accident.

wincy 10y ago

This is excellent, too bad I had to increase the volume size on our prod stack two weeks ago.

caw 10y ago

I had a volume that needed resizing, so went ahead and tried moving from a 50GB gp2 to a 100GB gp2 disk online. It took 1 hour 40 minutes to do the resize. It's not a quick operation, and from looking at the docs you're limited to 1 modification per 6 hours on any given volume.

gigatexal 10y ago
Whoa I guess that's more or less the same as downtime. Imagine if you were out of space but couldn't do anything for 100 minutes. I am watching EFS and hoping it falls in price or that EBS that backs EC2 instances becomes more automatically elastic.
- gigatexal 10y ago
  To further my point: what I'm getting at is that EFS has the chance to be the answer to persistent storage for docker swarm and just persistent storage for docker in general. But at .30 cents a gig it's just too much right now.
gbrits 10y ago
was it unusable during resize?
- caw 10y ago
  I didn't explicitly test that, though I expect it's usable as the docs suggest that after you resize a live root partition you should reboot the box after modifying the filesystem extents (resize2fs). I was modifying a scratch partition -- nothing blew up while the resize happened.

matt_wulfeck 10y ago

> Spiking Demand – You are running a relational database on a Provisioned IOPS volume that is set to handle a moderate amount of traffic during the month, with a 10x spike in traffic during the final three days of each month due to month-end processing. You can use Elastic Volumes to dial up the provisioning in order to handle the spike, and then dial it down afterward. At first I was a little dubious of the usefulness of this update, but that features is actually really useful. I'm not sure what the granularity is, but it would be nice to crank up he iops on a DB when doing some large batch operations.

mark242 10y ago
If this is possible with RDS, that's huge. It's extremely frustrating to deal with spikes of transitory data into RDS and have to have the RDS disk space be sized to the maximum amount of space always-- it's a huge cost waste.
- JamesSwift 10y ago
  Reading through the caveat page [1], it seems the marketing copy is not quite in line with reality in terms of ability to scale down directly: > Decreasing the size of an EBS volume is not supported. However, you can create a smaller volume and then migrate your data to it using application-level tools such as robocopy. [1] - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/considerations.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/consider...
  - inopinatus 10y ago
    The blurb is not misleading. PIOPS are a performance measure distinct from volume size.

agwa 10y ago

Do you still have to stripe over multiple volumes to get good performance?

rymohr 10y ago

> M3.medium instances are treated as current generation. M3.large, m3.xlarge, and m3.2xl instances are treated as previous generation. Why? These other m3 instances are listed as current generation here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#current-gen-instances http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-...

compsciphd 10y ago

There's one thing that really annoys me about EBS volumes. You can't attach existing (unattached) persistent volumes to an instance at instance creation time. So for instance, I can't have a machine boot and expect to see ebs vol-xyz on device /dev/xvdg at boot. I have to boot the instance, use the API to attach it and then have my instance mount it.