5 ms·

Amazon EBS Enables Live Volume Modifications with Elastic Volumes

38 points by Usu 10y ago

Cidan 10y ago
I haven't done this in AWS yet, but I've been growing mounted block devices/disks Google Cloud for a while now. It's more or less just like growing a disk on a SAN. You still have to grow the filesystem it self though, so if you're trying to grow the root mount, using a partitioned disk, and not using an abstraction like LVM you will need to reboot to be able to use the full disk. Glad to see AWS catch up in this area.
- brazzledazzle 10y ago
  I wonder if Linux can mimic the technique Microsoft uses to grow volumes and filesystems on the fly. It's something I've missed on the occasion. It's pretty slick.
  - Nux 10y ago
    All my VMs have a separate /boot and ext4 as / (xfs would work, too). With this setup - using KVM, mind you - I can change the disk size of the VM and resize the / filesystem inside by just using resize2fs, no need to reboot, umount or anything else.
    - brazzledazzle 10y ago
      The impression I got a long time ago was that it's potentially unsafe. Is that old information?
      - Nux 10y ago
        I have not read that anywhere, I did it many times without data loss or other issues. You just need to use a relatively recent distro (Centos 6 +). If you use xfs, watch out, you can grow a fs, but not shrink it - which is why I use ext4 when I want more flexibility.
  - Rapzid 10y ago
    It depends on the filesystem but yes, it can.
  - Rapzid 10y ago
    It depends on the filesystem but yes, it can.
  - maccam94 10y ago
    A number of Linux filesystems support online resize. See this handy chart: https://en.wikipedia.org/wiki/Comparison_of_file_systems#Resize_capabilities https://en.wikipedia.org/wiki/Comparison_of_file_systems#Res...
diziet 10y ago
Finally, I presume this lets us resize EBS volumes without having to do things like this: https://matt.berther.io/2015/02/03/how-to-resize-aws-ec2-ebs-volumes/ https://matt.berther.io/2015/02/03/how-to-resize-aws-ec2-ebs...
nidx 10y ago
I've been procrastinating on moving our app to aws because this was not supported, I was going to have to rewrite some horrible code to support using s3 for scaling (EFS is not in the new Canada region). This should save me a few hundred hours!
moofight 10y ago
Let's say I have an EBS volume of 500GB with 300GB of data. What happens if I mistakenly resize the EBS volume to 200GB? Do I get an error message or does part of my data get wiped out?
- falsedan 10y ago
  Good news: you won't lose any data from resizing your EBS down. Bad news: that's because you can't make your EBS smaller. > You can now increase volume size, adjust performance, or change the volume type while the volume is in use. Note the absence of 'increase OR decrease volume size'.
  - moofight 10y ago
    Yes, you are absolutely right. "EBS volumes can only be increased, not decreased".
    - sidcool 10y ago
      And I think that's fair. Decreasing would be way too risky.
      - falsedan 10y ago
        I fell there are better ways to protect your customers from data loss than forbidding a potentially-destructive action. I'm quite happy to lose data, or manage my data's physical location on disk, and do online decreases… but I can't!
        pjc50 10y ago
        I suspect it's the underlying systems that aren't good at handling online decreases. But the good news is, if you're happy to lose data, you can just delete the volume and create a smaller one. (Yes I know that's probably not what you meant, but it does highlight the question of what exactly is non-valuable data?)
timborden 10y ago
We went with EFS for our RethinkDB instances on EC2. Was set up as a big data store, so read latency wasn't really an issue. Works well. But that was before this announcement...nice addition AWS EBS team!
piquadrat 10y ago
As usual, Jeff Barr's blog post is much more informative than the announcement. https://aws.amazon.com/blogs/aws/amazon-ebs-update-new-elastic-volumes-change-everything/ https://aws.amazon.com/blogs/aws/amazon-ebs-update-new-elast...
deleted 10y ago
[deleted]
social_quotient 10y ago
So can I change to spinners at night with low I/O to save on daytime costs with higher SSD IOPs? "Automate" with lambda seems like it begging for cost optimization as well.
- mwarkentin 10y ago
  I think you can only do this with gp2 and provisioned IOPS volumes. In addition, you have to wait 6 hours before scaling again.
chillydawg 10y ago
Tried this just now. Spun up 8GB ec2+ebs instance. Booted and logged in as root. Deleted the root partition using fdisk, carefully recreated it from the same starting sector but to new 100GB capacity (check lsblk output to confirm). Then resize2fs /dev/xvda1 my ext4 FS. All online, hot. Obviously it's risky, but you can take a snapshot of the EBS before starting. Seems very nice for the common use case of slowly growing storage needs that are best served by a simple disk.
tomhoward 10y ago
Earlier post: https://news.ycombinator.com/item?id=13641499 https://news.ycombinator.com/item?id=13641499

38 points by Usu 10y ago

Cidan 10y ago

I haven't done this in AWS yet, but I've been growing mounted block devices/disks Google Cloud for a while now. It's more or less just like growing a disk on a SAN. You still have to grow the filesystem it self though, so if you're trying to grow the root mount, using a partitioned disk, and not using an abstraction like LVM you will need to reboot to be able to use the full disk. Glad to see AWS catch up in this area.

brazzledazzle 10y ago
I wonder if Linux can mimic the technique Microsoft uses to grow volumes and filesystems on the fly. It's something I've missed on the occasion. It's pretty slick.
- Nux 10y ago
  All my VMs have a separate /boot and ext4 as / (xfs would work, too). With this setup - using KVM, mind you - I can change the disk size of the VM and resize the / filesystem inside by just using resize2fs, no need to reboot, umount or anything else.
  - brazzledazzle 10y ago
    The impression I got a long time ago was that it's potentially unsafe. Is that old information?
    - Nux 10y ago
      I have not read that anywhere, I did it many times without data loss or other issues. You just need to use a relatively recent distro (Centos 6 +). If you use xfs, watch out, you can grow a fs, but not shrink it - which is why I use ext4 when I want more flexibility.
- Rapzid 10y ago
  It depends on the filesystem but yes, it can.
- Rapzid 10y ago
  It depends on the filesystem but yes, it can.
- maccam94 10y ago
  A number of Linux filesystems support online resize. See this handy chart: https://en.wikipedia.org/wiki/Comparison_of_file_systems#Resize_capabilities https://en.wikipedia.org/wiki/Comparison_of_file_systems#Res...

diziet 10y ago

Finally, I presume this lets us resize EBS volumes without having to do things like this: https://matt.berther.io/2015/02/03/how-to-resize-aws-ec2-ebs-volumes/ https://matt.berther.io/2015/02/03/how-to-resize-aws-ec2-ebs...

nidx 10y ago

I've been procrastinating on moving our app to aws because this was not supported, I was going to have to rewrite some horrible code to support using s3 for scaling (EFS is not in the new Canada region). This should save me a few hundred hours!

moofight 10y ago

Let's say I have an EBS volume of 500GB with 300GB of data. What happens if I mistakenly resize the EBS volume to 200GB? Do I get an error message or does part of my data get wiped out?

falsedan 10y ago
Good news: you won't lose any data from resizing your EBS down. Bad news: that's because you can't make your EBS smaller. > You can now increase volume size, adjust performance, or change the volume type while the volume is in use. Note the absence of 'increase OR decrease volume size'.
- moofight 10y ago
  Yes, you are absolutely right. "EBS volumes can only be increased, not decreased".
  - sidcool 10y ago
    And I think that's fair. Decreasing would be way too risky.
    - falsedan 10y ago
      I fell there are better ways to protect your customers from data loss than forbidding a potentially-destructive action. I'm quite happy to lose data, or manage my data's physical location on disk, and do online decreases… but I can't!
      - pjc50 10y ago
        I suspect it's the underlying systems that aren't good at handling online decreases. But the good news is, if you're happy to lose data, you can just delete the volume and create a smaller one. (Yes I know that's probably not what you meant, but it does highlight the question of what exactly is non-valuable data?)

timborden 10y ago

We went with EFS for our RethinkDB instances on EC2. Was set up as a big data store, so read latency wasn't really an issue. Works well. But that was before this announcement...nice addition AWS EBS team!

piquadrat 10y ago

As usual, Jeff Barr's blog post is much more informative than the announcement. https://aws.amazon.com/blogs/aws/amazon-ebs-update-new-elastic-volumes-change-everything/ https://aws.amazon.com/blogs/aws/amazon-ebs-update-new-elast...

deleted 10y ago

[deleted]

social_quotient 10y ago

So can I change to spinners at night with low I/O to save on daytime costs with higher SSD IOPs? "Automate" with lambda seems like it begging for cost optimization as well.

mwarkentin 10y ago
I think you can only do this with gp2 and provisioned IOPS volumes. In addition, you have to wait 6 hours before scaling again.

chillydawg 10y ago

Tried this just now. Spun up 8GB ec2+ebs instance. Booted and logged in as root. Deleted the root partition using fdisk, carefully recreated it from the same starting sector but to new 100GB capacity (check lsblk output to confirm). Then resize2fs /dev/xvda1 my ext4 FS. All online, hot. Obviously it's risky, but you can take a snapshot of the EBS before starting. Seems very nice for the common use case of slowly growing storage needs that are best served by a simple disk.

tomhoward 10y ago

Earlier post: https://news.ycombinator.com/item?id=13641499 https://news.ycombinator.com/item?id=13641499