4 ms·

As someone who works with a fairly large EC2 infrastructure, I find the move towards EBS-only instance types somewhat alarming. For me, the main draw of EBS is

As someone who works with a fairly large EC2 infrastructure, I find the move towards EBS-only instance types somewhat alarming. For me, the main draw of EBS is to ensure data is retained in the case of instance failure, but it comes with significantly lower performance than instance-store SSDs and is more expensive. I've resisted EBS for the most part, and all of my servers are treated as disposable, but AWS is obviously trying to get rid of it as an option. The only reason I can think of is the excessive amount of money they can charge for EBS instances (especially PIOPS)

toomuchtodo 10y ago
They're getting rid of instance stores because all of the bare metal can be diskless then except for the hypervisor boot media (they could even PXE bootstrap that if they wanted and run entirely ephemeral on the physical node). Think of the cost savings on the AWS compute hardware. Anyway! Provisioned IOPS are sufficient for most workloads; if they're not for yours, you might consider a ramdisk (expensive) or going bare metal on physical servers (cheap).
andrioni 10y ago
Well, they still have the instance types where instance storage matters most, IMO: d2 and i2 (which are great for running things like Cassandra and HDFS). Also, complementing toomuchtodo's comment, there's also now EBS types focused on almost all kinds of workloads: sc1 for cold data/archiving, st1 for workloads optimized for sequential scans (a Confluent benchmark even reported that Kafka ran better with st1 EBS than instance storage), io1/PIOPS for databases and gp2 for everything else.
helper 10y ago
YES. We have a strict no-EBS policy. Our main concern is that historically EBS has been a big single point of failure that can take down an entire AZ or worse a whole region. There were a number of outages in 2010-2012 that really bad for EBS instances[1][2]. The rate of outages has certainly gone down since then, but its hard to trust a system that has burned you badly multiple times in the past. There was a good post by Bryan Cantrill after one of those really bad outages[3]. While this post is from an AWS competitor, the arguments about the reliability of network storage are sound. While the trend by AWS has been to move away from ephemeral storage, it doesn't seem like they are working to completely kill it off. d2 and i2 fill a lot of our storage node needs. For our compute nodes I could see us moving to a model of booting off ebs and then pivot rooting into a ramdisk root fs so nothing would touch ebs after boot. [1]: https://aws.amazon.com/message/67457/ https://aws.amazon.com/message/67457/ [2]: http://www.agilesysadmin.net/ec2-outage-lessons http://www.agilesysadmin.net/ec2-outage-lessons [3]: https://www.joyent.com/blog/on-cascading-failures-and-amazons-elastic-block-store https://www.joyent.com/blog/on-cascading-failures-and-amazon...
- Terretta 10y ago
  AWS user since the start. Was firmly in the no-EBS camp. Still prefer it aesthetically. No parts is better than good parts. But they do seem to have figured it out, hasn't been the SPOF it used to be in half a decade now. // Knock wood, etc.
skywhopper 10y ago
It's probably a lot easier for them to manage EBS than per-host storage. And also probably a lot more efficient as most of that ephemeral storage is wasted, but EBS is elastic and you can just provision what you need. For most workloads, EBS is fine, and if you don't need lots of space, it's cheaper to get c4 instances with a small EBS volume than a slower comparable c3 instance with local storage. So the savings _is_ passed along. All that said, I expect the next generation will have at least one type with a mid-range local SSD available if only because there is real demand for it. But it's not going to be cheaper than using EBS, unless you use most of the disk.
NeutronBoy 10y ago
> The only reason I can think of is the excessive amount of money they can charge for EBS instances (especially PIOPS) The main driver is that they don't need to have instance storage - that is, they can uncouple the storage from their compute nodes entirely and keep it on SAN somewhere.

toomuchtodo 10y ago

They're getting rid of instance stores because all of the bare metal can be diskless then except for the hypervisor boot media (they could even PXE bootstrap that if they wanted and run entirely ephemeral on the physical node). Think of the cost savings on the AWS compute hardware. Anyway! Provisioned IOPS are sufficient for most workloads; if they're not for yours, you might consider a ramdisk (expensive) or going bare metal on physical servers (cheap).

andrioni 10y ago

Well, they still have the instance types where instance storage matters most, IMO: d2 and i2 (which are great for running things like Cassandra and HDFS). Also, complementing toomuchtodo's comment, there's also now EBS types focused on almost all kinds of workloads: sc1 for cold data/archiving, st1 for workloads optimized for sequential scans (a Confluent benchmark even reported that Kafka ran better with st1 EBS than instance storage), io1/PIOPS for databases and gp2 for everything else.

helper 10y ago

YES. We have a strict no-EBS policy. Our main concern is that historically EBS has been a big single point of failure that can take down an entire AZ or worse a whole region. There were a number of outages in 2010-2012 that really bad for EBS instances[1][2]. The rate of outages has certainly gone down since then, but its hard to trust a system that has burned you badly multiple times in the past. There was a good post by Bryan Cantrill after one of those really bad outages[3]. While this post is from an AWS competitor, the arguments about the reliability of network storage are sound. While the trend by AWS has been to move away from ephemeral storage, it doesn't seem like they are working to completely kill it off. d2 and i2 fill a lot of our storage node needs. For our compute nodes I could see us moving to a model of booting off ebs and then pivot rooting into a ramdisk root fs so nothing would touch ebs after boot. [1]: https://aws.amazon.com/message/67457/ https://aws.amazon.com/message/67457/ [2]: http://www.agilesysadmin.net/ec2-outage-lessons http://www.agilesysadmin.net/ec2-outage-lessons [3]: https://www.joyent.com/blog/on-cascading-failures-and-amazons-elastic-block-store https://www.joyent.com/blog/on-cascading-failures-and-amazon...

Terretta 10y ago
AWS user since the start. Was firmly in the no-EBS camp. Still prefer it aesthetically. No parts is better than good parts. But they do seem to have figured it out, hasn't been the SPOF it used to be in half a decade now. // Knock wood, etc.

skywhopper 10y ago

It's probably a lot easier for them to manage EBS than per-host storage. And also probably a lot more efficient as most of that ephemeral storage is wasted, but EBS is elastic and you can just provision what you need. For most workloads, EBS is fine, and if you don't need lots of space, it's cheaper to get c4 instances with a small EBS volume than a slower comparable c3 instance with local storage. So the savings _is_ passed along. All that said, I expect the next generation will have at least one type with a mid-range local SSD available if only because there is real demand for it. But it's not going to be cheaper than using EBS, unless you use most of the disk.

NeutronBoy 10y ago

> The only reason I can think of is the excessive amount of money they can charge for EBS instances (especially PIOPS) The main driver is that they don't need to have instance storage - that is, they can uncouple the storage from their compute nodes entirely and keep it on SAN somewhere.