3 ms·

The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is ver

by throwaway2016a 11y ago

The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Also, I actually can't recall (in 8 years of AWS) a time where I saw CPU being stolen. Perhaps I've been lucky. I also don't watch for it constantly. We actually did this with our dev servers. Although switch VSphere for VirtualBox on Linux and colo for a server room in our office. It worked great and saved a TON of cost. But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers.

toomuchtodo 11y ago
> The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.
- throwaway2016a 11y ago
  True. You have to design for failure with AWS. Which does have a large amount of cognitive overhead. If you don't design for failure you're going to have a bad time. However, designing for strict SLAs is not impossible with AWS. You just need to have multi-region redundancy and you can get very very good with multi-availability zone redundancy. I have no excuse for the IAM outage, it sucked for our ops team. I guess my only two consultations are: 1. We haven't had a customer-visible outage due to AWS in years because we follow best practices (which does cost more - but see my previous point on more smaller machines vs many large machines) 2. If we were running our own authentication and access control system similar to IAM, it too could have an outage. But I agree, that is a bad thing about AWS.
nirvdrum 11y ago
Well, this is what I was referring to with the HA features of vSphere. It works basically like a compute level RAID. The VMs were stored on a SAN and vSphere monitored each of the compute blades. If one went down, the VMs were seamlessly migrated to a hot spare blade. The vSphere docs claim this can be achieved with zero packet loss -- a claim I was never able to test or verify. If you're worried about losing multiple machines, just add multiple hot spares. Of course, this doesn't help if you lose an entire rack or the data center. I concede this was a trade-off. But given how many times an entire region went down when I was on EC2, I was satisfied with the risk based on the colo environment's uptime record. The provider did offer another facility, but the latency between the two was too high to be of practical use in a failover without keeping a completely mirrored configuration in both locations. It sounds like you have some experience with vSphere, so I don't intend to be patronizing. But there's a huge difference between "enterprise" virtualization and what you get with an ad hoc setup using desktop virtualization tools.

toomuchtodo 11y ago

> The tradeoff for CPU stealing is that now you have multiple services running on the same machine so if the machine goes down you lose them all. On AWS it is very unlikely that two of your VMs end up on the same physical machine. Yes. But its also very likely that when AWS has issues, the entire region is going to be having problems, like last Saturday (IAM, EC2, Autoscaling, etc broken badly for 6 hours). > But we don't need pesky things like 24hr uptime and dedicated bandwidth for dev servers. You're not getting this SLA unless you're managing everything yourself and are globally redundant. AWS doesn't provide bandwidth guarantees, nor will you get 100% uptime.

throwaway2016a 11y ago
True. You have to design for failure with AWS. Which does have a large amount of cognitive overhead. If you don't design for failure you're going to have a bad time. However, designing for strict SLAs is not impossible with AWS. You just need to have multi-region redundancy and you can get very very good with multi-availability zone redundancy. I have no excuse for the IAM outage, it sucked for our ops team. I guess my only two consultations are: 1. We haven't had a customer-visible outage due to AWS in years because we follow best practices (which does cost more - but see my previous point on more smaller machines vs many large machines) 2. If we were running our own authentication and access control system similar to IAM, it too could have an outage. But I agree, that is a bad thing about AWS.

nirvdrum 11y ago

Well, this is what I was referring to with the HA features of vSphere. It works basically like a compute level RAID. The VMs were stored on a SAN and vSphere monitored each of the compute blades. If one went down, the VMs were seamlessly migrated to a hot spare blade. The vSphere docs claim this can be achieved with zero packet loss -- a claim I was never able to test or verify. If you're worried about losing multiple machines, just add multiple hot spares. Of course, this doesn't help if you lose an entire rack or the data center. I concede this was a trade-off. But given how many times an entire region went down when I was on EC2, I was satisfied with the risk based on the colo environment's uptime record. The provider did offer another facility, but the latency between the two was too high to be of practical use in a failover without keeping a completely mirrored configuration in both locations. It sounds like you have some experience with vSphere, so I don't intend to be patronizing. But there's a huge difference between "enterprise" virtualization and what you get with an ad hoc setup using desktop virtualization tools.