3 ms·

It’s definitely largely accounting but that doesn’t mean it’s bad. Offloading virtualizing to a dedicated chip means that the virtual VM (theoretically) gets al

by pm90 6y ago

It’s definitely largely accounting but that doesn’t mean it’s bad. Offloading virtualizing to a dedicated chip means that the virtual VM (theoretically) gets all of the CPU and thus suddenly workloads in aws are much more efficient, at the cost of developing and deploying new hardware. It also seems like an easier engineering problem to be able to give 100% vcpu to the customer rather than adding a fancy middleware that tries to fairly allocate the requested number of cpus while transparently hiding the cost of virtualization. The efficiencies of running on dedicated hardware might be great but just being able to separate is in itself pretty amazing.

kevincox 6y ago
That's exactly the problem. It is only efficient if the hardware is more efficient. If you spend 40% the cost of the server on this new hardware then shift that 30% of the CPU to the new hardware you haven't actually gained anything. Your hardware cost is still (approximately) 140% of what you are selling to the customer. Making it simpler or making the jitter better is a valuable benefit. But the claim that they are running more efficiently was completely unsubstantiated.
- pm90 6y ago
  From the perspective of the customer who was previously billed for the 30% of the vcpu that was just overhead, it is more efficient though (if they’re being charged the same). I agree that hypothetically they could’ve just added 30% more vcpu to every request to achieve the same thing: yes that’s a solution. But this is another solution that gave them other benefits and that’s great. I still don’t see what the problem is.
- elcritch 6y ago
  Why would you presume that the new hardware would cost 40% of the server? Sure it’s an accounting trick to decide how to amortize the development costs. However they do it, I really doubt AWS would be continuing down the Nitro path if the TCO + savings wasn’t in the black. I’d like to see the energy efficiency of the new hardware and TCO estimates too, but probably that info is deemed “sensitive” and not released publicly. Likely the Nitro cards cost a fraction of that 30%, probably more like 5-15%. I’d wager that new hardware could have a huge energy efficiency gain for the specific Nitro workloads mentioned. The 30% resource utilization on Xen could be wasting a lot of time waiting for locks, context switch’s, etc between cores that could potentially be removed entirely from custom hardware optimized for handling multiple memory contexts simultaneously just by removing cache coherency at certain points.
  - kevincox 6y ago
    I never said it did cost that much. I just said that they implied a 30% savings but that was without counting the new hardware. It is disingenuous to say that it is ~100% efficient just by moving the management cost elsewhere. I agree it was probably a win for them, I'm just pointing out that the way it was presented in the article was misleading.

by pm90 6y ago

kevincox 6y ago

That's exactly the problem. It is only efficient if the hardware is more efficient. If you spend 40% the cost of the server on this new hardware then shift that 30% of the CPU to the new hardware you haven't actually gained anything. Your hardware cost is still (approximately) 140% of what you are selling to the customer. Making it simpler or making the jitter better is a valuable benefit. But the claim that they are running more efficiently was completely unsubstantiated.

pm90 6y ago
From the perspective of the customer who was previously billed for the 30% of the vcpu that was just overhead, it is more efficient though (if they’re being charged the same). I agree that hypothetically they could’ve just added 30% more vcpu to every request to achieve the same thing: yes that’s a solution. But this is another solution that gave them other benefits and that’s great. I still don’t see what the problem is.
elcritch 6y ago
Why would you presume that the new hardware would cost 40% of the server? Sure it’s an accounting trick to decide how to amortize the development costs. However they do it, I really doubt AWS would be continuing down the Nitro path if the TCO + savings wasn’t in the black. I’d like to see the energy efficiency of the new hardware and TCO estimates too, but probably that info is deemed “sensitive” and not released publicly. Likely the Nitro cards cost a fraction of that 30%, probably more like 5-15%. I’d wager that new hardware could have a huge energy efficiency gain for the specific Nitro workloads mentioned. The 30% resource utilization on Xen could be wasting a lot of time waiting for locks, context switch’s, etc between cores that could potentially be removed entirely from custom hardware optimized for handling multiple memory contexts simultaneously just by removing cache coherency at certain points.
- kevincox 6y ago
  I never said it did cost that much. I just said that they implied a 30% savings but that was without counting the new hardware. It is disingenuous to say that it is ~100% efficient just by moving the management cost elsewhere. I agree it was probably a win for them, I'm just pointing out that the way it was presented in the article was misleading.