4 ms·

AWS EC2 Spot instances can now be stopped and started like On-Demand instances

97 points by HedgedHuman 7y ago

Havoc 7y ago
Rapidly moving toward compute as a commodity priced in the same way global commodity markets are. Interesting times
- pram 7y ago
  When can I start trading options on EC2 pricing though??
  - heavenlyblue 7y ago
    When EC2 becomes an actual market, rather than a monopoly?
  - chad_strategic 7y ago
    If Amazon or wall st. can figure how to make a cut off the trading. At that point they will trade options. Sounds like something I would want to trade... kinda that whole Enron thing... (except hopefully the fraudulent part)
kapilvt 7y ago
Very cool, been running into issues with this previously trying to manage mixed instances kinds in an asg. One tip, as an easy way to track api changes in aws, checkout https://awsapichanges.info/ https://awsapichanges.info/
chad_strategic 7y ago
Aws spot instances have saved me so much money... it’s such a great service.
cliqueiq 7y ago
Obviously a response to yesterdays post: https://news.ycombinator.com/item?id=22027459 https://news.ycombinator.com/item?id=22027459
- sudhirj 7y ago
  Huh? They’re completely unrelated.
- a012 7y ago
  No, they're two different things.
etaioinshrdlu 7y ago
Spot also supports hibernate on some instance types: https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec2-spot-lets-you-pause-and-resume-your-workloads/ https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec... Hibernate is just like good old hibernate from windows XP days. You save the RAM to disk. And resume your processes on next boot and they won't even notice they were stopped! (Except for the time has changed...) It might be really helpful for low priority batch processing. Disclaimer: I've never tested this.
- HedgedHuman 7y ago
  Neat! Did not know that. Thanks :-) The 100GB RAM cap seems rather limiting though. The ideal case scenario imho (one step beyond this stop/start-at-will feature) would be "convertible" spot - being able to migrate to on-demand with zero downtime when ec2 needs that capacity back - that'd solve a lot of my problems.
  - teraflop 7y ago
    You can (effectively) do that already -- just set your spot bid to the maximum you're willing to pay, i.e. the on-demand price.
    - paulfurtado 7y ago
      This isn't actually effective. When there are instance shortages and someone launches an on-demand instance, no matter how high you set your spot bid price your spot instance will still be terminated in favor of the on-demand instance. We run hundreds of spot instances with their max price set higher than on-demand and still see terminations on popular instance types: there are few day-to-day, but periodically nearly all of them are terminated at once. During those situations, if you went to replace the terminated spot instance with an on-demand instance, you'll even get an error stating that they're out of on-demand capacity for the instance type in that zone, so it's pretty clear that it happens when they've completely run out of capacity for some instance type in a zone
MaxBarraclough 7y ago
This is a useful addition. It's for persistent spot requests, so presumably it doesn't include defined-duration spot instances. Perhaps that will come later. Perfectly possible to work around it even if not, of course.

97 points by HedgedHuman 7y ago

Havoc 7y ago

Rapidly moving toward compute as a commodity priced in the same way global commodity markets are. Interesting times

pram 7y ago
When can I start trading options on EC2 pricing though??
- heavenlyblue 7y ago
  When EC2 becomes an actual market, rather than a monopoly?
- chad_strategic 7y ago
  If Amazon or wall st. can figure how to make a cut off the trading. At that point they will trade options. Sounds like something I would want to trade... kinda that whole Enron thing... (except hopefully the fraudulent part)

kapilvt 7y ago

Very cool, been running into issues with this previously trying to manage mixed instances kinds in an asg. One tip, as an easy way to track api changes in aws, checkout https://awsapichanges.info/ https://awsapichanges.info/

chad_strategic 7y ago

Aws spot instances have saved me so much money... it’s such a great service.

cliqueiq 7y ago

Obviously a response to yesterdays post: https://news.ycombinator.com/item?id=22027459 https://news.ycombinator.com/item?id=22027459

sudhirj 7y ago
Huh? They’re completely unrelated.
a012 7y ago
No, they're two different things.

etaioinshrdlu 7y ago

Spot also supports hibernate on some instance types: https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec2-spot-lets-you-pause-and-resume-your-workloads/ https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-ec... Hibernate is just like good old hibernate from windows XP days. You save the RAM to disk. And resume your processes on next boot and they won't even notice they were stopped! (Except for the time has changed...) It might be really helpful for low priority batch processing. Disclaimer: I've never tested this.

HedgedHuman 7y ago
Neat! Did not know that. Thanks :-) The 100GB RAM cap seems rather limiting though. The ideal case scenario imho (one step beyond this stop/start-at-will feature) would be "convertible" spot - being able to migrate to on-demand with zero downtime when ec2 needs that capacity back - that'd solve a lot of my problems.
- teraflop 7y ago
  You can (effectively) do that already -- just set your spot bid to the maximum you're willing to pay, i.e. the on-demand price.
  - paulfurtado 7y ago
    This isn't actually effective. When there are instance shortages and someone launches an on-demand instance, no matter how high you set your spot bid price your spot instance will still be terminated in favor of the on-demand instance. We run hundreds of spot instances with their max price set higher than on-demand and still see terminations on popular instance types: there are few day-to-day, but periodically nearly all of them are terminated at once. During those situations, if you went to replace the terminated spot instance with an on-demand instance, you'll even get an error stating that they're out of on-demand capacity for the instance type in that zone, so it's pretty clear that it happens when they've completely run out of capacity for some instance type in a zone

MaxBarraclough 7y ago

This is a useful addition. It's for persistent spot requests, so presumably it doesn't include defined-duration spot instances. Perhaps that will come later. Perfectly possible to work around it even if not, of course.