3 ms·
> Is it for batch processing tasks that don't have tight deadlines? I used to work for a company (since shut down) which provided big data processing systems
by hiyer 5y ago
> Is it for batch processing tasks that don't have tight deadlines?
I used to work for a company (since shut down) which provided big data processing systems in the cloud. This was one of the typical use-cases for our customers. Big data systems like Hadoop, Spark, etc are built to handle this kind of disruption where you lose a few nodes once in a while, and we had built in further optimizations to do it even better. This fact, combined with the much cheaper price of Spot Instances (upto 90% less than on-demand) make them a compelling alternative to on-demand instances.
In practice - at least on AWS - spot loss used to be quite rare. When it happened, it happened by the truckload, but we used to have spot instances run for several days without termination.
- khuey 5y agoI've had AWS spot instances run for months without termination.