3 ms·

Here’s a cache: http://webcache.googleusercontent.com/search?q=cache:pxXMUXY7iKcJ:https://rare-technologies.com/machine-learning-hardware-benchmarks/&num=1&clie

Here’s a cache: http://webcache.googleusercontent.com/search?q=cache:pxXMUXY7iKcJ:https://rare-technologies.com/machine-learning-hardware-benchmarks/&num=1&client=safari&hl=en&gl=us&prmd=ivn&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:pxXMUXY... Looks like the instances they use are not GPU instances, so I’m not sure what the point of this is. Also they do not take into account EC2 spot pricing, which saves a ton of money and are the main reason I use AWS over alternate clouds.

taeric 9y ago
I misread your first line as "here's the catch." :). I feel like your second paragraph delivered on my misreading. Do you have any numbers on that take? Or just a general feel that it works out?
- drenvuk 9y ago
  Here are your numbers for spot instance pricing[1]. Machine learning stuff will typically happen in batches, openai even went through the effort to publish a k8s library for the task[2] of autoscaling clusters to minimize cost and manage it. [1]https://aws.amazon.com/ec2/spot/instance-advisor/ https://aws.amazon.com/ec2/spot/instance-advisor/ [2]https://github.com/openai/kubernetes-ec2-autoscaler https://github.com/openai/kubernetes-ec2-autoscaler
  - taeric 9y ago
    Thanks! I meant more if you had numbers on how you managed your workflow to be able to take advantage of this pricing. I'm assuming there is no magic "this is the cheaper option" for most folks. Instead, you have to have a pipeline built up that can take advantage of this style of pricing. It is ironic that sometimes those pipelines are themselves cheaper to build up than building on fixed cost alternatives. But I don't think it is a given. (I would be surprised, but not shocked, to find I am wrong on that.)
- OkGoDoIt 9y ago
  The specific projects I’m working on basically process data from an SQS queue. I have a bunch of spot instances set to launch at the minimum prices. If the prices jump (which they very rarely do) I simply have less throughput until the prices come back down. I also have a t2 instance as the web frontend. I’m significantly more budget constrained than I am time constrained, these are basically side projects. For specific numbers, the AllThePeople.net web scraping and profile compilation runs on Windows m1.small instances. The normal price is 8 cents per hour each while the spot instance costs me only a couple cents per hour each. I set the majority of my spot requests to the minimum possible price. I leave a couple at a higher price point of 8 cents per hour, so at least I have some work getting done if the prices go up. Those numbers are mostly for memory, it’s been a while since I fiddled with it much.
thesandlord 9y ago
Have you tried GCP's Preemptible VMs? They are like Spot but without the bidding. If you have, what do you like better about Spot? https://cloud.google.com/preemptible-vms/ https://cloud.google.com/preemptible-vms/ (I work for GCP)
- rpedela 9y ago
  FYI you no longer have to bid for AWS spot. https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlined-access-to-spot-capacity-smooth-price-changes-instance-hibernation/ https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlin...
  - thesandlord 9y ago
    Oh good to know! Looks like a similar model to GCE.
- tedivm 9y ago
  Since you work for GCP can you give us any insight into whether google will be offering any of the Volta line of GPUs?
- OkGoDoIt 9y ago
  Thanks for the head’s up, I had not heard of that. Then again GCP is not an option for me until they support Windows and C#.
  - 5ersi 9y ago
    Google Compute Engine are virtual machines so you can put any OS on them. In fact they have preconfigured Windows instances: https://cloud.google.com/compute/docs/quickstart-windows https://cloud.google.com/compute/docs/quickstart-windows
    - OkGoDoIt 9y ago
      Thanks, this is news to me. But then again I haven’t looked at Google cloud platform in years. I guess I should check it out again and see how it’s evolved.

taeric 9y ago

I misread your first line as "here's the catch." :). I feel like your second paragraph delivered on my misreading. Do you have any numbers on that take? Or just a general feel that it works out?

drenvuk 9y ago
Here are your numbers for spot instance pricing[1]. Machine learning stuff will typically happen in batches, openai even went through the effort to publish a k8s library for the task[2] of autoscaling clusters to minimize cost and manage it. [1]https://aws.amazon.com/ec2/spot/instance-advisor/ https://aws.amazon.com/ec2/spot/instance-advisor/ [2]https://github.com/openai/kubernetes-ec2-autoscaler https://github.com/openai/kubernetes-ec2-autoscaler
- taeric 9y ago
  Thanks! I meant more if you had numbers on how you managed your workflow to be able to take advantage of this pricing. I'm assuming there is no magic "this is the cheaper option" for most folks. Instead, you have to have a pipeline built up that can take advantage of this style of pricing. It is ironic that sometimes those pipelines are themselves cheaper to build up than building on fixed cost alternatives. But I don't think it is a given. (I would be surprised, but not shocked, to find I am wrong on that.)
OkGoDoIt 9y ago
The specific projects I’m working on basically process data from an SQS queue. I have a bunch of spot instances set to launch at the minimum prices. If the prices jump (which they very rarely do) I simply have less throughput until the prices come back down. I also have a t2 instance as the web frontend. I’m significantly more budget constrained than I am time constrained, these are basically side projects. For specific numbers, the AllThePeople.net web scraping and profile compilation runs on Windows m1.small instances. The normal price is 8 cents per hour each while the spot instance costs me only a couple cents per hour each. I set the majority of my spot requests to the minimum possible price. I leave a couple at a higher price point of 8 cents per hour, so at least I have some work getting done if the prices go up. Those numbers are mostly for memory, it’s been a while since I fiddled with it much.

thesandlord 9y ago

Have you tried GCP's Preemptible VMs? They are like Spot but without the bidding. If you have, what do you like better about Spot? https://cloud.google.com/preemptible-vms/ https://cloud.google.com/preemptible-vms/ (I work for GCP)

rpedela 9y ago
FYI you no longer have to bid for AWS spot. https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlined-access-to-spot-capacity-smooth-price-changes-instance-hibernation/ https://aws.amazon.com/blogs/aws/amazon-ec2-update-streamlin...
- thesandlord 9y ago
  Oh good to know! Looks like a similar model to GCE.
tedivm 9y ago
Since you work for GCP can you give us any insight into whether google will be offering any of the Volta line of GPUs?
OkGoDoIt 9y ago
Thanks for the head’s up, I had not heard of that. Then again GCP is not an option for me until they support Windows and C#.
- 5ersi 9y ago
  Google Compute Engine are virtual machines so you can put any OS on them. In fact they have preconfigured Windows instances: https://cloud.google.com/compute/docs/quickstart-windows https://cloud.google.com/compute/docs/quickstart-windows
  - OkGoDoIt 9y ago
    Thanks, this is news to me. But then again I haven’t looked at Google cloud platform in years. I guess I should check it out again and see how it’s evolved.