4 ms·

I am surprised Amazon has not jumped in the game, renting out an accelerator like Google does with the TPUs

PeterStuer 7y ago
You mean something not like EC2 Accelerated Computing Instances? https://aws.amazon.com/ec2/instance-types/#Accelerated_Computing https://aws.amazon.com/ec2/instance-types/#Accelerated_Compu... Edit: I didn't see the parent meant a hardware accelerator created by Amazon itself. Thx to @jsty and @paol for pointing this out. An ASIC by Amazon was announced last year and is known as 'AWS Inferentia' https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/ https://perspectives.mvdirona.com/2018/11/aws-inferentia-machine-learning-processor/ https://perspectives.mvdirona.com/2018/11/aws-inferentia-mac...
- paol 7y ago
  Those are just regular GPUs (and one FPGA offering)
- jsty 7y ago
  I think they were referring to the fact that those are all bought in from outside vendors, rather than having their own custom accelerator chips produced like e.g. Google's TPUs.
  - deleted 7y ago
    [deleted]
nl 7y ago
They do, although inference (not training) so far. They have a custom chip call AWS Inferentia[0]. I believe it is available via Elastic Inference[1][2] (or maybe soon will be). [0] https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/ [1] https://docs.aws.amazon.com/elastic-inference/latest/developerguide/what-is-ei.html https://docs.aws.amazon.com/elastic-inference/latest/develop... [2] https://aws.amazon.com/machine-learning/elastic-inference/ https://aws.amazon.com/machine-learning/elastic-inference/
baybal2 7y ago
> I am surprised Amazon has not jumped in the game Why should they? There is not a lot of money to be gained from renting niche product in comparison to enormous capital expenditure for anything hardware related. Lot's of dotcom companies burned themselves badly while chasing trendsetters with custom silicon. A cookie cutter 40nm SoC may cost "just" 10M today, but by involving yourself into custom silicon game you risk loosing in it. Not to mention that your operations troubles will increase n-fold. Managing operations of hosting business with hundreds of thousands customer is hard enough. Logistics, server lifecycle, DC management, managing procurement contracts with unruly OEMs... Now try to dock all troubles you have with chipmakers to it. It will become a nightmare.
Jack000 7y ago
AWS gpu compute is extremely expensive. If this is due to datacenter licensing costs, I hope they come out with their own hardware soon to reduce these costs. If on the otherhand, it's because their value-add is not in renting out the hardware but burst scalability, then I'm less optimistic that they'd cannibalize their own cloud product. Currently, it only takes about 1 month to break even if you buy a consumer gpu like the rtx 2080ti, compared with AWS time. For training purposes it doesn't seem to make sense. - just looked up the numbers and google tpus are pretty similar in terms of pricing. I think any aws equivalent would probably be just as expensive compared to a diy pc.
- dannyw 7y ago
  That's because NVIDIA's EULA requires you to buy a NVIDIA Tesla in datacentre developments; and Teslas are marked up by thousands of dollars.
- sabalaba 7y ago
  Exactly. AWS is simply too expensive. You can buy a Lambda Quad GPU workstation and pay it back in a couple of months. If you want to save more you can just build it yourself. https://lambdalabs.com/blog/8-v100-server-on-prem-vs-p3-instance-tco-analysis-cost-comparison/ https://lambdalabs.com/blog/8-v100-server-on-prem-vs-p3-inst...

You mean something not like EC2 Accelerated Computing Instances? https://aws.amazon.com/ec2/instance-types/#Accelerated_Computing https://aws.amazon.com/ec2/instance-types/#Accelerated_Compu... Edit: I didn't see the parent meant a hardware accelerator created by Amazon itself. Thx to @jsty and @paol for pointing this out. An ASIC by Amazon was announced last year and is known as 'AWS Inferentia' https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/ https://perspectives.mvdirona.com/2018/11/aws-inferentia-machine-learning-processor/ https://perspectives.mvdirona.com/2018/11/aws-inferentia-mac...

paol 7y ago
Those are just regular GPUs (and one FPGA offering)
jsty 7y ago
I think they were referring to the fact that those are all bought in from outside vendors, rather than having their own custom accelerator chips produced like e.g. Google's TPUs.
- deleted 7y ago
  [deleted]

nl 7y ago

They do, although inference (not training) so far. They have a custom chip call AWS Inferentia[0]. I believe it is available via Elastic Inference[1][2] (or maybe soon will be). [0] https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/ [1] https://docs.aws.amazon.com/elastic-inference/latest/developerguide/what-is-ei.html https://docs.aws.amazon.com/elastic-inference/latest/develop... [2] https://aws.amazon.com/machine-learning/elastic-inference/ https://aws.amazon.com/machine-learning/elastic-inference/

baybal2 7y ago

> I am surprised Amazon has not jumped in the game Why should they? There is not a lot of money to be gained from renting niche product in comparison to enormous capital expenditure for anything hardware related. Lot's of dotcom companies burned themselves badly while chasing trendsetters with custom silicon. A cookie cutter 40nm SoC may cost "just" 10M today, but by involving yourself into custom silicon game you risk loosing in it. Not to mention that your operations troubles will increase n-fold. Managing operations of hosting business with hundreds of thousands customer is hard enough. Logistics, server lifecycle, DC management, managing procurement contracts with unruly OEMs... Now try to dock all troubles you have with chipmakers to it. It will become a nightmare.

Jack000 7y ago

AWS gpu compute is extremely expensive. If this is due to datacenter licensing costs, I hope they come out with their own hardware soon to reduce these costs. If on the otherhand, it's because their value-add is not in renting out the hardware but burst scalability, then I'm less optimistic that they'd cannibalize their own cloud product. Currently, it only takes about 1 month to break even if you buy a consumer gpu like the rtx 2080ti, compared with AWS time. For training purposes it doesn't seem to make sense. - just looked up the numbers and google tpus are pretty similar in terms of pricing. I think any aws equivalent would probably be just as expensive compared to a diy pc.

dannyw 7y ago
That's because NVIDIA's EULA requires you to buy a NVIDIA Tesla in datacentre developments; and Teslas are marked up by thousands of dollars.
sabalaba 7y ago
Exactly. AWS is simply too expensive. You can buy a Lambda Quad GPU workstation and pay it back in a couple of months. If you want to save more you can just build it yourself. https://lambdalabs.com/blog/8-v100-server-on-prem-vs-p3-instance-tco-analysis-cost-comparison/ https://lambdalabs.com/blog/8-v100-server-on-prem-vs-p3-inst...