4 ms·

Amazon Bets $150B on Data Centers Required for AI Boom

46 points by thelastgallon 3y ago

andrewstuart 3y ago
I'm surprised they are not developing their own GPUs.
- grogenaut 3y ago
  They are... https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/
  - my123 3y ago
    Those aren't anywhere near the flexibility of a GPU. It's a more specialized accelerator tailored to machine learning workloads.
    - w-ll 3y ago
      thats the point. they dont want GPUs, they want AIPUs
    - wmf 3y ago
      Which is what people want.
      - schneems 3y ago
        Brawndo, it's got what plants crave.
    - rockwotj 3y ago
      Google already did this with TPUs and it supposedly saved them a ton in energy costs. I read somewhere that NVIDIA doesn’t gain as much by making the chips more cost effective, just making them more powerful
    - mr_toad 3y ago
      > It's a more specialized accelerator tailored to machine learning workloads. Hard to imagine anyone using a cloud GPU for anything else.
  - singhrac 3y ago
    and Trainium for training of course
- wenc 3y ago
  And compete with CUDA? It's not the hardware that's the lockin but CUDA.
  - AgentOrange1234 3y ago
    How much of the big expensive training jobs are CUDA specific? If it’s billions of dollars of compute, rewriting the software to use whatever hardware is cheapest may make sense?
    - vineyardmike 3y ago
      It takes time to re-engineer an entire ecosystem of tools. The whole 9-pregnant ladies in 1 month analogy comes to mind. If you’re trying to accomplish a goal, how long are you willing to wait for your entire dependency tree to be engineered in-house. It’s happening slowly, but teams have to ship, and can’t wait for other teams to build fresh tools. Additionally, the compute hardware is rented, and if there’s no alternatives available for rent it doesn’t matter. Data centers aren’t full of NVidia and AMD GPUs and TPUs (because the support isn’t there). It’s a crazy chicken/egg situation where everyone benefits but no one makes the move. It’s slowly happening, but it’s not there yet to totally replace them.
      - delfinom 3y ago
        That's the thing, they can be working on multiple paths in parallel. They can be building on nvidia and have a semiconductor team in another corner experimenting for alternate in the future. When profit margins are insane, there is always competition quietly fomenting. We just won't know until they release it because it's also a competitive advantage to keep your own plans underwrap until it's ready. Or else it may start an arms race that'll only drive up costs to get done faster.
  - xbpx 3y ago
    The CUDA lock-in is over played. Tensorflow, Pytorch and any large framework supports multiple hardware including Google TPUs. Any company making significant investment will steer some of that towards hardware support in the software they need.
anonylizard 3y ago
That's just $10b a year for 15 years, fairly conservative I'd say. I expect a GPT-5 training cluster to cost $10b (Say 100k blackwell chips + associated infrastructure), and a GPT-6 capable cluster to cost $100b. It sounds like a lot, but its just a typical mega-infrastructure funding. The Californian high speed rail is also around $100b for example, and Microsoft has $90b in cash reserves. The gulf states are also dogpiling in AI, because they finally see a true viable alternative to oil: renting out GPU clusters. Just swapping to renewables is not enough, as their economies is predicated on having a huge, simple, defendable profit source that is exportable. Now they can redeploy the spare oil money to a use case, that has predictable demand, extremely capital intensive, and far more productive than building new cities in the desert. The world in the end built $trillions in value of power stations and roads. I expect the same for data centers, having entire farms being replaced by humming GPU racks. But that will take many decades.
- lyime 3y ago
  [flagged]
- hn_throwaway_99 3y ago
  > I expect a GPT-5 training cluster to cost $10b (Say 100k blackwell chips + associated infrastructure), and a GPT-6 capable cluster to cost $100b. You just completely made up those numbers.
  - anonylizard 3y ago
    GPT-4 cost about $100 mil to train. That's opex (Renting the GPUs). Expect the corresponding capex (buying the GPUs) on Azure's end to be 10x of that. Hence $1 bil GPT-5 being 10x the training cost of GPT-4 is a reasonable estimate. GPT-4 was itself 10x the size of GPT-3. So $10 bil. GPT-6 being another 10x over GPT-5 is therefore just a extrapolation, hence $100 bil.
    - xcv123 3y ago
      The cost of computing decreases over time. No one will pay $100 Billion to train GPT-6. That is absurd. The current top supercomputer in the world (Frontier) cost $600M. It is rumoured that GPT-4 was trained on 10,000 A100 processors released in 2020. Total cost is $100M at $10k each. Today they can buy more powerful hardware and train much larger models for the same cost. https://nvidianews.nvidia.com/news/nvidia-blackwell-platform-arrives-to-power-a-new-era-of-computing https://nvidianews.nvidia.com/news/nvidia-blackwell-platform... https://www.cerebras.net/product-chip https://www.cerebras.net/product-chip
singularity2001 3y ago
I can't wait for the whole Nvidia and data center stack to be disrupted by a completely new kind of computing device specifically for deep learning: analog and or photonic integrators. (probably associated with an adaptation of architecture like Hintons forward forward networks). Time is right and the incentives are in the trillions. hook me up if you share the vision or know a guy.

46 points by thelastgallon 3y ago

andrewstuart 3y ago

I'm surprised they are not developing their own GPUs.

grogenaut 3y ago
They are... https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/
- my123 3y ago
  Those aren't anywhere near the flexibility of a GPU. It's a more specialized accelerator tailored to machine learning workloads.
  - w-ll 3y ago
    thats the point. they dont want GPUs, they want AIPUs
  - wmf 3y ago
    Which is what people want.
    - schneems 3y ago
      Brawndo, it's got what plants crave.
  - rockwotj 3y ago
    Google already did this with TPUs and it supposedly saved them a ton in energy costs. I read somewhere that NVIDIA doesn’t gain as much by making the chips more cost effective, just making them more powerful
  - mr_toad 3y ago
    > It's a more specialized accelerator tailored to machine learning workloads. Hard to imagine anyone using a cloud GPU for anything else.
- singhrac 3y ago
  and Trainium for training of course
wenc 3y ago
And compete with CUDA? It's not the hardware that's the lockin but CUDA.
- AgentOrange1234 3y ago
  How much of the big expensive training jobs are CUDA specific? If it’s billions of dollars of compute, rewriting the software to use whatever hardware is cheapest may make sense?
  - vineyardmike 3y ago
    It takes time to re-engineer an entire ecosystem of tools. The whole 9-pregnant ladies in 1 month analogy comes to mind. If you’re trying to accomplish a goal, how long are you willing to wait for your entire dependency tree to be engineered in-house. It’s happening slowly, but teams have to ship, and can’t wait for other teams to build fresh tools. Additionally, the compute hardware is rented, and if there’s no alternatives available for rent it doesn’t matter. Data centers aren’t full of NVidia and AMD GPUs and TPUs (because the support isn’t there). It’s a crazy chicken/egg situation where everyone benefits but no one makes the move. It’s slowly happening, but it’s not there yet to totally replace them.
    - delfinom 3y ago
      That's the thing, they can be working on multiple paths in parallel. They can be building on nvidia and have a semiconductor team in another corner experimenting for alternate in the future. When profit margins are insane, there is always competition quietly fomenting. We just won't know until they release it because it's also a competitive advantage to keep your own plans underwrap until it's ready. Or else it may start an arms race that'll only drive up costs to get done faster.
- xbpx 3y ago
  The CUDA lock-in is over played. Tensorflow, Pytorch and any large framework supports multiple hardware including Google TPUs. Any company making significant investment will steer some of that towards hardware support in the software they need.

anonylizard 3y ago

That's just $10b a year for 15 years, fairly conservative I'd say. I expect a GPT-5 training cluster to cost $10b (Say 100k blackwell chips + associated infrastructure), and a GPT-6 capable cluster to cost $100b. It sounds like a lot, but its just a typical mega-infrastructure funding. The Californian high speed rail is also around $100b for example, and Microsoft has $90b in cash reserves. The gulf states are also dogpiling in AI, because they finally see a true viable alternative to oil: renting out GPU clusters. Just swapping to renewables is not enough, as their economies is predicated on having a huge, simple, defendable profit source that is exportable. Now they can redeploy the spare oil money to a use case, that has predictable demand, extremely capital intensive, and far more productive than building new cities in the desert. The world in the end built $trillions in value of power stations and roads. I expect the same for data centers, having entire farms being replaced by humming GPU racks. But that will take many decades.

lyime 3y ago
[flagged]
hn_throwaway_99 3y ago
> I expect a GPT-5 training cluster to cost $10b (Say 100k blackwell chips + associated infrastructure), and a GPT-6 capable cluster to cost $100b. You just completely made up those numbers.
- anonylizard 3y ago
  GPT-4 cost about $100 mil to train. That's opex (Renting the GPUs). Expect the corresponding capex (buying the GPUs) on Azure's end to be 10x of that. Hence $1 bil GPT-5 being 10x the training cost of GPT-4 is a reasonable estimate. GPT-4 was itself 10x the size of GPT-3. So $10 bil. GPT-6 being another 10x over GPT-5 is therefore just a extrapolation, hence $100 bil.
  - xcv123 3y ago
    The cost of computing decreases over time. No one will pay $100 Billion to train GPT-6. That is absurd. The current top supercomputer in the world (Frontier) cost $600M. It is rumoured that GPT-4 was trained on 10,000 A100 processors released in 2020. Total cost is $100M at $10k each. Today they can buy more powerful hardware and train much larger models for the same cost. https://nvidianews.nvidia.com/news/nvidia-blackwell-platform-arrives-to-power-a-new-era-of-computing https://nvidianews.nvidia.com/news/nvidia-blackwell-platform... https://www.cerebras.net/product-chip https://www.cerebras.net/product-chip

singularity2001 3y ago

I can't wait for the whole Nvidia and data center stack to be disrupted by a completely new kind of computing device specifically for deep learning: analog and or photonic integrators. (probably associated with an adaptation of architecture like Hintons forward forward networks). Time is right and the incentives are in the trillions. hook me up if you share the vision or know a guy.