4 ms·

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficientl

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?

aurareturn 2mo ago
Because it's actually Broadcom that is doing most of the work.
airspresso 2mo ago
By leveraging the experience Broadcom has in this area. Still remains to be seen how that goes when they want to scale production.
chris_money202 2mo ago
In the short and medium term, it probably won't be more economical to produce for OpenAI. Where OpenAI is benefitting from their own chip is being able to tailor it to their models and workloads. When you buy off the shelf Nvidia, its not perfectly tailored and OpenAI has to spend marginally more to run off that chip. At the scale OpenAI is operating at and plans to operate at, that margin becomes pretty big $$
dpe82 2mo ago
NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost. One objective of the project might be simply to provide credible negotiating leverage when dealing with existing suppliers like NVidia. You don't have to deploy at scale for that to work, but you do have to look like you could if pushed hard enough.
- vntok 2mo ago
  > NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost. But then that means you have no actual moat against the behemot, right? Your competitor can move into the market as soon as they want to, at much better cost (so at slightly better price)... and Nvidia certainly can adapt much faster around hard hardware specs innovation than a new entrant ever could.
  - acedTrex 2mo ago
    But they are also PURCHASING from nvidia so any time nvidia lowers their prices they save money.
  - dpe82 2mo ago
    Those are not OpenAI's concerns - they just need to scare NVidia enough to lower their prices more than they'd otherwise want.
toasterlovin 2mo ago
Replace OpenAI with Apple and Nvidia with Intel.
VirusNewbie 2mo ago
"How can OpenAI produce a LLM at scale more economically than Google, Amazon, and Microsoft which have experience in planet scale software and scale efficiencies unlike them". One answer is they're quite good at poaching talent.

How can OpenAI mass produce this chip at scale more economically than Nvidia which has experience in the supply chain and scale efficiencies to do it efficiently?

aurareturn 2mo ago

Because it's actually Broadcom that is doing most of the work.

airspresso 2mo ago

By leveraging the experience Broadcom has in this area. Still remains to be seen how that goes when they want to scale production.

chris_money202 2mo ago

In the short and medium term, it probably won't be more economical to produce for OpenAI. Where OpenAI is benefitting from their own chip is being able to tailor it to their models and workloads. When you buy off the shelf Nvidia, its not perfectly tailored and OpenAI has to spend marginally more to run off that chip. At the scale OpenAI is operating at and plans to operate at, that margin becomes pretty big $$

dpe82 2mo ago

NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost. One objective of the project might be simply to provide credible negotiating leverage when dealing with existing suppliers like NVidia. You don't have to deploy at scale for that to work, but you do have to look like you could if pushed hard enough.

vntok 2mo ago
> NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost. But then that means you have no actual moat against the behemot, right? Your competitor can move into the market as soon as they want to, at much better cost (so at slightly better price)... and Nvidia certainly can adapt much faster around hard hardware specs innovation than a new entrant ever could.
- acedTrex 2mo ago
  But they are also PURCHASING from nvidia so any time nvidia lowers their prices they save money.
- dpe82 2mo ago
  Those are not OpenAI's concerns - they just need to scare NVidia enough to lower their prices more than they'd otherwise want.

toasterlovin 2mo ago

Replace OpenAI with Apple and Nvidia with Intel.

VirusNewbie 2mo ago

"How can OpenAI produce a LLM at scale more economically than Google, Amazon, and Microsoft which have experience in planet scale software and scale efficiencies unlike them". One answer is they're quite good at poaching talent.