2 ms·
If "inference is cheap," why is OpenAI spending a ton getting Broadcom to design custom AI chips that make inference cheaper? Reports suggest their custom silic
by dathanb82 3mo ago
If "inference is cheap," why is OpenAI spending a ton getting Broadcom to design custom AI chips that make inference cheaper? Reports suggest their custom silicon isn't all that good for training, it's all to make inference more efficient. That shouldn't be necessary if inference is already quite cheap.
- petra 3mo agoA large part of the market will be ad based. For that, having the lowest cost inference is useful. Also for agent doing r&d, cheaper tokens allows doing more, which is always good.