3 ms·
LLM's Are the New CPU's but Is OpenAI the New Intel?
- brucethemoose2 4y agoSpeaking of hardware, LLMs dont run on air. OpenAI is likely using a boatload of Nvidia A100s. And this is where the memory analogy starts to break down. OpenAI probably can't fund a hardware "LLM accelerator" to reduce costs. But you can bet Google, Amazon, Microsoft (who are silicon designers) are pondering just that, and Nvidia is probably thinking of hosting one themselves (which they can obviously do at far lower cost). Maybe OpenAI can leverage a startup like Cerebras or Tenstorrent, but I am skeptical.
- dragonwriter 4y ago> OpenAI probably can’t fund a hardware “LLM accelerator” to reduce costs. But you can bet Google, Amazon, Microsoft (who are silicon designers) are pondering just that Aren’t TPUs/Tensor Cores/Inferentia/etc. already neural-net accelerators?
- brucethemoose2 4y agoYep. I was thinking of something specifically designed for LLMs though. Aka massive memory pools, among other things. Right now, the I think the model has to be split up for inference, which is inefficient.
- villgax 4y agoJM2C OpenAI's portion of corpus that is built citing fair-use scraping & then preventing users from creating a competitor from their output does not sit well at all.