9 ms·
> OpenAI probably can’t fund a hardware “LLM accelerator” to reduce costs. But you can bet Google, Amazon, Microsoft (who are silicon designers) are pondering j
by dragonwriter 4y ago
> OpenAI probably can’t fund a hardware “LLM accelerator” to reduce costs. But you can bet Google, Amazon, Microsoft (who are silicon designers) are pondering just that
Aren’t TPUs/Tensor Cores/Inferentia/etc. already neural-net accelerators?
- brucethemoose2 4y agoYep. I was thinking of something specifically designed for LLMs though. Aka massive memory pools, among other things. Right now, the I think the model has to be split up for inference, which is inefficient.