4 ms·
>This is probably a massive overreaction. No. I know people working on even tighter optimizations of LLMs. The magic is not in compute, its in design, and its
by DiscourseFan 2y ago
>This is probably a massive overreaction.
No. I know people working on even tighter optimizations of LLMs. The magic is not in compute, its in design, and its feasible that once all the kinks are ironed out something akin to O1 could be run effectively on a Raspberry Pi; a whole GPU would yield much better results but it probably won't be necessary to buy 3, or 4, or a whole rack of them. Nvidia's stock prices soared on the expectation (from those who weren't in the know) that if massive amounts of capital were simply moved to the right places it continuously improve. The thing that drives tech forward is people, however, not capital.
- BorisMelnik 2y agoif o1's can be run from a rasp pi, do you see farms of ARMs in the future?
- hakfoo 2y agoAre farms the future? I could see an "AI appliance" like the old Google Search Appliance: A single rack unit with a single $1000 GPU would probably be enough to run a pretty-robust DeepSeek style product, sold as "100% self-contained and on-prem, so you can trust it with propriatery data".