4 ms·
You mean now. Commodity soon.
by objektif 3y ago
You mean now. Commodity soon.
- empath-nirvana 3y agoBecause of scaling advantages, it likely never will be. Even if you could run a GPT4 quality model on your laptop (you can't), by the time you can, Open Ai will be running a model that uses orders of magnitude more compute.
- jjjjj55555 3y agoStupid question, but why aren't they doing it already? What's their bottleneck right now?
- danielmarkbruce 3y agoTry shoving a couple trill (estimate) parameters in memory. Or, shuttling it back and forth from disk as required to generate a single token. And then shuttling that into the chip. Assuming they are using FP16 it's 4 TB give or take. They aren't kidding when they call them "Large".
- mindwok 3y agoThis could easily end up like the PC revolution though. ChatGPT being the mainframe of AI, and when the hardware catches up and performance is “good enough” we’re all running our own models all over the place.
- objektif 3y agoI dont think compute capacity will be the determining factor for moat. It will most likely be access to various types of data and I believe on that front Meta and google are way ahwad
- danielmarkbruce 3y agoIt's going the other way. The number of levers to pull around architecture, data amount/type/quality, hardware optimizations, decoding optimizations, evals, system architecture (distinct from model architecture), alignment tricks and various other things means the complexity is just getting started.