4 ms·
[2017]
by gryfft 3y ago
[2017]
- danielbln 3y agoWhat a rollercoaster.
- riku_iki 3y ago137 billion parameters were outrageously large at that time.
- tomohelix 3y agoAnd in 2029 people would call 10 trillion params models "medium size"... I mean, with about $120K, we can technically run a 137B model locally. Any small company or rich individuals can afford this easily. So the "outrageous" back then is pretty "middle" now.
- sheepscreek 3y agoI can run a 30B parameter model on my M1 laptop at a pretty decent token rate. I can run two instances of the same model in parallel without affecting the speed. A maxed out M2 Mac Studio can easily run 70B models and very likely 137B models too. The key is quantization. So it’s all within reach for under $7k if you’re willing to accept a slightly lower quality (~5 bit quantization).
- dang 3y agoAdded above. Thanks!