3 ms·
Modern ML builds on two pillars: GPUs and autodiff. Given that GPUs are running out of steam, I wonder what we should focus on now.
by adamnemecek 11mo ago
Modern ML builds on two pillars: GPUs and autodiff. Given that GPUs are running out of steam, I wonder what we should focus on now.
- Mars008 11mo agoThe price, power, and size. Make it cheap, low power, and small enough for mobile. One way to do this is inference in 4, 2, 1 bit. Also GPUs are parallel, most tasks can be split on several GPUs. Just by adding they you can scale up to infinity. In theory. So datacenters aren't going anywhere, they will still dominate. Another way is CPU+ + fast memory, like Apple does. It's limited but power efficient. Looks like with ecosystem development we need the whole spectrum from big models+tools running on datacenters to smaller running locally, to even smaller on mobile devices and robots.
- adamnemecek 11mo agoMy point is that revising autodiff is overdue.
- adamnemecek 11mo ago* revisiting