6 ms·
With today’s AI, we’re taking a big hammer to the problem using unoptimized but vastly flexible machines like GPUs. Once the code settles down, expect ASICs to
by 1-6 3y ago
With today’s AI, we’re taking a big hammer to the problem using unoptimized but vastly flexible machines like GPUs. Once the code settles down, expect ASICs to run much of the show and lower energy consumption.
- mwhitfield 3y agoYou're maybe not wrong, but I'm pretty sure I was reading this same comment 10 years ago, when we were just calling it deep learning.
- bongodongobob 3y agoYeah and it actually works now. It's not just weird dog faces hallucinated with Starry Night.
- quonn 3y agoThis is really unlikely, because the operations are pretty basic math and the GPUs do these about as efficiently as possible. Surely those GPUs will be improved further, but a custom ASIC probably won‘t help much with training. And NVIDIA has been producing GPUs that are specifically targeted at deep learning and they control both the software library and the hardware. I don‘t quite see how they couldn‘t optimize whatever they want to optimize.
- xadhominemx 3y agoThe chips won't be that much more efficient on a like for like basis but the models will be much smaller so the chips can be smaller or run much larger batches
- pixl97 3y agoBeing that us meatbags operate on a pretty limited power budget I'm guessing there is still a lot of algorithmic optimization to be done, and those algorithms will need hardware to run on.
- londons_explore 3y agoFor inference, there is the option to produce ASIC's with the weights hard coded into the chip. They'll be super fast and power efficient, with the huge downside they aren't reprogrammable.
- Cacti 3y agoThe models will simply fill the unused compute with additional training or more expensive models.
- UncleOxidant 3y agoThere are already some custom chips out there such as groq's[1]. And isn't Google using a lot of TPUs for this sort of thing now? Microsoft also said to be working on custom AI chips. [1]https://groq.com/ https://groq.com/
- kcb 3y agoThe H100 at this point includes just the bare minimum hardware to be considered a GPU. As long as AI is primarily matrix multiplication, GPUs are going to be near ideal.
- Dylan16807 3y agoIt's still spending most of its space on generic floating point units. And there's no hardware support for compressed weights as far as I know.
- moffkalast 3y agoIt is on Groq's LPU units iirc.
- ultra_nick 3y agoAgreed, using neuron layers as vectors seems like a waste. Moving to a graph-like compute model seems like it'd be more efficient.
- krunck 3y ago... and then the energy saved by using more efficient hardware and software will be used to power more hardware to increase total AI power. There is no "enough" AI yet. There may never be.
- bonton89 3y agoJevon's paradox basically. AI use cases that did not have RoI will open up as it becomes more efficient. The enough is always as much as it can.
- iamacyborg 3y ago> Once the code settles down, expect ASICs to run much of the show and lower energy consumption. If history is anything to go by energy consumption will remain the same or increase, we’ll just have more output
- paulmd 3y ago> expect ASICs to run much of the show and lower energy consumption. further increasing the efficiency of the process will only increase total consumption even further. the entirety of the AI boom has been a perfect demonstration of Jevons Paradox in action. Previously these intractable problems simply went unsolved. Now it is possible to solve them with a plausible degree of efficiency, and that increases the demand for solved problems hugely because the price per solved problem has declined. If you push energy even lower it will only fuel even further demand for more solved problems because wow, it’s cheap now!