5 ms·
It interests me that the $200-600 billion number seems to be all-derived from GPUs. Are LLMs/AIs totally dependent on GPUs? I read last week (https://news.ycomb
by beejiu 2y ago
It interests me that the $200-600 billion number seems to be all-derived from GPUs. Are LLMs/AIs totally dependent on GPUs? I read last week (https://news.ycombinator.com/item?id=40787349 https://news.ycombinator.com/item?id=40787349) that there is research ongoing to run LLMs on FPGAs at greater energy efficiency.
I'm reminded of Bitcoin/crpyto, which in its early history was all operated on GPUs. And, then, almost overnight, the whole thing was run on ASICs.
Is there an intrinsic reason something similar couldn't happen with LLMs? If so, the idea of a bubble seems even more concerning.
- cma 2y agoNvidia and AMD bought out the big FPGA makers.
- jamessinghal 2y agoAMD bought Xilinx, but Intel recently spun off Altera.
- cma 2y agoOpps, yeah was thinking of Intel's Altera buy, not Nvidia, which I guess is now undone.
- wmf 2y agoThese numbers are just the hole from GPUs that have already been bought/ordered. Today's GPUs will inevitably be replaced by something, whether it be better GPUs, NPUs/TPUs, ASICs, or FPGAs. As chips get cheaper in the future the hole will grow at a slower rate but it only grows.
- sason 2y agoThere is a fairly new ASIC named "Sohu" that is purpose-built for transformers. They have some bold claims that are impressive if true. I found a short discussion[2] you may find useful. [1]: https://www.etched.com/ https://www.etched.com/ [2]: https://www.lesswrong.com/posts/qhpB9NjcCHjdNDsMG/new-fast-transformer-inference-asic-sohu-by-etched https://www.lesswrong.com/posts/qhpB9NjcCHjdNDsMG/new-fast-t...
- cmcclellan 2y agoThis is just a PowerPoint slide at this point.
- pixl97 2y ago>Is there an intrinsic reason something similar couldn't happen with LLMs? Only if we can increase the efficiency of LLMs by 2-3 orders of magnitude, there are only some in lab examples of this and nothing really being publicly shown. Even then the models are still going to require rather large amounts of memory, and any performance increases that could boost model efficiency would very likely increase performance on GPU hardware to the point we could get continuous learning models from multimodal input like video data and other sensors.
- light_hue_1 2y agoThere is a reason why we don't use ASICs and instead use GPUs. While people may say something is a Transformer that's more of a general description. It's not a specific algorithm; there are countless transformers and people are making progress on finding new ones. Bitcoin runs a specific algorithm that never changes. That's for an ASIC. AI/ML runs a large class of models. GPUs are already finely tuned for his case.