4 ms·
Yeah but then it's basically an AI ASIC with an FPGA block inside it. Basically the less FPGA-like an FPGA is, ie the more dedicated silicon in the FPGA for the
by cgyvbunji 2mo ago
Yeah but then it's basically an AI ASIC with an FPGA block inside it. Basically the less FPGA-like an FPGA is, ie the more dedicated silicon in the FPGA for the task in question, the more power efficient it is, because custom logic in an FPGA is done in LUTs which is RAM and RAM is way way more power hungry than actual logic gates, and the fabric is apparently power hungry too. It's unfortunate to me because I like FPGAs and wish they weren't so niche, but they are inherently limited in this way.
- adrian_b 2mo agoI agree.
- imtringued 2mo agoIf you understood FPGAs you wouldn't think of them as incredibly niche. The more application specific you get, the smaller the total volume of chips. The very nature of application specifity ruins the economics of ASICs. Every time someone tells me an ASIC is more energy efficient I'm thinking, you just ruined the business case. The vast majority of application specific designs are not economically viable unless you use FPGAs to implement them.
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- cgyvbunji 2mo ago> The more application specific you get, the smaller the total volume of chips Sure, and that is the definition of niche. FPGAs are niche. You could have made a good point that an application specific design that already requires an FPGA might want to now also have a local LLM, so putting the LLM right on the FPGA might be the most expedient option in that case. Other than that, I don't think people are reaching for FPGAs to do LLM training or inference in general because I don't think it can be cost effective vs other options.