4 ms·

> They should be building programmable AI chips with FPGA Eh, people have been saying this for over a decade, and I think that opportunity has passed. FPGAs a

> They should be building programmable AI chips with FPGA Eh, people have been saying this for over a decade, and I think that opportunity has passed. FPGAs are not going to beat GPUs anytime soon due to the software ecosystem (among other things), and ultimately they are not going to outrun ASICs that are now economically viable (especially in the embedded space).

crotchfire 3y ago
> FPGAs are not going to beat GPUs anytime soon due to the software ecosystem Completely true as long as the chip companies stonewall open-source toolchains. If Xilinx or Altera published the bitstream format for their highest-volume high-end chip (i.e. one or two generations back from the bleeding edge) you'd see the effect on the entire AI space within twelve months. I'm not kidding. The interest in this kind of access from developers is enormous, and the problems in this space are extremely regular. There are massive opportunities for FPGAs to avoid spilling to DRAM or even to SRAM -- you have an ocean of tiny register-speed memories in LUTRAM mode. But it will never happen. And so FPGAs will continue to be trinkets for weapons manufacturers and not much else.
- someguydave 3y ago
  If they adopt open-source tool chains Xilinx & Altera would be signing their own death-warrants. The hardware would be reproduced by third parties (the open source code would contain all you need to know to produce the chips) and it would become a commodity like DRAM. Profit margins for the incumbent companies would go to zero and the users would profit immensely. Instead, FPGAs are sold to big entities like governments and big corps who can throw bodies at the crappy toolchains until they produce what they want.
btown 3y ago
Now I'm wondering about a world where OpenAI can treat their compute not as "dedicate this time slice of an A100 to generating tokens for customer A's query" but rather "there is a custom pipeline at a gate level that exactly matches our model architecture, hardcoding weights at the gate level, and can treat parallel requests the same way a processor would pipeline instructions." I don't think any human could write the TensorFlow compiler for this, but theoretically a generative AI could learn how to, experimenting on its own compute hardware? I don't know that we've seen the last of FPGAs.

crotchfire 3y ago

> FPGAs are not going to beat GPUs anytime soon due to the software ecosystem Completely true as long as the chip companies stonewall open-source toolchains. If Xilinx or Altera published the bitstream format for their highest-volume high-end chip (i.e. one or two generations back from the bleeding edge) you'd see the effect on the entire AI space within twelve months. I'm not kidding. The interest in this kind of access from developers is enormous, and the problems in this space are extremely regular. There are massive opportunities for FPGAs to avoid spilling to DRAM or even to SRAM -- you have an ocean of tiny register-speed memories in LUTRAM mode. But it will never happen. And so FPGAs will continue to be trinkets for weapons manufacturers and not much else.

someguydave 3y ago
If they adopt open-source tool chains Xilinx & Altera would be signing their own death-warrants. The hardware would be reproduced by third parties (the open source code would contain all you need to know to produce the chips) and it would become a commodity like DRAM. Profit margins for the incumbent companies would go to zero and the users would profit immensely. Instead, FPGAs are sold to big entities like governments and big corps who can throw bodies at the crappy toolchains until they produce what they want.

btown 3y ago

Now I'm wondering about a world where OpenAI can treat their compute not as "dedicate this time slice of an A100 to generating tokens for customer A's query" but rather "there is a custom pipeline at a gate level that exactly matches our model architecture, hardcoding weights at the gate level, and can treat parallel requests the same way a processor would pipeline instructions." I don't think any human could write the TensorFlow compiler for this, but theoretically a generative AI could learn how to, experimenting on its own compute hardware? I don't know that we've seen the last of FPGAs.