4 ms·

Is anyone aware of any examples of FPGAs used to implement neural nets or other connectionist architectures? What's exciting about FPGAs is that they make "cus

Is anyone aware of any examples of FPGAs used to implement neural nets or other connectionist architectures? What's exciting about FPGAs is that they make "custom" chips affordable for colleges and even individuals... At this point in time, with the AI world fixated on LLMs, it seems that there's an opportunity for small groups/individuals to use FPGAs to experiment with more animal-like ground-up approaches to intelligence.

le-mark 1y ago
Why the downvotes? My understanding ai is not a use for fpgas because models require a lot of ram (preferably fast ram) and fpga themselves don’t have a lot of it. An fpga will have pins you can attach fast ram, but then board layout and layering traces becomes very non trivial. Hence gfx cards (and a lot of them) are more suited.
- throwaway48476 1y ago
  FPGA will never be as fast as an ASIC. NV data center chips cant do graphics anymore, just compute.
  - random_duck 1y ago
    Funny you say that, if you look at the git repo for the blog there are a bunch of asic screenshots currently being committed.
  - HarHarVeryFunny 1y ago
    Sure - not as fast or flexible, but relatively cheap and reprogrammable - good for experimentation, maybe for building a real-world proof-of-concept of a learning architecture ?
  - imtringued 1y ago
    This doesn't actually matter in practice, because either nobody produces your specific architecture in the first place, or the existing architectures are significantly more power inefficient. FPGAs run at reduced clock speeds, but then again, you'd have to run the GPU or CPU at the same clock speed to obtain the same energy efficiency.
- HarHarVeryFunny 1y ago
  FPGA certainly not ideal for an experimental AI architecture, but if data is just flowing thru it (sensory input in, motor control out) then I don't see RAM as an issue. Could be an interesting option for experimenting on a mobile robot.
Aurornis 1y ago
> Is anyone aware of any examples of FPGAs used to implement neural nets or other connectionist architectures? Do you mean as a specialized processor for high speed inference, or as if the neural net including weights was directly synthesized into the FPGA architecture? Neither use case would be as powerful as other options. FPGAs don’t provide infinite leeway for implementing completely custom logic. They have a finite number of blocks that can be reconfigured in certain ways as synthesized by the software. This architecture isn’t a good fit for the large LLMs we think about which need a lot of high bandwidth memory access. You can connect an FPGA to high speed memory, but the current crop of GPUs are going to be much better at the job.
nickpsecurity 1y ago
https://arxiv.org/abs/2404.10076 https://arxiv.org/abs/2404.10076
Unaimend 1y ago
Have a look into differentiable logic gate networks
bschup 1y ago
Certainly, we use networks on FPGA at FNAL for inference on LHC/CMS high energy physics results. There are also cases involving microsecond control of dynamical systems. https://fastmachinelearning.org/hls4ml/ https://fastmachinelearning.org/hls4ml/ https://indico.cern.ch/event/797510/contributions/3313676/attachments/1795772/2927344/irishep_dsr_hls4ml_13feb2019.pdf https://indico.cern.ch/event/797510/contributions/3313676/at... https://cdrdv2-public.intel.com/833277/deploying-tinyml-fpgas-whitepaper.pdf https://cdrdv2-public.intel.com/833277/deploying-tinyml-fpga... https://ml4physicalsciences.github.io/2019/files/NeurIPS_ML4PS_2019_74.pdf https://ml4physicalsciences.github.io/2019/files/NeurIPS_ML4...

le-mark 1y ago

Why the downvotes? My understanding ai is not a use for fpgas because models require a lot of ram (preferably fast ram) and fpga themselves don’t have a lot of it. An fpga will have pins you can attach fast ram, but then board layout and layering traces becomes very non trivial. Hence gfx cards (and a lot of them) are more suited.

throwaway48476 1y ago
FPGA will never be as fast as an ASIC. NV data center chips cant do graphics anymore, just compute.
- random_duck 1y ago
  Funny you say that, if you look at the git repo for the blog there are a bunch of asic screenshots currently being committed.
- HarHarVeryFunny 1y ago
  Sure - not as fast or flexible, but relatively cheap and reprogrammable - good for experimentation, maybe for building a real-world proof-of-concept of a learning architecture ?
- imtringued 1y ago
  This doesn't actually matter in practice, because either nobody produces your specific architecture in the first place, or the existing architectures are significantly more power inefficient. FPGAs run at reduced clock speeds, but then again, you'd have to run the GPU or CPU at the same clock speed to obtain the same energy efficiency.
HarHarVeryFunny 1y ago
FPGA certainly not ideal for an experimental AI architecture, but if data is just flowing thru it (sensory input in, motor control out) then I don't see RAM as an issue. Could be an interesting option for experimenting on a mobile robot.

Aurornis 1y ago

> Is anyone aware of any examples of FPGAs used to implement neural nets or other connectionist architectures? Do you mean as a specialized processor for high speed inference, or as if the neural net including weights was directly synthesized into the FPGA architecture? Neither use case would be as powerful as other options. FPGAs don’t provide infinite leeway for implementing completely custom logic. They have a finite number of blocks that can be reconfigured in certain ways as synthesized by the software. This architecture isn’t a good fit for the large LLMs we think about which need a lot of high bandwidth memory access. You can connect an FPGA to high speed memory, but the current crop of GPUs are going to be much better at the job.

nickpsecurity 1y ago

https://arxiv.org/abs/2404.10076 https://arxiv.org/abs/2404.10076

Unaimend 1y ago

Have a look into differentiable logic gate networks

bschup 1y ago

Certainly, we use networks on FPGA at FNAL for inference on LHC/CMS high energy physics results. There are also cases involving microsecond control of dynamical systems. https://fastmachinelearning.org/hls4ml/ https://fastmachinelearning.org/hls4ml/ https://indico.cern.ch/event/797510/contributions/3313676/attachments/1795772/2927344/irishep_dsr_hls4ml_13feb2019.pdf https://indico.cern.ch/event/797510/contributions/3313676/at... https://cdrdv2-public.intel.com/833277/deploying-tinyml-fpgas-whitepaper.pdf https://cdrdv2-public.intel.com/833277/deploying-tinyml-fpga... https://ml4physicalsciences.github.io/2019/files/NeurIPS_ML4PS_2019_74.pdf https://ml4physicalsciences.github.io/2019/files/NeurIPS_ML4...