5 ms·

Parallel Programming for FPGAs

80 points by bjourne 4y ago

moring 4y ago
Slightly off-topic: I see lots of learning resources for FPGAs appearing that explain how to "think parallel", but are there good resources that explain how to deal with practical problems such as badly-documented primitives (looking at you, ECP5) or how to debug a non-working design (1), or how to find what causes the synthesis tool to segfault? 1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use. The situation somewhat reminds me of the earlier LaTeX days when every tutorial started with "I'm assuming you have already installed LaTeX" as if that wasn't the biggest problem for most users, and then continues explaining the easy stuff that you can easily learn by trial and error.
- dayjaby 4y ago
  > how to debug a non-working design In LiteX, an open source toolset for making FPGA designs, you can embed an oscilloscope into your FPGA: https://github.com/enjoy-digital/litex/wiki/Use-LiteScope-To-Debug-A-SoC https://github.com/enjoy-digital/litex/wiki/Use-LiteScope-To...
  - moring 4y ago
    How does it solve the problem that the FPGA doesn't have a path from the signal-to-observe to the logic blocks?
    - dayjaby 4y ago
      You can specify the list of signals to observe, which is built into the gateware. As memory is also limited, you specify a condition like if signal A becomes 0x1234, then log the aforementioned signals for N cycles.
      - moring 4y ago
        I have to admit I still don't get it. When I try to log the signal using my own signal logger (which I guess is somewhat like that "virtual oscilloscope"), the synthesis tool rejects the design and tells me that I cannot use that signal as an input because the FPGA doesn't have a path from there into the logic blocks. What does LiteX do differently to observe a signal that my own gateware cannot access?
        proto_lambda 4y ago
        What kind of primitives are you talking about here? In my experience you don't really have to manually instantiate anything other than a PLL or maybe a multiplier unless you're doing something very specialized.
        moring 4y ago
        I was specifically thinking about DDR3 related primitives.
    - simne 4y ago
      Only simulator could check signal to which could not connect physically. But, you could split project into small parts, connected by some interface (could use triggers or 2-way ram cells), and debug them separate. Sure, you will loss some capacity, but this is life. This is not equal to test real solution, because physical arrangement matter, but most probably, you will find most obvious mistakes.
- CorrectHorseBat 4y ago
  >1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use. Does the problem not arise in simulation?
  - moring 4y ago
    I don't have simulation models for the primitives, and I can't write them due to the lack of documentation. One of the downsides of working purely with the free / open-source toolchain, I guess. edit for clarification: The lack of simulation models is due to the free toolchain. The documentation is fully available, it's just so bad that you don't even know what some of the primitives are meant to do.
- krapht 4y ago
  You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer. Are you a hobbyist? Ha, have fun. Digging through the Xilinx forums is also productive. Edit: I see you're strictly working with only free/open source tooling. I commend your patience. I also don't see the point. It's not the right choice for either learning or getting things done at the current moment.
  - misnome 4y ago
    > You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer. Basically this. I've been using the Intel OneAPI tools to write for their FPGA's in SYCL, and seem to have run into a large number of fundamental bugs (including plain routing errors) that makes me wonder how many other people are actually using the tools.
    - krapht 4y ago
      Well, now you know the answer is almost nobody :). Xilinx and Altera have been promising efficient high level C++/openCL/CUDA -> FPGA compilation for decades now and almost everybody I know has been burnt by it at work, so thanks for confirming my bias, I'll avoid OneAPI too for the present. Guess VHDL and Verilog and the old standard tooling will live for another 10 years. Everything is always amazing for the first 80% but the last 20% is impossible to fix and you end up not saving any dev time at all.
      - misnome 4y ago
        > I'll avoid OneAPI too for the present. I wouldn't write it off _completely_, for everyone. Compiler bugs notwithstanding, it seems to work reasonably well if you don't mind extremely long compiler cycles for running on the hardware. It gives you pretty reasonable emulation tooling to work out how well it runs, whether things are pipelined and, efficient if you know what you are looking at - but it doesn't seem to save you actually having to learn how the FPGAs work and model data flow. Of course, you are also beholden on drivers/adaptors being written to actually be able to use them.... which at this point is two chipsets, the newer of which has just been deprecated. Evaluating Xilinx HLS is next on my list, at least it seems to have the benefit of being around for much longer, and we have collaborators who have had success with the tooling.
eachro 4y ago
How does one actually program on an FPGA? I know for GPU, you can buy an nvidia graphics cardm, rent an aws gpu machine or use google colab. Is there some consumer facing equivalent for FPGA programming?
- nicolas_t 4y ago
  There are some development boards like the Terasic DE10 Nano. It's sometimes a bit hard to come by because it's used by Misterfpga (a retrogaming fpga based simulator) but it's perfect to learn to develop on. Other than that I really recommend 8bitworkshop and the Designing Video Game Hardware in Verilog. It's a great way to get started without an actual fpga board to test with.
- dailykoder 4y ago
  You could just download the Vendor tools (Xilinx, Intel, Lattice) and start with simulations. They are usually free of charge for private use. Often including their smaller chips. For Xilinx you could, for example, get the Digilent Arty-A7 board, which is based on Xilinx' Artix-7 chip and has plenty of space. There are also free tools for Simulation like GHDL+Gtkwave, but i am completely unsure if there is any that lets you do HLS FPGA hardware is expensive though
- Teknoman117 4y ago
  There are some FPGAs that have been _fully_ reverse engineered and now have open source toolchains, such as Lattice iCE40 and Lattice ECP5(-5G). Others are coming along, such as Xilinx 7 series and Lattice's Nexus FPGAs (CrossLink and Certus products). See: https://f4pga.org/ https://f4pga.org/ The open source tools operate at an RTL level (register transfer language, such as SystemVerilog) though, I'm not aware of any open source HLS tools. Problem at the moment is that FPGAs are being hit really hard by the current global supply chain problems. They're almost impossible to get unless you're a billion dollar corp and the prices are going through the roof for the available supply. As far as boards go, there are many, many out there. Some of my favorites are: Alchitry Au: https://www.sparkfun.com/products/16527 https://www.sparkfun.com/products/16527 Olimex iCE40HX8K-EVN: https://www.olimex.com/Products/FPGA/iCE40/iCE40HX8K-EVB/open-source-hardware https://www.olimex.com/Products/FPGA/iCE40/iCE40HX8K-EVB/ope... ECP5-5G evaluation kit: https://www.latticesemi.com/products/developmentboardsandkits/ecp5evaluationboard https://www.latticesemi.com/products/developmentboardsandkit...
deleted 4y ago
[deleted]
stefanpie 4y ago
When joining my current research lab, this book was a god send. For context this book is directed at users who are interested in high level synthesis which at the simplest level let's you write C++ code to convert to hardware constructs in Verilog or VHDL. HLS is really powerful and have allowed us to develop complex hardware like deep learning accelerators much faster but at the cost of poor tooling and many hours of debugging broken tools. Documentation is "good enough" with non existent support beyond your lab mates and your experience. I hope that the eventually support from industry tools will get better, open source work on this area will be developed more, or labs are more willing to share their knowledge and "secrets" with other on how to get the most out of these tools.

80 points by bjourne 4y ago

Slightly off-topic: I see lots of learning resources for FPGAs appearing that explain how to "think parallel", but are there good resources that explain how to deal with practical problems such as badly-documented primitives (looking at you, ECP5) or how to debug a non-working design (1), or how to find what causes the synthesis tool to segfault? 1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use. The situation somewhat reminds me of the earlier LaTeX days when every tutorial started with "I'm assuming you have already installed LaTeX" as if that wasn't the biggest problem for most users, and then continues explaining the easy stuff that you can easily learn by trial and error.

dayjaby 4y ago
> how to debug a non-working design In LiteX, an open source toolset for making FPGA designs, you can embed an oscilloscope into your FPGA: https://github.com/enjoy-digital/litex/wiki/Use-LiteScope-To-Debug-A-SoC https://github.com/enjoy-digital/litex/wiki/Use-LiteScope-To...
- moring 4y ago
  How does it solve the problem that the FPGA doesn't have a path from the signal-to-observe to the logic blocks?
  - dayjaby 4y ago
    You can specify the list of signals to observe, which is built into the gateware. As memory is also limited, you specify a condition like if signal A becomes 0x1234, then log the aforementioned signals for N cycles.
    - moring 4y ago
      I have to admit I still don't get it. When I try to log the signal using my own signal logger (which I guess is somewhat like that "virtual oscilloscope"), the synthesis tool rejects the design and tells me that I cannot use that signal as an input because the FPGA doesn't have a path from there into the logic blocks. What does LiteX do differently to observe a signal that my own gateware cannot access?
      - proto_lambda 4y ago
        What kind of primitives are you talking about here? In my experience you don't really have to manually instantiate anything other than a PLL or maybe a multiplier unless you're doing something very specialized.
        moring 4y ago
        I was specifically thinking about DDR3 related primitives.
  - simne 4y ago
    Only simulator could check signal to which could not connect physically. But, you could split project into small parts, connected by some interface (could use triggers or 2-way ram cells), and debug them separate. Sure, you will loss some capacity, but this is life. This is not equal to test real solution, because physical arrangement matter, but most probably, you will find most obvious mistakes.
CorrectHorseBat 4y ago
>1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use. Does the problem not arise in simulation?
- moring 4y ago
  I don't have simulation models for the primitives, and I can't write them due to the lack of documentation. One of the downsides of working purely with the free / open-source toolchain, I guess. edit for clarification: The lack of simulation models is due to the free toolchain. The documentation is fully available, it's just so bad that you don't even know what some of the primitives are meant to do.
krapht 4y ago
You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer. Are you a hobbyist? Ha, have fun. Digging through the Xilinx forums is also productive. Edit: I see you're strictly working with only free/open source tooling. I commend your patience. I also don't see the point. It's not the right choice for either learning or getting things done at the current moment.
- misnome 4y ago
  > You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer. Basically this. I've been using the Intel OneAPI tools to write for their FPGA's in SYCL, and seem to have run into a large number of fundamental bugs (including plain routing errors) that makes me wonder how many other people are actually using the tools.
  - krapht 4y ago
    Well, now you know the answer is almost nobody :). Xilinx and Altera have been promising efficient high level C++/openCL/CUDA -> FPGA compilation for decades now and almost everybody I know has been burnt by it at work, so thanks for confirming my bias, I'll avoid OneAPI too for the present. Guess VHDL and Verilog and the old standard tooling will live for another 10 years. Everything is always amazing for the first 80% but the last 20% is impossible to fix and you end up not saving any dev time at all.
    - misnome 4y ago
      > I'll avoid OneAPI too for the present. I wouldn't write it off _completely_, for everyone. Compiler bugs notwithstanding, it seems to work reasonably well if you don't mind extremely long compiler cycles for running on the hardware. It gives you pretty reasonable emulation tooling to work out how well it runs, whether things are pipelined and, efficient if you know what you are looking at - but it doesn't seem to save you actually having to learn how the FPGAs work and model data flow. Of course, you are also beholden on drivers/adaptors being written to actually be able to use them.... which at this point is two chipsets, the newer of which has just been deprecated. Evaluating Xilinx HLS is next on my list, at least it seems to have the benefit of being around for much longer, and we have collaborators who have had success with the tooling.

eachro 4y ago

How does one actually program on an FPGA? I know for GPU, you can buy an nvidia graphics cardm, rent an aws gpu machine or use google colab. Is there some consumer facing equivalent for FPGA programming?

nicolas_t 4y ago
There are some development boards like the Terasic DE10 Nano. It's sometimes a bit hard to come by because it's used by Misterfpga (a retrogaming fpga based simulator) but it's perfect to learn to develop on. Other than that I really recommend 8bitworkshop and the Designing Video Game Hardware in Verilog. It's a great way to get started without an actual fpga board to test with.
dailykoder 4y ago
You could just download the Vendor tools (Xilinx, Intel, Lattice) and start with simulations. They are usually free of charge for private use. Often including their smaller chips. For Xilinx you could, for example, get the Digilent Arty-A7 board, which is based on Xilinx' Artix-7 chip and has plenty of space. There are also free tools for Simulation like GHDL+Gtkwave, but i am completely unsure if there is any that lets you do HLS FPGA hardware is expensive though
Teknoman117 4y ago
There are some FPGAs that have been _fully_ reverse engineered and now have open source toolchains, such as Lattice iCE40 and Lattice ECP5(-5G). Others are coming along, such as Xilinx 7 series and Lattice's Nexus FPGAs (CrossLink and Certus products). See: https://f4pga.org/ https://f4pga.org/ The open source tools operate at an RTL level (register transfer language, such as SystemVerilog) though, I'm not aware of any open source HLS tools. Problem at the moment is that FPGAs are being hit really hard by the current global supply chain problems. They're almost impossible to get unless you're a billion dollar corp and the prices are going through the roof for the available supply. As far as boards go, there are many, many out there. Some of my favorites are: Alchitry Au: https://www.sparkfun.com/products/16527 https://www.sparkfun.com/products/16527 Olimex iCE40HX8K-EVN: https://www.olimex.com/Products/FPGA/iCE40/iCE40HX8K-EVB/open-source-hardware https://www.olimex.com/Products/FPGA/iCE40/iCE40HX8K-EVB/ope... ECP5-5G evaluation kit: https://www.latticesemi.com/products/developmentboardsandkits/ecp5evaluationboard https://www.latticesemi.com/products/developmentboardsandkit...

deleted 4y ago

[deleted]

stefanpie 4y ago

When joining my current research lab, this book was a god send. For context this book is directed at users who are interested in high level synthesis which at the simplest level let's you write C++ code to convert to hardware constructs in Verilog or VHDL. HLS is really powerful and have allowed us to develop complex hardware like deep learning accelerators much faster but at the cost of poor tooling and many hours of debugging broken tools. Documentation is "good enough" with non existent support beyond your lab mates and your experience. I hope that the eventually support from industry tools will get better, open source work on this area will be developed more, or labs are more willing to share their knowledge and "secrets" with other on how to get the most out of these tools.