5 ms·

It seems that operations on FPGAs can run much more efficiently than their cpu equivalent. For an 'AND' operation, a cpu needs to load code and data from a memo

It seems that operations on FPGAs can run much more efficiently than their cpu equivalent. For an 'AND' operation, a cpu needs to load code and data from a memory into registers, run the logic and write the result register back to some memory. This while filling up the pipeline for subsequent operations. The FPGA on the other hand has the output ready one clock cycle after the inputs stream in, and can have many such operations in parallel. One might ask, why are cpus not being replaced by FPGAs? Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? I could ask GPT those questions, but the HN community will provide more insight I guess.

JackSlateur 4y ago
For the same reasons we do not replace CPUs with GPUs: not the right tool Check out the instruction set of modern CPUs
burnished 4y ago
They both use the same kind of components; the FPGA does not have a speed advantage, you are simply comparing the speed of a very simple circuit element to the speed of a very complicated pipeline. You would use an FPGA to simulate a special purpose circuit, which would be faster than a CPU for its specific purpose. We have CPUs because having a general purpose processing chip is incredibly handy when you want to be able to do more than one thing. EDIT: I forgot to mention that the device outputs in one clock cycle by definition: if your clock is too fast then your components output signals dont have time to stabilize and you will get read errors, so you ensure your clock is slow enough for everything to stabilize.
convolvatron 4y ago
one big problem is memory. basic cpus have alot of facilities for high-speed synchronous interface with DRAM, and truly vast amount of resource for cache. partially as a result, a good model for compiling code to fpgas uses a dataflow paradigm, since we don't need to serialize all operations through a memory fetch, cache, or even register file. if we hadn't decided to move all our computing to the cloud, I suspect fpga accelerator boards for applications which map well to that model would have some traction in specialized areas. signal processing is definitely one such.
quadrature 4y ago
>One might ask, why are cpus not being replaced by FPGAs? they do sometimes !, for very specific applications. The problem is that an FPGA is programmed for one specific task and would have to be taken offline and reprogrammed if you wanted to do something else with it. Its not general purpose like a CPU where you can load up any program and have it run. Programming an FPGA is also comparatively much harder to reason about than a CPU because of the parallelism and timing you described.
- MSFT_Edging 4y ago
  Some of the more modern Xilinx stuff has features where you don't need to take down the whole FPGA to reload a bitstream onto part of the chip. Its really neat, you can do live reprogramming of one component and leave the others alone or have an A/B setup where one updates while the other is unchanged.
  - JohnFen 4y ago
    Yes, I'm working on a Xilinx ARM processor with an FPGA. The FPGA and the CPU are independent units in the chip that can each operate with or without the other. We can indeed reprogram the FPGA without taking the system down.
    - davemp 4y ago
      It goes even further. You can partially reconfigure the FPGA fabric itself: https://support.xilinx.com/s/article/34924?language=en_US https://support.xilinx.com/s/article/34924?language=en_US
      - MSFT_Edging 3y ago
        That's what I was referring to. Neat stuff.
  - quadrature 4y ago
    That is really cool, hadn't heard of that before.
barelyauser 4y ago
What is simpler: making logical circuit "A" or making a circuit that emulates logical circuit "A" and its relatives?
Lramseyer 4y ago
These are really good questions to be asking, and to help with that let's consider 3 attributes of compute complexity: time, space, and memory The traditional way of computing on a CPU is in essence a list of instructions to be computed. These instructions all go to the same place (the CPU core) to be computed. Since the space is constant, the instructions are computed sequentially in time. Most programmers aren't concerned with redesigning a CPU, so we typically only think about computing in time (and memory of course) On an FPGA (and custom silicon) the speedup comes from being able to compute in both time and space. Instead of your instructions existing in memory, and computed in time, they can be represented in separate logic elements (in space) and they can each do separate things in time. So in a way, you're trading space for time. This is how the speed gains are achieved. Where this all breaks down is the optimization and scheduling. A sequential task is relatively easy to optimize since you're optimizing in time (and memory to an extent.) Scheduling is easy too, since, they can be prioritized and queued up. However, when you're computing in space, you have to optimize in 2 spatial dimensions and in time. When you have multiple tasks that that need to be completed, you then need to place them together and not have them overlap. Think trying to fit a ton of different shaped tiles on a table, where you need to be constantly adding and removing tiles in a way that doesn't disrupt the placement of other tiles (at least not too often.) It's kind of a pain, but for some more constrained problem sets, it might make sense. These aren't impossible problems, and for some tasks, the time or power usage savings is worth the additional complexity. But sequential optimization is way easier, and good enough for most tasks. However, if our desire for faster computing outpaces our ability to make faster CPUs, you may see more FPGAs doing this sort of thing. We already have FPGAs that are capable of partial reconfiguration, and some pretty good software tools to go along with it. TL;DR: Geometry is hard.
toast0 4y ago
> The FPGA on the other hand has the output ready one clock cycle after the inputs stream in, and can have many such operations in parallel. One might ask, why are cpus not being replaced by FPGAs? FPGAs are more or less a flexible replacement for an application specific (logic level) integrated circuit. A CPU can do a wide variety of tasks, with a small penalty for switching tasks. An ASIC can do one thing and that's it, a FPGA can do many things, but with a large penalty for task switching. (you can have a CPU as an ASIC or an FPGA, but...). ASICs require a lot of upfront design work and costs, so you can't use them for everything. ASICs and especially CPUs tend to be able to achieve a higher clock speed that FPGAs, but it kind of depends. > Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? Not really; the way problems are solved is drastically different, and I'd expect most things would need to be reconceptualized to fit. And a lot of software isn't really suited to living as a logic circuit. Exceptions would be encoding, compression, encryption, the inverses of all of those, signal processing, etc. Things where you have a data pipeline and 'the same thing' happens to all the data.
jcranmer 4y ago
FPGAs are the next big frontier for software development, and have been since the '90s, they just need the programming model worked out. This is the traditional story told about FPGAs, but GPGPU programming suddenly overtaking FPGA development about 2010 despite their awkward programming models makes that story rather suspect. The thing is, a lot of the benefits of FPGAs are really best-case scenarios, and when you move to more typical scenarios, their competitiveness as an architecture dwindles dramatically. Pipelining on an FPGA requires being able to find, and fill, spatial duplication of the operations being done. If you've got conditional operations in a pipeline, now your pipeline isn't so full anymore, and this hurts performance on an FPGA far more than on a CPU (which spends a lot of power trying to keep its pipelines full). But needing to keep the pipelines spatially connected also means you have to be able to find a physical connection between the two stages of a pipeline, and the physical length of that connection also imposes limitations on the frequency you can run the FPGA at. If you care about FLOPS (or throughput in general), the problem with FPGAs is that they are running at a clock speed about a tenth of a CPU. This requires a 10x improvement in performance just to stand still; given that software development for FPGAs requires essentially a completely different mindset than for CPUs or even GPUs, it's not common to have use cases that work well on FPGAs. (I should say that a lot of my information about programming FPGAs comes from ex-FPGA developers, and the "ex-" part will certainly have its own form of bias in these opinions).
- davemp 4y ago
  Yeah I don't really see FPGAs ever making their way down to consumers the way GPUs and CPUs have (end users actually programming them). For (semi) fixed pipeline operations FPGAs will basically always be worse than some slightly more specialized ASIC like a GPU/AI engine. One area FPGAs can be exceptionally good at is real-time operations. You have much better control over timing in the general on FPGAs vs MCU/CPUs, but I don't think that's inherent (you could probably alter the mcu architecture a bit and close the gap). I could be wrong but I also think you get better power draw for things like mid to low volume glue chips in embedded systems because you're not powering big SRAM banks and DMAs just to pipe data between a couple hardware interfaces. This is only because of market forces though obviously, because if mid to low volume ASICs become viable in terms of dev time they'll be much better.
pjc50 4y ago
> One might ask, why are cpus not being replaced by FPGAs? Most of the time you want data-dependent execution. FPGA systems excel at "fixed pipeline" systems, where you have e.g. an audio filter chain .. but even that is usually done in efficient DSP CPUs. > Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? A subset can. Things like recursion are right out. Various companies have tools to do this, but you usually end up having to rework either the source you're feeding them, or the HDL output.
pfyra 4y ago
> Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? Yes. At least for c and c++. It is called High Level Synthesis.
rcxdude 3y ago
The very lowest-level operations on FPGAs can run more efficiently than emulating them on an CPU. The higher level operations, which is usually what we want, are slower, as a 'gate' in an FPGA is approximately an order of magnitude slower, more power hungry, and expensive than a gate in a CPU (or GPU). modern CPUs excel at executing data-dependent operations of a wide variety while exploiting as much parallelism as available, while GPUs excel at churning through a huge amount of memory doing parallel arithmetic operations: for both of theses tasks an FPGA is going to be worse. For an FPGA to be the better option you need a task which is specialised enough that the speedup from designing a specialised circuit to do it overcomes the intrinsic slowdown from it being implemented in an FPGA as opposed to raw silicon, and these are not your typical software tasks, usually much more hardware-type tasks. The two areas where FPGAs excel are having extremely low and predictable latency, and having lots of extremely flexible high-bandwidth IO (still less bandwidth than the memory or PCI-E bandwidth of a GPU, but it's close and it's much more flexible). For computation throughput, however, there's only a few niche applications, though one important one is simulating the logic which is going to be implemented in a CPU, GPU, or ASIC to validate it before the expensive process of actually etching some silicon.
cpgxiii 3y ago
> The FPGA on the other hand has the output ready one clock cycle after the inputs stream in, and can have many such operations in parallel. One might ask, why are cpus not being replaced by FPGAs? You need to be careful with interpreting cycle counts as execution time. A naive operation on an FPGA may take a single cycle because the author has asked for that, rather than constructing and using a pipelined multi-cycle ALU [1]. Therefore the design tool will synthesize a layout that performs the operation in a single cycle, but the cycle time (and thus clock frequency for that portion of the layout) will be limited by however long that operation needs to complete, and will be much longer than the cycle time of the CPU you are comparing against. Note that a CPU almost certainly has "better" ALUs than those you'd get on an FPGA - optimized fixed-function circuits can always be faster than their generic programmable equivalents - the advantage of the FPGA is that you can have only (and more of) the operations you need, running in parallel as many times as the device will support. You could make a CPU by which every instruction (even very complicated vector instructions like AVX or SVE) was completed entirely in a single cycle, but that cycle itself would be very long, and all of the incredible throughput benefits of pipelined execution would be lost. [1] Pretty much all of the vendor design tools can instantiate an ALU for your device, given parameters like # of bits, operation types, # of cycles, etc.

JackSlateur 4y ago

For the same reasons we do not replace CPUs with GPUs: not the right tool Check out the instruction set of modern CPUs

burnished 4y ago

They both use the same kind of components; the FPGA does not have a speed advantage, you are simply comparing the speed of a very simple circuit element to the speed of a very complicated pipeline. You would use an FPGA to simulate a special purpose circuit, which would be faster than a CPU for its specific purpose. We have CPUs because having a general purpose processing chip is incredibly handy when you want to be able to do more than one thing. EDIT: I forgot to mention that the device outputs in one clock cycle by definition: if your clock is too fast then your components output signals dont have time to stabilize and you will get read errors, so you ensure your clock is slow enough for everything to stabilize.

convolvatron 4y ago

one big problem is memory. basic cpus have alot of facilities for high-speed synchronous interface with DRAM, and truly vast amount of resource for cache. partially as a result, a good model for compiling code to fpgas uses a dataflow paradigm, since we don't need to serialize all operations through a memory fetch, cache, or even register file. if we hadn't decided to move all our computing to the cloud, I suspect fpga accelerator boards for applications which map well to that model would have some traction in specialized areas. signal processing is definitely one such.

quadrature 4y ago

>One might ask, why are cpus not being replaced by FPGAs? they do sometimes !, for very specific applications. The problem is that an FPGA is programmed for one specific task and would have to be taken offline and reprogrammed if you wanted to do something else with it. Its not general purpose like a CPU where you can load up any program and have it run. Programming an FPGA is also comparatively much harder to reason about than a CPU because of the parallelism and timing you described.

MSFT_Edging 4y ago
Some of the more modern Xilinx stuff has features where you don't need to take down the whole FPGA to reload a bitstream onto part of the chip. Its really neat, you can do live reprogramming of one component and leave the others alone or have an A/B setup where one updates while the other is unchanged.
- JohnFen 4y ago
  Yes, I'm working on a Xilinx ARM processor with an FPGA. The FPGA and the CPU are independent units in the chip that can each operate with or without the other. We can indeed reprogram the FPGA without taking the system down.
  - davemp 4y ago
    It goes even further. You can partially reconfigure the FPGA fabric itself: https://support.xilinx.com/s/article/34924?language=en_US https://support.xilinx.com/s/article/34924?language=en_US
    - MSFT_Edging 3y ago
      That's what I was referring to. Neat stuff.
- quadrature 4y ago
  That is really cool, hadn't heard of that before.

barelyauser 4y ago

What is simpler: making logical circuit "A" or making a circuit that emulates logical circuit "A" and its relatives?

Lramseyer 4y ago

These are really good questions to be asking, and to help with that let's consider 3 attributes of compute complexity: time, space, and memory The traditional way of computing on a CPU is in essence a list of instructions to be computed. These instructions all go to the same place (the CPU core) to be computed. Since the space is constant, the instructions are computed sequentially in time. Most programmers aren't concerned with redesigning a CPU, so we typically only think about computing in time (and memory of course) On an FPGA (and custom silicon) the speedup comes from being able to compute in both time and space. Instead of your instructions existing in memory, and computed in time, they can be represented in separate logic elements (in space) and they can each do separate things in time. So in a way, you're trading space for time. This is how the speed gains are achieved. Where this all breaks down is the optimization and scheduling. A sequential task is relatively easy to optimize since you're optimizing in time (and memory to an extent.) Scheduling is easy too, since, they can be prioritized and queued up. However, when you're computing in space, you have to optimize in 2 spatial dimensions and in time. When you have multiple tasks that that need to be completed, you then need to place them together and not have them overlap. Think trying to fit a ton of different shaped tiles on a table, where you need to be constantly adding and removing tiles in a way that doesn't disrupt the placement of other tiles (at least not too often.) It's kind of a pain, but for some more constrained problem sets, it might make sense. These aren't impossible problems, and for some tasks, the time or power usage savings is worth the additional complexity. But sequential optimization is way easier, and good enough for most tasks. However, if our desire for faster computing outpaces our ability to make faster CPUs, you may see more FPGAs doing this sort of thing. We already have FPGAs that are capable of partial reconfiguration, and some pretty good software tools to go along with it. TL;DR: Geometry is hard.

toast0 4y ago

> The FPGA on the other hand has the output ready one clock cycle after the inputs stream in, and can have many such operations in parallel. One might ask, why are cpus not being replaced by FPGAs? FPGAs are more or less a flexible replacement for an application specific (logic level) integrated circuit. A CPU can do a wide variety of tasks, with a small penalty for switching tasks. An ASIC can do one thing and that's it, a FPGA can do many things, but with a large penalty for task switching. (you can have a CPU as an ASIC or an FPGA, but...). ASICs require a lot of upfront design work and costs, so you can't use them for everything. ASICs and especially CPUs tend to be able to achieve a higher clock speed that FPGAs, but it kind of depends. > Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? Not really; the way problems are solved is drastically different, and I'd expect most things would need to be reconceptualized to fit. And a lot of software isn't really suited to living as a logic circuit. Exceptions would be encoding, compression, encryption, the inverses of all of those, signal processing, etc. Things where you have a data pipeline and 'the same thing' happens to all the data.

jcranmer 4y ago

FPGAs are the next big frontier for software development, and have been since the '90s, they just need the programming model worked out. This is the traditional story told about FPGAs, but GPGPU programming suddenly overtaking FPGA development about 2010 despite their awkward programming models makes that story rather suspect. The thing is, a lot of the benefits of FPGAs are really best-case scenarios, and when you move to more typical scenarios, their competitiveness as an architecture dwindles dramatically. Pipelining on an FPGA requires being able to find, and fill, spatial duplication of the operations being done. If you've got conditional operations in a pipeline, now your pipeline isn't so full anymore, and this hurts performance on an FPGA far more than on a CPU (which spends a lot of power trying to keep its pipelines full). But needing to keep the pipelines spatially connected also means you have to be able to find a physical connection between the two stages of a pipeline, and the physical length of that connection also imposes limitations on the frequency you can run the FPGA at. If you care about FLOPS (or throughput in general), the problem with FPGAs is that they are running at a clock speed about a tenth of a CPU. This requires a 10x improvement in performance just to stand still; given that software development for FPGAs requires essentially a completely different mindset than for CPUs or even GPUs, it's not common to have use cases that work well on FPGAs. (I should say that a lot of my information about programming FPGAs comes from ex-FPGA developers, and the "ex-" part will certainly have its own form of bias in these opinions).

davemp 4y ago
Yeah I don't really see FPGAs ever making their way down to consumers the way GPUs and CPUs have (end users actually programming them). For (semi) fixed pipeline operations FPGAs will basically always be worse than some slightly more specialized ASIC like a GPU/AI engine. One area FPGAs can be exceptionally good at is real-time operations. You have much better control over timing in the general on FPGAs vs MCU/CPUs, but I don't think that's inherent (you could probably alter the mcu architecture a bit and close the gap). I could be wrong but I also think you get better power draw for things like mid to low volume glue chips in embedded systems because you're not powering big SRAM banks and DMAs just to pipe data between a couple hardware interfaces. This is only because of market forces though obviously, because if mid to low volume ASICs become viable in terms of dev time they'll be much better.

pjc50 4y ago

> One might ask, why are cpus not being replaced by FPGAs? Most of the time you want data-dependent execution. FPGA systems excel at "fixed pipeline" systems, where you have e.g. an audio filter chain .. but even that is usually done in efficient DSP CPUs. > Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? A subset can. Things like recursion are right out. Various companies have tools to do this, but you usually end up having to rework either the source you're feeding them, or the HDL output.

pfyra 4y ago

> Another interesting question, can software (recipes for cpus) be transpiled to be efficiently run on FPGAs? Yes. At least for c and c++. It is called High Level Synthesis.

rcxdude 3y ago

The very lowest-level operations on FPGAs can run more efficiently than emulating them on an CPU. The higher level operations, which is usually what we want, are slower, as a 'gate' in an FPGA is approximately an order of magnitude slower, more power hungry, and expensive than a gate in a CPU (or GPU). modern CPUs excel at executing data-dependent operations of a wide variety while exploiting as much parallelism as available, while GPUs excel at churning through a huge amount of memory doing parallel arithmetic operations: for both of theses tasks an FPGA is going to be worse. For an FPGA to be the better option you need a task which is specialised enough that the speedup from designing a specialised circuit to do it overcomes the intrinsic slowdown from it being implemented in an FPGA as opposed to raw silicon, and these are not your typical software tasks, usually much more hardware-type tasks. The two areas where FPGAs excel are having extremely low and predictable latency, and having lots of extremely flexible high-bandwidth IO (still less bandwidth than the memory or PCI-E bandwidth of a GPU, but it's close and it's much more flexible). For computation throughput, however, there's only a few niche applications, though one important one is simulating the logic which is going to be implemented in a CPU, GPU, or ASIC to validate it before the expensive process of actually etching some silicon.

cpgxiii 3y ago

> The FPGA on the other hand has the output ready one clock cycle after the inputs stream in, and can have many such operations in parallel. One might ask, why are cpus not being replaced by FPGAs? You need to be careful with interpreting cycle counts as execution time. A naive operation on an FPGA may take a single cycle because the author has asked for that, rather than constructing and using a pipelined multi-cycle ALU [1]. Therefore the design tool will synthesize a layout that performs the operation in a single cycle, but the cycle time (and thus clock frequency for that portion of the layout) will be limited by however long that operation needs to complete, and will be much longer than the cycle time of the CPU you are comparing against. Note that a CPU almost certainly has "better" ALUs than those you'd get on an FPGA - optimized fixed-function circuits can always be faster than their generic programmable equivalents - the advantage of the FPGA is that you can have only (and more of) the operations you need, running in parallel as many times as the device will support. You could make a CPU by which every instruction (even very complicated vector instructions like AVX or SVE) was completed entirely in a single cycle, but that cycle itself would be very long, and all of the incredible throughput benefits of pipelined execution would be lost. [1] Pretty much all of the vendor design tools can instantiate an ALU for your device, given parameters like # of bits, operation types, # of cycles, etc.