8 ms·

One clarification on the comment about latency. FPGAs are typically clocked much slower than a modern CPU. Typically, they run somewhere in the low 100's of MHz

One clarification on the comment about latency. FPGAs are typically clocked much slower than a modern CPU. Typically, they run somewhere in the low 100's of MHz, whereas an Intel CPU clocks in at around 3GHz last time I went to the Apple Store. With a typical x86 multiply instruction having a latency of about, say, 3 cycles, putting that workload on an FPGA would result in a ~10x slow-down! The real benefit of an FPGA is that you get to decide what happens in any given cycle. So rather than being able to multiply two numbers in a single cycle like on x86, you could make your FPGA design do, say, 20 multiplications in a single cycle (space allowing). Which means that you can now multiply 20 numbers in 1/10th of the time it would take on x86. (In reality I think you have something like four execution units capable of perfoming parallel ALU operations per cycle, depending on the family.) So the latency benefit of an FPGA comes from flexible, almost (almost) unbounded potential for parallelism in a given cycle, not clock frequency. Hardware has to be designed to exploit this potential, otherwise it's not going to see any improvement in latency. Anyways, just something that is maybe obvious once you're told it, but isn't always mentioned in discussions like this. It certainly something that I didn't fully appreciate before getting involved in hardware design.

pensono 8y ago
FPGA latency is also much more predictable. Since you have cycle level control, you can know exactly how long any given operation will take. In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt. It's also worth noting that FPGAs are often much closer to the I/O than a CPU on a motherboard. Pin to pin latencies are much reduced in a situation where direct access to I/O is available. An Ethernet stack can sit on the same silicon without having to serialize data between the CPU and the north bridge.
- 0xffff2 8y ago
  >In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt. If you're considering an FPGA, isn't the alternative likely bare-metal CPU programming? In that case, you have just as much control over thread scheduling as you do over FPGA timing don't you?
  - namibj 8y ago
    Not since Ring -1 came to existence.
  - stefan_ 8y ago
    Throughput and latency are opposite ends of the same tradeoff. FPGAs enable cycle perfect timing control, while for bare-metal CPU programming.. even if you just have an infinite loop running bare-metal on one CPU, looking at the assembly can't tell you anything about the timing. Modern CPUs have multiple layers of caches with penalties for a miss coming in at hundreds or thousands of cycles. They do parallel and speculative execution of instructions. Instructions cost variable amount of cycles and take variably long depending on instructions before and after. And then, with the SoCs that are everywhere now.. you are poking some memory mapped register, say you toggle a GPIO output value.. how long does it take to change on the actual physical pin? Is the GPIO peripheral part of the processor or some IP they bought in and then interconnected over an AXI bus? Is it buffered? It's all entirely impossible to say.
    - jrockway 8y ago
      I feel like people are taking the hybrid approach of having a dedicated microcontroller on the same die as their CPU, like the PRU on TI's ARM chips: http://processors.wiki.ti.com/index.php/PRU-ICSS http://processors.wiki.ti.com/index.php/PRU-ICSS This by no means replaces an FPGA, but if you just want your pins to flip and code to run at a predictable interval, this gets you that. (As for reading pins, they have another peripheral on the die that will just timestamp when an event occurs. This is much more consistent than dealing with your OS's interrupt handler, but again, it's no FPGA.)
      - stefan_ 8y ago
        Yeah, the TI PRUs are what you get approaching this topic from a processor manufacturer standpoint, the XILINX Zynq is how a FPGA manufacturers would do it.
        snaky 8y ago
fpgaminer 8y ago
Another thing is that you can pipeline, for example, multiplies. So in a CPU multiply you give the CPU inputs, wait a couple cycles, and then get the result. In an FPGA you can build a pipelined multiply. It's built such that you can feed it input every cycle and get an output every cycle. The only caveat is that the outputs are delayed relative to the inputs. i.e. you may give it (2, 3) to multiply on one cycle, but you won't see the result (6) of that particular input on the output until a couple cycles later. [Yup, "pipeline" here is the same term used to describe how modern CPUs get their performance. They, too, are pipelining their instruction execution so that many instructions can be in the process of executing at the same time. Though what I describe is a more extreme and specific kind of pipelining.] This is kind of like having a bunch of multiplies in parallel. But it's useful for, for example, real-time calculations. You get to perform all the calculations you need every single cycle, no matter how complex; your results are just delayed by X cycles. Pipelines are usually also more efficient than straight parallelism (i.e. an 8 deep pipelined multiplier uses less silicon than 8 individual "serial" multiplier units). Another interesting thing: In a previous life I built an FPGA based video processing device. It sat in an HDMI chain, so it had to be real-time. If that had been built with a GPU, most engineers would build the system to buffer up a frame, perform the processing, and then feed the processed frame out. That results in at least 1 frame of delay. In contrast, because we used an FPGA, it was simple to just pipeline the entire design and thus only needed to buffer up the few lines that we needed. This meant A) we needed no external memory (cheaper) and our latency was on the order of microseconds. In my travels with that job I ran into tons of other companies building video processing devices. They _all_ used frame buffers, which made their devices unacceptable for, e.g., gaming.
- ignoramous 8y ago
  Thanks. Just to clarify whilst I have your attention: isn't it a common practice to pipeline frame buffers too? For instance, Android has three frame buffers. IIRC, one is with the user space handing off display lists, one used for composition, and one is used by driver for rasterization? In that case, how good, you reckon, would the perf be compared to FGPA?
  - 8y ago
chrisBob 8y ago
For large calculations, the magic of an FPGA is in its throughput. Imagine that you have some mess of addition, multiplication, ... The time for the first calculation hardly matters. Even if the FPGA is slower getting through the first calculation that took 100 clock cycles on a CPU vs 10 on an FPGA, what happens on the next clock cycle? The FPGA cranked through an entire second iteration while the CPU is a few steps in. Next clock cycle? Now the FPGA has pushed another whole calculation through the pipeline.
- joshvm 8y ago
  My favourite personal example of this is using a GPU to perform an all-pairs nearest neighbour lookup in an image (for all pixels, find the nearest keypoint). That's something like 2 billion comparisons per image. A decent CPU (parallelised) took minutes to do that, at the time. By far the simplest solution was to brute force it on a GPU. Probably took longer "single threaded", as there was no optimisation at all, but over O(10e6) pixels with a list of O(10e3) keypoints in shared memory it was basically instant. It was a great lesson in premature optimisation. I could have spent days tweaking the CPU method with heuristics, sorting the inputs, etc. In the end it was less than 100 lines of OpenCL I think.
deleted 8y ago
[deleted]
vardump 8y ago
> One clarification on the comment about latency. FPGAs are typically clocked much slower than a modern CPU. Typically, they run somewhere in the low 100's of MHz, whereas an Intel CPU clocks in at around 3GHz last time I went to the Apple Store. With a typical x86 multiply instruction having a latency of about, say, 3 cycles, putting that workload on an FPGA would result in a ~10x slow-down! No, not that latency. Latency from external input to external output. Like the latency of receiving an ethernet frame to the point you can decide what to do with it, forward, modify, etc. Or reacting to sensor input. CPUs process interrupts slowly: X86 CPUs running a non-realtime OS can take sweet 50 microseconds+ just to get to run interrupt service routine let alone doing something useful with the data. Oh, and generally no FPU, SIMD, etc. is available in this ISR [0]. So ISR generally queues the event (on Linux a Bottom Half and on Windows a DPC, deferred procedure call) to actually process the event. The queued events might execute tens of milliseconds later in bad cases. Microcontrollers can sometimes process external interrupts pretty quickly, it's not impossible to be done in 100 nanoseconds. But it really depends, and you usually have to be pretty careful to achieve this. And even then you often have significant jitter in the processing time. FPGAs can start to handle an external I/O event on the same clock cycle, and might very well be driving I/O output pin a clock cycle later, if no pipelining is required. So an FPGA might be done in mere 10 nanoseconds in some ideal cases. With little to no jitter. [0]: Well, you can do FPU/SIMD in interrupt service routine, but need to handle all register state storage on your own, and if you mess up, it'll cause very interesting bugs in the userland application software.

pensono 8y ago

FPGA latency is also much more predictable. Since you have cycle level control, you can know exactly how long any given operation will take. In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt. It's also worth noting that FPGAs are often much closer to the I/O than a CPU on a motherboard. Pin to pin latencies are much reduced in a situation where direct access to I/O is available. An Ethernet stack can sit on the same silicon without having to serialize data between the CPU and the north bridge.

0xffff2 8y ago
>In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt. If you're considering an FPGA, isn't the alternative likely bare-metal CPU programming? In that case, you have just as much control over thread scheduling as you do over FPGA timing don't you?
- namibj 8y ago
  Not since Ring -1 came to existence.
- stefan_ 8y ago
  Throughput and latency are opposite ends of the same tradeoff. FPGAs enable cycle perfect timing control, while for bare-metal CPU programming.. even if you just have an infinite loop running bare-metal on one CPU, looking at the assembly can't tell you anything about the timing. Modern CPUs have multiple layers of caches with penalties for a miss coming in at hundreds or thousands of cycles. They do parallel and speculative execution of instructions. Instructions cost variable amount of cycles and take variably long depending on instructions before and after. And then, with the SoCs that are everywhere now.. you are poking some memory mapped register, say you toggle a GPIO output value.. how long does it take to change on the actual physical pin? Is the GPIO peripheral part of the processor or some IP they bought in and then interconnected over an AXI bus? Is it buffered? It's all entirely impossible to say.
  - jrockway 8y ago
    I feel like people are taking the hybrid approach of having a dedicated microcontroller on the same die as their CPU, like the PRU on TI's ARM chips: http://processors.wiki.ti.com/index.php/PRU-ICSS http://processors.wiki.ti.com/index.php/PRU-ICSS This by no means replaces an FPGA, but if you just want your pins to flip and code to run at a predictable interval, this gets you that. (As for reading pins, they have another peripheral on the die that will just timestamp when an event occurs. This is much more consistent than dealing with your OS's interrupt handler, but again, it's no FPGA.)
    - stefan_ 8y ago
      Yeah, the TI PRUs are what you get approaching this topic from a processor manufacturer standpoint, the XILINX Zynq is how a FPGA manufacturers would do it.
      - snaky 8y ago

fpgaminer 8y ago

Another thing is that you can pipeline, for example, multiplies. So in a CPU multiply you give the CPU inputs, wait a couple cycles, and then get the result. In an FPGA you can build a pipelined multiply. It's built such that you can feed it input every cycle and get an output every cycle. The only caveat is that the outputs are delayed relative to the inputs. i.e. you may give it (2, 3) to multiply on one cycle, but you won't see the result (6) of that particular input on the output until a couple cycles later. [Yup, "pipeline" here is the same term used to describe how modern CPUs get their performance. They, too, are pipelining their instruction execution so that many instructions can be in the process of executing at the same time. Though what I describe is a more extreme and specific kind of pipelining.] This is kind of like having a bunch of multiplies in parallel. But it's useful for, for example, real-time calculations. You get to perform all the calculations you need every single cycle, no matter how complex; your results are just delayed by X cycles. Pipelines are usually also more efficient than straight parallelism (i.e. an 8 deep pipelined multiplier uses less silicon than 8 individual "serial" multiplier units). Another interesting thing: In a previous life I built an FPGA based video processing device. It sat in an HDMI chain, so it had to be real-time. If that had been built with a GPU, most engineers would build the system to buffer up a frame, perform the processing, and then feed the processed frame out. That results in at least 1 frame of delay. In contrast, because we used an FPGA, it was simple to just pipeline the entire design and thus only needed to buffer up the few lines that we needed. This meant A) we needed no external memory (cheaper) and our latency was on the order of microseconds. In my travels with that job I ran into tons of other companies building video processing devices. They _all_ used frame buffers, which made their devices unacceptable for, e.g., gaming.

ignoramous 8y ago
Thanks. Just to clarify whilst I have your attention: isn't it a common practice to pipeline frame buffers too? For instance, Android has three frame buffers. IIRC, one is with the user space handing off display lists, one used for composition, and one is used by driver for rasterization? In that case, how good, you reckon, would the perf be compared to FGPA?
- 8y ago

chrisBob 8y ago

For large calculations, the magic of an FPGA is in its throughput. Imagine that you have some mess of addition, multiplication, ... The time for the first calculation hardly matters. Even if the FPGA is slower getting through the first calculation that took 100 clock cycles on a CPU vs 10 on an FPGA, what happens on the next clock cycle? The FPGA cranked through an entire second iteration while the CPU is a few steps in. Next clock cycle? Now the FPGA has pushed another whole calculation through the pipeline.

joshvm 8y ago
My favourite personal example of this is using a GPU to perform an all-pairs nearest neighbour lookup in an image (for all pixels, find the nearest keypoint). That's something like 2 billion comparisons per image. A decent CPU (parallelised) took minutes to do that, at the time. By far the simplest solution was to brute force it on a GPU. Probably took longer "single threaded", as there was no optimisation at all, but over O(10e6) pixels with a list of O(10e3) keypoints in shared memory it was basically instant. It was a great lesson in premature optimisation. I could have spent days tweaking the CPU method with heuristics, sorting the inputs, etc. In the end it was less than 100 lines of OpenCL I think.

deleted 8y ago

[deleted]

vardump 8y ago

> One clarification on the comment about latency. FPGAs are typically clocked much slower than a modern CPU. Typically, they run somewhere in the low 100's of MHz, whereas an Intel CPU clocks in at around 3GHz last time I went to the Apple Store. With a typical x86 multiply instruction having a latency of about, say, 3 cycles, putting that workload on an FPGA would result in a ~10x slow-down! No, not that latency. Latency from external input to external output. Like the latency of receiving an ethernet frame to the point you can decide what to do with it, forward, modify, etc. Or reacting to sensor input. CPUs process interrupts slowly: X86 CPUs running a non-realtime OS can take sweet 50 microseconds+ just to get to run interrupt service routine let alone doing something useful with the data. Oh, and generally no FPU, SIMD, etc. is available in this ISR [0]. So ISR generally queues the event (on Linux a Bottom Half and on Windows a DPC, deferred procedure call) to actually process the event. The queued events might execute tens of milliseconds later in bad cases. Microcontrollers can sometimes process external interrupts pretty quickly, it's not impossible to be done in 100 nanoseconds. But it really depends, and you usually have to be pretty careful to achieve this. And even then you often have significant jitter in the processing time. FPGAs can start to handle an external I/O event on the same clock cycle, and might very well be driving I/O output pin a clock cycle later, if no pipelining is required. So an FPGA might be done in mere 10 nanoseconds in some ideal cases. With little to no jitter. [0]: Well, you can do FPU/SIMD in interrupt service routine, but need to handle all register state storage on your own, and if you mess up, it'll cause very interesting bugs in the userland application software.