4 ms·

Firstly, if this FPGA-based system is using DRAM and implements caching, results may vary due to a single variable or data structured being in a cache line or n

Firstly, if this FPGA-based system is using DRAM and implements caching, results may vary due to a single variable or data structured being in a cache line or not, which may or may not delay a code path by up to 10-100 ns (typical DRAM latency). This discrepancy may never be discovered in tests because the access patterns by which an FPGA (or CPU) decides to cache data are very dependent on the state of the system. Clearly, the author did not know what he is talking about.

inoop 15y ago
I'm not an expert on such systems. Could you to elaborate on this a bit more?
- alain94040 15y ago
  The only design I can think of in an FPGA that would use a "DRAM cache" is a processor. And by definition, no one assumes processor code execution to be timing accurate to 10ns.
  - gvb 15y ago
    Lot of people use SDRAM with FPGAs. http://www.google.com/search?channel=fs&q=fpga+sdram&ie=utf-8&oe=utf-8 http://www.google.com/search?channel=fs&q=fpga+sdram&#38... SDRAM is used commonly with FPGAs for storing largish amounts of data at a reasonable price (static RAM is fast, predictable, and very expensive for large fast memory arrays). SDRAMs are horribly complex state machine driven pipelined devices. On a system I was working on, a simple write operation to the SDRAM could vary from several clock cycles to 52 (IIRC) clock cycles, depending on what was going on before the "simple" write. I forgot what the worst case stack up was, but it was probably something about a refresh followed by a page miss that needed a precharge... AnandTech has a good intro... note the simplified state machine diagram (Figure 2) is not simple. If they published the full state machine, your head would explode. http://www.anandtech.com/show/3851/everything-you-always-wanted-to-know-about-sdram-memory-but-were-afraid-to-ask/ http://www.anandtech.com/show/3851/everything-you-always-wan... Trivia: The data throughput listed by SDRAM manufacturers is the best case sequential access fully pipelined scenario when all -nine- eight planets align. Speculating here: The variability of SDRAM access latency is one reason for putting a high speed cache in front of it. For instance, the cache could be a simple FIFO operation to buffer the data to avoid dropping data when an operation got hit by a long latency stackup. The SDRAM may have the average throughput to handle the data stream, but the (unpredictable) major bumps in latency may required "cache" buffering to keep from dropping data.
- HardyLeung 15y ago
  The author assumes that FPGA works in a way similar to CPU, and makes various assertion. But the way it works is that it implements a logic/circuitry except that such logic/circuitry was specified by some bits in the memory (hence the word "programmable"). The logic stays unchanged and all bits in the memory that specifies the logic needs to be available at any given time, so the clock cycle is rather deterministic and there is no such thing as "caching" the bits that describe the logic. Granted, FPGA does interface with other components such as external DRAM so technically it could be affected by some caching behavior there, but it is highly unlikely that caching plays as big of a role as would be if it is run by a CPU.
- CamperBob 15y ago
  An FPGA is not a magic, nondeterministic device, but nothing more or less than a lot of ordinary logic gates in one convenient package, implemented with bundles of small LUTs and flip-flops. If you don't use your FPGA to implement a memory controller or other device with variable latency, you don't get a device with variable latency. Timing instruments can be challenging to design, but as a rule, they are fairly trivial to test (waves hands dismissively). I wouldn't look for the problem at the gate or RTL level in the timing gear... but the interface between the insanely complex detector hardware and the timing gear is harder to dismiss with hand waving. If there's a bug, it will be in some part of the system that can't be easily characterized for its so-called "residual noise" performance.
mrb 15y ago
It is pretty common to use DRAM with an FPGA to manipulate large amounts of data. I clarified my essay: "eg. to store or manipulate large quantities of timestamps or events data that do not fit in SRAM".
- HardyLeung 15y ago
  If so, I suggest you change the title to "Neutrinos Faster Than Light, or DRAM artifacts?"
  - mrb 15y ago
    Changed to "artifacts of the FPGA" (as in "the FPGA platform", which includes the crystal oscillator, DRAM, and everything else that might be responsible).
    - HardyLeung 15y ago
      I understand you don't want to change because it does invalidate a big portion of your article. But by dancing around the problem, instead of addressing it, you did weaken the trustworthiness of your article, and make it sound more of a witch hunt (read: chasing idea without much evidence). You have two points that I think are reasonable to raise: (1) is there a mistake (intentional or not) in the assumed margin of error in the measuring equipment? (2) is the DRAM caching behavior taken into consideration? Neither of these two points have anything to do with FPGA per se. Now by including the enchilada word "FPGA platform" into the fray (whereas it was clearly not the intention at the beginning), I feel like you are making things up as you go, which is unfortunate since part of your argument is valid.
      - mrb 15y ago
        I cannot title it "DRAM artifacts" because it would be only 1 of the 3 different causes I speculate on (DRAM, oscillator, FPGA clock). I chose the word "FPGA" to match the terminology of the paper which uses the word as a synonym for the Data Acquisition System.

inoop 15y ago

I'm not an expert on such systems. Could you to elaborate on this a bit more?

alain94040 15y ago
The only design I can think of in an FPGA that would use a "DRAM cache" is a processor. And by definition, no one assumes processor code execution to be timing accurate to 10ns.
- gvb 15y ago
  Lot of people use SDRAM with FPGAs. http://www.google.com/search?channel=fs&q=fpga+sdram&ie=utf-8&oe=utf-8 http://www.google.com/search?channel=fs&q=fpga+sdram&#38... SDRAM is used commonly with FPGAs for storing largish amounts of data at a reasonable price (static RAM is fast, predictable, and very expensive for large fast memory arrays). SDRAMs are horribly complex state machine driven pipelined devices. On a system I was working on, a simple write operation to the SDRAM could vary from several clock cycles to 52 (IIRC) clock cycles, depending on what was going on before the "simple" write. I forgot what the worst case stack up was, but it was probably something about a refresh followed by a page miss that needed a precharge... AnandTech has a good intro... note the simplified state machine diagram (Figure 2) is not simple. If they published the full state machine, your head would explode. http://www.anandtech.com/show/3851/everything-you-always-wanted-to-know-about-sdram-memory-but-were-afraid-to-ask/ http://www.anandtech.com/show/3851/everything-you-always-wan... Trivia: The data throughput listed by SDRAM manufacturers is the best case sequential access fully pipelined scenario when all -nine- eight planets align. Speculating here: The variability of SDRAM access latency is one reason for putting a high speed cache in front of it. For instance, the cache could be a simple FIFO operation to buffer the data to avoid dropping data when an operation got hit by a long latency stackup. The SDRAM may have the average throughput to handle the data stream, but the (unpredictable) major bumps in latency may required "cache" buffering to keep from dropping data.
HardyLeung 15y ago
The author assumes that FPGA works in a way similar to CPU, and makes various assertion. But the way it works is that it implements a logic/circuitry except that such logic/circuitry was specified by some bits in the memory (hence the word "programmable"). The logic stays unchanged and all bits in the memory that specifies the logic needs to be available at any given time, so the clock cycle is rather deterministic and there is no such thing as "caching" the bits that describe the logic. Granted, FPGA does interface with other components such as external DRAM so technically it could be affected by some caching behavior there, but it is highly unlikely that caching plays as big of a role as would be if it is run by a CPU.
CamperBob 15y ago
An FPGA is not a magic, nondeterministic device, but nothing more or less than a lot of ordinary logic gates in one convenient package, implemented with bundles of small LUTs and flip-flops. If you don't use your FPGA to implement a memory controller or other device with variable latency, you don't get a device with variable latency. Timing instruments can be challenging to design, but as a rule, they are fairly trivial to test (waves hands dismissively). I wouldn't look for the problem at the gate or RTL level in the timing gear... but the interface between the insanely complex detector hardware and the timing gear is harder to dismiss with hand waving. If there's a bug, it will be in some part of the system that can't be easily characterized for its so-called "residual noise" performance.

mrb 15y ago

It is pretty common to use DRAM with an FPGA to manipulate large amounts of data. I clarified my essay: "eg. to store or manipulate large quantities of timestamps or events data that do not fit in SRAM".

HardyLeung 15y ago
If so, I suggest you change the title to "Neutrinos Faster Than Light, or DRAM artifacts?"
- mrb 15y ago
  Changed to "artifacts of the FPGA" (as in "the FPGA platform", which includes the crystal oscillator, DRAM, and everything else that might be responsible).
  - HardyLeung 15y ago
    I understand you don't want to change because it does invalidate a big portion of your article. But by dancing around the problem, instead of addressing it, you did weaken the trustworthiness of your article, and make it sound more of a witch hunt (read: chasing idea without much evidence). You have two points that I think are reasonable to raise: (1) is there a mistake (intentional or not) in the assumed margin of error in the measuring equipment? (2) is the DRAM caching behavior taken into consideration? Neither of these two points have anything to do with FPGA per se. Now by including the enchilada word "FPGA platform" into the fray (whereas it was clearly not the intention at the beginning), I feel like you are making things up as you go, which is unfortunate since part of your argument is valid.
    - mrb 15y ago
      I cannot title it "DRAM artifacts" because it would be only 1 of the 3 different causes I speculate on (DRAM, oscillator, FPGA clock). I chose the word "FPGA" to match the terminology of the paper which uses the word as a synonym for the Data Acquisition System.