6 ms·

They are crazy! Very interesting case. How they support such DB? As I know FPGA has limits on updates. What tools they use to create relational algebra?

blackguardx 10y ago
Most FPGAs (all except for a dozen or so) use SRAM to hold the configuration while in operation. SRAM has no limit to how many times it can change states. Off-chip flash typically stores the configuration for long term needs and the FPGA loads the configuration from that at power-up. Applications that require frequent changing of the configuration use processors to load from somewhere else instead (hdd, network, some other storage meachanism.)
duskwuff 10y ago
> As I know FPGA has limits on updates. I'm not sure what you're trying to refer to here. Are you confusing FPGAs with flash memory?
- teemwerk 10y ago
  There are FPGAs (mostly Microsemi) that use internal flash memory to hold their configuration, but you are correct, most use eeproms/
- MaDeuce 10y ago
  Probably not. A hardware description language (HDL) is used to program the behavior of the FPGA (that's the FP in FPGA). The HDL is usually stored in flash or EEPROM. As you mention, flash supports a finite number of writes; I'm not sure about EEPROM behavior. The bottom line is that the typical FPGA can only be programmed a fixed number of times (e.g., I think Altera is ~100). Some FPGAs can only be programmed once.
  - theyoungestgun 10y ago
    Many FPGAs these days have flash memories that are good for ~10K writes.
  - trsohmers 10y ago
    EEPROMs are mostly only used for CPLDs (very cheap very very small FPGAs)... "Big" FPGAs as used here use SRAM, and are rated to be reprogrammed 100s of thousands of times, but should last for millions. I have reprogrammed my Xilinx and Altera FPGAs hundreds of times over a course of a couple of days ;) EDIT: I should note that in the case of using SRAM, the device would only be programmed as long as the device is powered on. You would use EEPROM even on large FPGAs if you have the same design to be flashed to the FPGA between reboots. I personally never use EEPROM for storing the design as I have my FPGAs constantly connected to a computer for programming. My main point is that the FPGA can be programmed a functionally infinite number of times, though if you want the design to remain through reboots through an external memory, the limitation is that external memory, not the FPGA itself.
    - nickpsecurity 10y ago
      Are you able to simulate your Rex chip on one FPGA, do you just do a piece at a time, or over several? And what kind of FPGA's? Curious what hardware it takes to support developing something like that.
      - trsohmers 10y ago
        We do most stuff on a single FPGA... Our development flow is parameterized in a way so we can generate RTL with different numbers of tiles, SRAM per tile, and many other details within seconds of an RTL change. We have an Achronix Speedster that can fit 16 of our cores on it (our full test chip), while our cheaper Xilinx Kintex Ultrascale boards can comfortably fit 8 but can do 16 if we cut the SRAM size down from what we have in the real chip. You could split designs across FPGAs (The Dini Group sells systems using Xilinx and Altera chips built for this), but I have heard it is a real PITA, and we're doing fine with what we have right now. I'd love to get a real ASIC simulation/emulation platform like Synopsys's Zebu or HAPS... Both use multiple Xilinx FPGAs linked together and easily integrate with the Synopsys EDA flow. Cadence also has the Palladium emulation platform, but those boxes approach the cost of a house in the Bay Area.
        10y ago
  - duskwuff 10y ago
    > The bottom line is that the typical FPGA can only be programmed a fixed number of times This is only true if the configuration is rewritten to flash every time the device is reconfigured, which would not be typical. Any design which depended on reconfiguring an FPGA frequently would reconfigure the device directly using JTAG or active serial programming, or would switch between multiple configurations in flash. I personally have several FPGA development boards which I've reconfigured hundreds of times. There is effectively no limit to the number of times a device can be reconfigured; the configuration is stored in SRAM, which has no write limit.
  - Steeeve 10y ago
    Typically you only program them a handful of times. (first release of logic, then re-programming for updates if there are any) Once a logic structure is in place, it's just a matter of passing signals/data through the FPGA.
  - chocolatebunny 10y ago
    most Fpgas can be passively loaded from a processor, one word at a time. That processor can get the FPGA image from anywhere. It takes longer to load an FPGA this way but it gives you a lot more flexibility and avoids the issue you're concerned about.
  - astrodust 10y ago
    "Some FPGAs can only be programmed once." That's not an FPGA then. That's a PLA. Even then most of those can be reset using ultra-violet light, a simple process.
    - adwn 10y ago
      > That's not an FPGA then. That's a PlA. No, it's still called "FPGA". The "field-programmable" part refers to them being programmable after they have been manufactured, by someone else than the chip manufacturer. It doesn't matter if you can't reprogram it after it's been soldered onto a board, that's not what "field-programmable" means. > Even then most of those can be reset using ultra-violet light, a simple process. Once-programmable FPGAs are typically anti-fuse based. There is no way to reset their programming (we're working with space-grade single-programmable FPGAs, and yes, we're extremely careful about what gets put on them).
alain94040 10y ago
The host PC does all the database accesses and the FPGAs do the processing (search, filter, etc.) on the fly. Therefore there is no problem with updates.
chollida1 10y ago
> As I know FPGA has limits on updates. HFT firms have been banging on the drum of making FPGA's infinity programmable for a decade and lots of commercial FPGA's can be re-flashed now without worrying about the number of times you do it. I've seen re-flash counts up to 100,000 times. Hey put this in the "what has HFT actually done for the world" list of answers:)
- astrodust 10y ago
  I've never seen an advertised limit on FPGAs. You'd probably grow old and die before you ran out of reflashes since that process takes a fair amount of time. 100,000 flashes at even five minutes each is over a year of relentless flashing.
  - rhizome 10y ago
    Is there a Jenkins plugin for that? :)
- adwn 10y ago
  What are you talking about? > HFT firms have been banging on the drum of making FPGA's infinity programmable for a decade That's strange, because FPGAs have been infinitely reprogrammable for longer than that. > lots of commercial FPGA's can be re-flashed now That's even stranger, because the vast majority of commercial FPGAs are not flash-based, but SRAM-based and can be reprogrammed as often as the registers in your CPU. Besides, HFT firms would be using SRAM-based FPGAs because they are faster and larger than flash-based ones, the advantages of flash-based ones are useless to them. > "what has HFT actually done for the world" Well, not this, that's for sure.
deleted 10y ago
[deleted]
NwtnsMthd 10y ago
FPGAs generally use RAM based look up tables and have to reload their configuration (programming) from non-volatile memory at power up. Since an FPGA is a RAM based device, it doesn't really have a limit on how many times you can reprogram it.

blackguardx 10y ago

Most FPGAs (all except for a dozen or so) use SRAM to hold the configuration while in operation. SRAM has no limit to how many times it can change states. Off-chip flash typically stores the configuration for long term needs and the FPGA loads the configuration from that at power-up. Applications that require frequent changing of the configuration use processors to load from somewhere else instead (hdd, network, some other storage meachanism.)

duskwuff 10y ago

> As I know FPGA has limits on updates. I'm not sure what you're trying to refer to here. Are you confusing FPGAs with flash memory?

teemwerk 10y ago
There are FPGAs (mostly Microsemi) that use internal flash memory to hold their configuration, but you are correct, most use eeproms/
MaDeuce 10y ago
Probably not. A hardware description language (HDL) is used to program the behavior of the FPGA (that's the FP in FPGA). The HDL is usually stored in flash or EEPROM. As you mention, flash supports a finite number of writes; I'm not sure about EEPROM behavior. The bottom line is that the typical FPGA can only be programmed a fixed number of times (e.g., I think Altera is ~100). Some FPGAs can only be programmed once.
- theyoungestgun 10y ago
  Many FPGAs these days have flash memories that are good for ~10K writes.
- trsohmers 10y ago
  EEPROMs are mostly only used for CPLDs (very cheap very very small FPGAs)... "Big" FPGAs as used here use SRAM, and are rated to be reprogrammed 100s of thousands of times, but should last for millions. I have reprogrammed my Xilinx and Altera FPGAs hundreds of times over a course of a couple of days ;) EDIT: I should note that in the case of using SRAM, the device would only be programmed as long as the device is powered on. You would use EEPROM even on large FPGAs if you have the same design to be flashed to the FPGA between reboots. I personally never use EEPROM for storing the design as I have my FPGAs constantly connected to a computer for programming. My main point is that the FPGA can be programmed a functionally infinite number of times, though if you want the design to remain through reboots through an external memory, the limitation is that external memory, not the FPGA itself.
  - nickpsecurity 10y ago
    Are you able to simulate your Rex chip on one FPGA, do you just do a piece at a time, or over several? And what kind of FPGA's? Curious what hardware it takes to support developing something like that.
    - trsohmers 10y ago
      We do most stuff on a single FPGA... Our development flow is parameterized in a way so we can generate RTL with different numbers of tiles, SRAM per tile, and many other details within seconds of an RTL change. We have an Achronix Speedster that can fit 16 of our cores on it (our full test chip), while our cheaper Xilinx Kintex Ultrascale boards can comfortably fit 8 but can do 16 if we cut the SRAM size down from what we have in the real chip. You could split designs across FPGAs (The Dini Group sells systems using Xilinx and Altera chips built for this), but I have heard it is a real PITA, and we're doing fine with what we have right now. I'd love to get a real ASIC simulation/emulation platform like Synopsys's Zebu or HAPS... Both use multiple Xilinx FPGAs linked together and easily integrate with the Synopsys EDA flow. Cadence also has the Palladium emulation platform, but those boxes approach the cost of a house in the Bay Area.
      - 10y ago
- duskwuff 10y ago
  > The bottom line is that the typical FPGA can only be programmed a fixed number of times This is only true if the configuration is rewritten to flash every time the device is reconfigured, which would not be typical. Any design which depended on reconfiguring an FPGA frequently would reconfigure the device directly using JTAG or active serial programming, or would switch between multiple configurations in flash. I personally have several FPGA development boards which I've reconfigured hundreds of times. There is effectively no limit to the number of times a device can be reconfigured; the configuration is stored in SRAM, which has no write limit.
- Steeeve 10y ago
  Typically you only program them a handful of times. (first release of logic, then re-programming for updates if there are any) Once a logic structure is in place, it's just a matter of passing signals/data through the FPGA.
- chocolatebunny 10y ago
  most Fpgas can be passively loaded from a processor, one word at a time. That processor can get the FPGA image from anywhere. It takes longer to load an FPGA this way but it gives you a lot more flexibility and avoids the issue you're concerned about.
- astrodust 10y ago
  "Some FPGAs can only be programmed once." That's not an FPGA then. That's a PLA. Even then most of those can be reset using ultra-violet light, a simple process.
  - adwn 10y ago
    > That's not an FPGA then. That's a PlA. No, it's still called "FPGA". The "field-programmable" part refers to them being programmable after they have been manufactured, by someone else than the chip manufacturer. It doesn't matter if you can't reprogram it after it's been soldered onto a board, that's not what "field-programmable" means. > Even then most of those can be reset using ultra-violet light, a simple process. Once-programmable FPGAs are typically anti-fuse based. There is no way to reset their programming (we're working with space-grade single-programmable FPGAs, and yes, we're extremely careful about what gets put on them).

alain94040 10y ago

The host PC does all the database accesses and the FPGAs do the processing (search, filter, etc.) on the fly. Therefore there is no problem with updates.

chollida1 10y ago

> As I know FPGA has limits on updates. HFT firms have been banging on the drum of making FPGA's infinity programmable for a decade and lots of commercial FPGA's can be re-flashed now without worrying about the number of times you do it. I've seen re-flash counts up to 100,000 times. Hey put this in the "what has HFT actually done for the world" list of answers:)

astrodust 10y ago
I've never seen an advertised limit on FPGAs. You'd probably grow old and die before you ran out of reflashes since that process takes a fair amount of time. 100,000 flashes at even five minutes each is over a year of relentless flashing.
- rhizome 10y ago
  Is there a Jenkins plugin for that? :)
adwn 10y ago
What are you talking about? > HFT firms have been banging on the drum of making FPGA's infinity programmable for a decade That's strange, because FPGAs have been infinitely reprogrammable for longer than that. > lots of commercial FPGA's can be re-flashed now That's even stranger, because the vast majority of commercial FPGAs are not flash-based, but SRAM-based and can be reprogrammed as often as the registers in your CPU. Besides, HFT firms would be using SRAM-based FPGAs because they are faster and larger than flash-based ones, the advantages of flash-based ones are useless to them. > "what has HFT actually done for the world" Well, not this, that's for sure.

deleted 10y ago

[deleted]

NwtnsMthd 10y ago

FPGAs generally use RAM based look up tables and have to reload their configuration (programming) from non-volatile memory at power up. Since an FPGA is a RAM based device, it doesn't really have a limit on how many times you can reprogram it.