9 ms·

Inside the IBM FlashSystem 900 All-Flash Array

34 points by desdiv 10y ago

DigitalJack 10y ago
They must not be expecting to produce these at scale if they are loading them with FPGAs. Or the markup is enormous. Being IBM, the latter might be it. edit: looking at the picture, they are XC7K410T's. 500,000 flops... almost 2000 multipliers. Looking at the specs on FPGAs these days makes me giddy. http://www.xilinx.com/support/documentation/data_sheets/ds180_7Series_Overview.pdf http://www.xilinx.com/support/documentation/data_sheets/ds18...
- pliu 10y ago
  Your comment made me curious, so I went looking and found a price sheet. I see two SKU's, one at $32K and one at $39K, I think probably this is just for the chassis. The storage modules list at $22K, $46K and $92K for the 1.2TB, 2.9TB and 5.7TB models, respectively. I don't think too many people really end up paying list, but I have to imagine the markups are very healthy. This thing looks rad af. I work exclusively with IAAS stuff now, but in a past life I used to spend a lot of time with enterprise hardware like this. I bet those modules feel real good when you snap em in. http://www-01.ibm.com/common/ssi/rep_ca/3/897/ENUS115-003/ENUS-115-003-LIST_PRICES_2015_02_23.PDF http://www-01.ibm.com/common/ssi/rep_ca/3/897/ENUS115-003/EN...
  - Sephr 10y ago
    The SM950 can be had at $300 for 512GB with 2.5/1.5 GB/s sequential read/write and 300K IOPS. RAID together 6 of them in parallel and you'll have something with better specs and storage than the 2.9TB module ($46K) for just $1.8K (albeit with less endurance, though not sure how much less).
    - trhway 10y ago
      IBM's margins are heavenly, no argument here. Yet what you propose is apple to oranges. The SM950 would have endurance of 1K to may be 3K writes. The IBM's flash is 10K. This is enterprise/database, not desktop, ie. you need the write endurance. It is one of the main reasons why we don't see many database servers with SSD arrays like you proposed, and instead we see huge RAM servers with RAID-ed HDDs.
      - deleted 10y ago
        [deleted]
      - kev009 10y ago
        That is bad engineering, HDDs have about the same drive writes per day as SDDs once you get into the TB sized SSDs.
kev009 10y ago
Curious to see FPGAs at the center of a product like this, especially from IBM which has always been willing to do low volume tapeouts (helps to own a foundary)
- wl 10y ago
  Perhaps this indicates that it's still a work in progress and they may want to update the programming on the FPGAs in the future?
- NKCSS 10y ago
  Think it's closely related to their Netezza line; a 'Big Data' storeage solution that also features a FCPA-centered design for the storage/T-SQL engine.
  - kev009 10y ago
    I don't. I think they are using an FPGA because it is convenient and they are willing to throw away some of the margins on their product for ease of implementation. That particular FPGA has a PCIe hard IP, and they probably implement the NAND read/write/balancing using its LUTs and interface using its digital I/O. A consumer SSD (or really any product) might start off life like this, and then tape out to an ASIC before release if it is going to ship in any significant volume. The ASIC will have drastically lower power and, once sufficient volume hits, much higher margins. I don't buy the programmable argument either, you can implement programmable cores (this is what basically any high feature NIC is like Chelsio, SolarFlare, etc) while still baking in the PCIe, NAND interfaces, offloads like CRC/erasure coding. Those FPGAs each cost several hundred dollars. Imagine if each PCI device in your computer (sound, network, video) had a full Xeon on board. Not entirely dissimilar.
    - duskwuff 10y ago
      > Those FPGAs each cost several hundred dollars. Several hundred? Try several THOUSAND. Here's the part that's seen in the photos: http://www.digikey.com/short/3pmq5w http://www.digikey.com/short/3pmq5w A fully loaded FlashSystem has around $100,000 of FPGA silicon in it (12 modules x 4 FPGAs/module x ~$2000/FPGA).
homero 10y ago
Looks like it's for algorithmic trading
- NKCSS 10y ago
  ...or the AD industry...
DonHopkins 10y ago
When will the memory itself be running a database server? Or is that already a thing?
- trhway 10y ago
  https://en.wikipedia.org/wiki/NVDIMM https://en.wikipedia.org/wiki/NVDIMM NVDIMM-N is basically flash backed RAM
tthayer 10y ago
Doesn't the Pure Flashblade do something similar? https://regmedia.co.uk/2016/03/13/flashblade_950.jpg https://regmedia.co.uk/2016/03/13/flashblade_950.jpg

34 points by desdiv 10y ago

They must not be expecting to produce these at scale if they are loading them with FPGAs. Or the markup is enormous. Being IBM, the latter might be it. edit: looking at the picture, they are XC7K410T's. 500,000 flops... almost 2000 multipliers. Looking at the specs on FPGAs these days makes me giddy. http://www.xilinx.com/support/documentation/data_sheets/ds180_7Series_Overview.pdf http://www.xilinx.com/support/documentation/data_sheets/ds18...

pliu 10y ago
Your comment made me curious, so I went looking and found a price sheet. I see two SKU's, one at $32K and one at $39K, I think probably this is just for the chassis. The storage modules list at $22K, $46K and $92K for the 1.2TB, 2.9TB and 5.7TB models, respectively. I don't think too many people really end up paying list, but I have to imagine the markups are very healthy. This thing looks rad af. I work exclusively with IAAS stuff now, but in a past life I used to spend a lot of time with enterprise hardware like this. I bet those modules feel real good when you snap em in. http://www-01.ibm.com/common/ssi/rep_ca/3/897/ENUS115-003/ENUS-115-003-LIST_PRICES_2015_02_23.PDF http://www-01.ibm.com/common/ssi/rep_ca/3/897/ENUS115-003/EN...
- Sephr 10y ago
  The SM950 can be had at $300 for 512GB with 2.5/1.5 GB/s sequential read/write and 300K IOPS. RAID together 6 of them in parallel and you'll have something with better specs and storage than the 2.9TB module ($46K) for just $1.8K (albeit with less endurance, though not sure how much less).
  - trhway 10y ago
    IBM's margins are heavenly, no argument here. Yet what you propose is apple to oranges. The SM950 would have endurance of 1K to may be 3K writes. The IBM's flash is 10K. This is enterprise/database, not desktop, ie. you need the write endurance. It is one of the main reasons why we don't see many database servers with SSD arrays like you proposed, and instead we see huge RAM servers with RAID-ed HDDs.
    - deleted 10y ago
      [deleted]
    - kev009 10y ago
      That is bad engineering, HDDs have about the same drive writes per day as SDDs once you get into the TB sized SSDs.

kev009 10y ago

Curious to see FPGAs at the center of a product like this, especially from IBM which has always been willing to do low volume tapeouts (helps to own a foundary)

wl 10y ago
Perhaps this indicates that it's still a work in progress and they may want to update the programming on the FPGAs in the future?
NKCSS 10y ago
Think it's closely related to their Netezza line; a 'Big Data' storeage solution that also features a FCPA-centered design for the storage/T-SQL engine.
- kev009 10y ago
  I don't. I think they are using an FPGA because it is convenient and they are willing to throw away some of the margins on their product for ease of implementation. That particular FPGA has a PCIe hard IP, and they probably implement the NAND read/write/balancing using its LUTs and interface using its digital I/O. A consumer SSD (or really any product) might start off life like this, and then tape out to an ASIC before release if it is going to ship in any significant volume. The ASIC will have drastically lower power and, once sufficient volume hits, much higher margins. I don't buy the programmable argument either, you can implement programmable cores (this is what basically any high feature NIC is like Chelsio, SolarFlare, etc) while still baking in the PCIe, NAND interfaces, offloads like CRC/erasure coding. Those FPGAs each cost several hundred dollars. Imagine if each PCI device in your computer (sound, network, video) had a full Xeon on board. Not entirely dissimilar.
  - duskwuff 10y ago
    > Those FPGAs each cost several hundred dollars. Several hundred? Try several THOUSAND. Here's the part that's seen in the photos: http://www.digikey.com/short/3pmq5w http://www.digikey.com/short/3pmq5w A fully loaded FlashSystem has around $100,000 of FPGA silicon in it (12 modules x 4 FPGAs/module x ~$2000/FPGA).

homero 10y ago

Looks like it's for algorithmic trading

NKCSS 10y ago
...or the AD industry...

DonHopkins 10y ago

When will the memory itself be running a database server? Or is that already a thing?

trhway 10y ago
https://en.wikipedia.org/wiki/NVDIMM https://en.wikipedia.org/wiki/NVDIMM NVDIMM-N is basically flash backed RAM

tthayer 10y ago

Doesn't the Pure Flashblade do something similar? https://regmedia.co.uk/2016/03/13/flashblade_950.jpg https://regmedia.co.uk/2016/03/13/flashblade_950.jpg