3 ms·

At current enterprise NVMe prices, the drives alone for this must easily push past the $500k to $1M mark. It's fascinating to see this level of density, but it’

At current enterprise NVMe prices, the drives alone for this must easily push past the $500k to $1M mark. It's fascinating to see this level of density, but it’s strictly going to be hyperscaler or high-end defense/research budget territory for a long time.

mikestorrent 6mo ago
I'd double your guess on this one
buzer 6mo ago
The list price seems to be ~40M. https://www.dell.com/en-hk/shop/servers-storage-and-networking/poweredge-r7725xd/spd/poweredge-r7725xd/asper7725xd https://www.dell.com/en-hk/shop/servers-storage-and-networki... Select the 40 slot chassis and put 40 of those 245TB disks in. Comes out at ~HK$317M. Of course HK prices might also be higher than what Dell USA offers. Now how heavy the discounts you can get I don't know.
- tempest_ 6mo ago
  My understanding is no one actually pays Dell sticker price. They list that price on the website but if you talk to whatever they are calling their sales reps you get the real price.
  - buzer 6mo ago
    I know, I'm just not sure how high discount you are going to get in this kind of system. My understanding is that the discounts normally tend to be something like 30-40% so it would still be within same magnitude (10M+).
- kristjansson 6mo ago
  That can't be right. You could buy an aisle of NVL72 or _one_ 2U server?
twotwotwo 6mo ago
It is kinda neat how the density can trickle down. When an individual SSD can hold tens of TBs, recent-gen drives can do millions of random reads/s each, and one socket can handle lots of RAM and many cores, it doesn't take the fancier chassis with two sockets or lots of storage bays to handle pretty substantial data work. On the other hand, current part prices are not neat; a commodity platform only helps so much if none of what you want put in it is affordable! And other factors like power and cooling can push you away from optimizing for density. I just like that along with the ludicrious becoming possible, merely great stuff becomes more feasible.
rbanffy 6mo ago
I believe the rationale is that they are so much denser than they will compensate the price difference over the entire lifetime of the system. A 10 TB install is a full rack otherwise and this is 5% of the space. Colocation for HFT tends to be expensive and using 5% of the space for the same amount of data might make sense.
- fancyfredbot 6mo ago
  Your average HFT algorithm will not have time to access storage at all, let alone 10 PB of it. Back testing an HFT algo could very plausibly need 10PB, but I don't see that being co-located with the exchange.
  - rbanffy 6mo ago
    What would be good cases for data-heavy applications in expensive places?
    - fancyfredbot 6mo ago
      I'm not sure but I suspect that people who've run out of space in an existing data centre will be willing to pay for increased density rather than relocating.
      - rbanffy 6mo ago
        Unless latency is at a premium. This is why I thought about HFT.

mikestorrent 6mo ago

I'd double your guess on this one

buzer 6mo ago

The list price seems to be ~40M. https://www.dell.com/en-hk/shop/servers-storage-and-networking/poweredge-r7725xd/spd/poweredge-r7725xd/asper7725xd https://www.dell.com/en-hk/shop/servers-storage-and-networki... Select the 40 slot chassis and put 40 of those 245TB disks in. Comes out at ~HK$317M. Of course HK prices might also be higher than what Dell USA offers. Now how heavy the discounts you can get I don't know.

tempest_ 6mo ago
My understanding is no one actually pays Dell sticker price. They list that price on the website but if you talk to whatever they are calling their sales reps you get the real price.
- buzer 6mo ago
  I know, I'm just not sure how high discount you are going to get in this kind of system. My understanding is that the discounts normally tend to be something like 30-40% so it would still be within same magnitude (10M+).
kristjansson 6mo ago
That can't be right. You could buy an aisle of NVL72 or _one_ 2U server?

twotwotwo 6mo ago

It is kinda neat how the density can trickle down. When an individual SSD can hold tens of TBs, recent-gen drives can do millions of random reads/s each, and one socket can handle lots of RAM and many cores, it doesn't take the fancier chassis with two sockets or lots of storage bays to handle pretty substantial data work. On the other hand, current part prices are not neat; a commodity platform only helps so much if none of what you want put in it is affordable! And other factors like power and cooling can push you away from optimizing for density. I just like that along with the ludicrious becoming possible, merely great stuff becomes more feasible.

rbanffy 6mo ago

I believe the rationale is that they are so much denser than they will compensate the price difference over the entire lifetime of the system. A 10 TB install is a full rack otherwise and this is 5% of the space. Colocation for HFT tends to be expensive and using 5% of the space for the same amount of data might make sense.

fancyfredbot 6mo ago
Your average HFT algorithm will not have time to access storage at all, let alone 10 PB of it. Back testing an HFT algo could very plausibly need 10PB, but I don't see that being co-located with the exchange.
- rbanffy 6mo ago
  What would be good cases for data-heavy applications in expensive places?
  - fancyfredbot 6mo ago
    I'm not sure but I suspect that people who've run out of space in an existing data centre will be willing to pay for increased density rather than relocating.
    - rbanffy 6mo ago
      Unless latency is at a premium. This is why I thought about HFT.