4 ms·

I'm curious what the cost would be to run this type of hardware at any cloud vendor? Does it even exist?

jeffbee 6y ago
That's the wrong way to think about the cloud. A better way to think about it would be "how much database traffic (and storage) can I serve from Cloud Whatever for $xxx". Then you need to think about what your realistic effective utilization would be. This server has 153600 GB of raw storage. That kind of storage would cost you $46000 (retail) in Cloud Spanner every month, but I doubt that's the right comparison. The right math would probably be that they have 250 million customers and perhaps 1KB of real information per customer. Now the question becomes why you would ever buy 24×6400 GB of flash memory to store this scale of data.
whitepoplar 6y ago
Not completely comparable, but Hetzner offers the following dedicated server that costs 637 euro/month (maxed out): - 32-core AMD EPYC - 512GB ECC memory - 8x 3.84TB NVMe datacenter drives - Unmetered 1gbps bandwidth
dastbe 6y ago
(I work at AWS, but this is just for fun) Checking out AWS side, the closest I think you'd get is the x1.32xlarge, which would translate to 128 vCPU (which on intel generally means 64 physical cores) and close to 2TB of RAM. nvme storage is only a paltry 4TB, so you'd have to make up the rest with EBS volumes. You'd also get a lower clock speed than they are getting out of the EPICs
- WatchDog 6y ago
  If you eschew RDS, the largest you can go up to seems to be a u-24tb1.metal. 448 vcpu, 24TiB of RAM, $70 an hour. ~$52k per month.
  - Aeolun 6y ago
    Oh, only $70 per hour. I think I can start that for an hour or so before my wallet is exhausted :P
    - ficklepickle 6y ago
      I spun up a 32 vCore spot instance to crack my own WiFi password once. It only cost ~$2 and I felt so powerful.
- PeterCorless 6y ago
  spittakes reading the suggestion of replacing NVMe with EBS I mean, yeah, I guess you can. But a lot depends on your use case and SLA. If you need to keep ultra-low p99s — single digits — then EBS is not a real option. But if you don't mind latencies, then yeah, fine. Don't get me wrong: EBS is great. But it's not a panacea and strikes me as a mismatch for a high performance monster system. If you need NVMe, you need NVMe.
  - dastbe 6y ago
    It wasn't a suggestion of replacement, it was saying that even the closest instance still doesn't get you 24TB of on-host storage, having to use EBS instead.
    - PeterCorless 6y ago
      This starts going down a rabbit hole of tiered storage then. Because unless you design for that, you can only be as fast as your slowest storage. It'd be like tying a 20 kg weight to the leg of a sprinter otherwise.

That's the wrong way to think about the cloud. A better way to think about it would be "how much database traffic (and storage) can I serve from Cloud Whatever for $xxx". Then you need to think about what your realistic effective utilization would be. This server has 153600 GB of raw storage. That kind of storage would cost you $46000 (retail) in Cloud Spanner every month, but I doubt that's the right comparison. The right math would probably be that they have 250 million customers and perhaps 1KB of real information per customer. Now the question becomes why you would ever buy 24×6400 GB of flash memory to store this scale of data.

whitepoplar 6y ago

Not completely comparable, but Hetzner offers the following dedicated server that costs 637 euro/month (maxed out): - 32-core AMD EPYC - 512GB ECC memory - 8x 3.84TB NVMe datacenter drives - Unmetered 1gbps bandwidth

dastbe 6y ago

(I work at AWS, but this is just for fun) Checking out AWS side, the closest I think you'd get is the x1.32xlarge, which would translate to 128 vCPU (which on intel generally means 64 physical cores) and close to 2TB of RAM. nvme storage is only a paltry 4TB, so you'd have to make up the rest with EBS volumes. You'd also get a lower clock speed than they are getting out of the EPICs

WatchDog 6y ago
If you eschew RDS, the largest you can go up to seems to be a u-24tb1.metal. 448 vcpu, 24TiB of RAM, $70 an hour. ~$52k per month.
- Aeolun 6y ago
  Oh, only $70 per hour. I think I can start that for an hour or so before my wallet is exhausted :P
  - ficklepickle 6y ago
    I spun up a 32 vCore spot instance to crack my own WiFi password once. It only cost ~$2 and I felt so powerful.
PeterCorless 6y ago
spittakes reading the suggestion of replacing NVMe with EBS I mean, yeah, I guess you can. But a lot depends on your use case and SLA. If you need to keep ultra-low p99s — single digits — then EBS is not a real option. But if you don't mind latencies, then yeah, fine. Don't get me wrong: EBS is great. But it's not a panacea and strikes me as a mismatch for a high performance monster system. If you need NVMe, you need NVMe.
- dastbe 6y ago
  It wasn't a suggestion of replacement, it was saying that even the closest instance still doesn't get you 24TB of on-host storage, having to use EBS instead.
  - PeterCorless 6y ago
    This starts going down a rabbit hole of tiered storage then. Because unless you design for that, you can only be as fast as your slowest storage. It'd be like tying a 20 kg weight to the leg of a sprinter otherwise.