4 ms·

I love this!! :) How much does 24 real cores + 126GB ram cost on the cloud? A million dollars?

by dizhn 4y ago

marginalia_nu 4y ago
Yeah I wouldn't try to run this in the cloud. Would be broke as a joke in a week. This is $5000 worth of consumer hardware, give or take.
- TremendousJudge 4y ago
  nice
alex_sf 4y ago
More than the cores and RAM, you have bigger issues with I/O (both throughput and latency) to disk and the network from cloud providers. Physical hardware, even when comparing cores/RAM 1:1, is outrageously faster than cloud services.
- nine_k 4y ago
  Don't use EBS, use the local SSD which are an option for most cloud VMs.
  - deleted 4y ago
    [deleted]
  - Nextgrid 4y ago
    Problem is that those volumes are ephemeral and may not provide the reliability guarantees that the EBS volumes do, so they're only really good as cache and not for any persistent data you care about.
    - amluto 4y ago
      I would argue that a rack-mounted chassis with lots of disks is also ephemeral, just less so: most failures in a server can be fixed by swapping some parts, and your data is still there. But AWS and its competitors don’t have an offering even close to comparable to what you can get in a commodity server. A 1U server with one or two CPU sockets and 8-12 hot-swap NVMe bays is easy to buy and not terribly expensive, and you can easily fill it with 100+ TiB of storage with several hundred Gbps of storage bandwidth and more IOPS than you are likely able to use. EC2 has no comparable offering at any price. (A Micron 9400 drive supposedly has 7GBps / 56Gbps of usable bandwidth. 10 of them gives 560Gbps, and a modern machine with lots of PCIe 5.0 lanes may actually be able to use a lot of this. As far as I can tell, you literally cannot pay AWS for anywhere near this much bandwidth, but you can buy it for $20k or so.)
      - Nextgrid 4y ago
        > I would argue that a rack-mounted chassis with lots of disks is also ephemeral, just less so: most failures in a server can be fixed by swapping some parts, and your data is still there. True but this also depends on design decisions AWS made with regards to those volumes. Indeed it could be that the volume is internally (at the hypervisor level) redundant (maybe with something like ZFS or other proprietary RAID), but there's no way to know. Furthermore, AWS doesn't allow you to really keep a tab or reservation on the physical machine your VM is on - every time a VM is powered up, it gets assigned a random host machine. If there is a hardware failure they advise you to reboot the instance so it gets rescheduled on another machine, so even though technically your data may still be on that physical host machine, you have no way to get it back. AWS' intent with these seems to be to act as transient cache/scratchpad so they don't seem to offer much durability or recovery strategies for those volumes. Their hypervisor seems to treat them as disposable which is a fair design decision considering the planned use-case, but it means you can't/shouldn't use it for any persistent data. Being in control of your own hardware (or at the very least, renting physical hardware from a provider as opposed to a VM like in AWS) will indeed allow you to get reliable direct-attach storage.
  - amluto 4y ago
    Like this? https://instances.vantage.sh/aws/ec2/im4gn.8xlarge?region=us-east-1&os=linux&cost_duration=annually&reserved_term=Standard.noUpfront https://instances.vantage.sh/aws/ec2/im4gn.8xlarge?region=us... I can buy a rather nicer 1U machine with substantially better local storage for something like half the 1-year reserved annual cost of this thing. If you buy your own servers, you can mix and match CPUs and storage, and you can get a lot of NVMe storage capacity and bandwidth, and cloud providers don’t seem to have comparable products.
DHolzer 4y ago
in Azure that's roughly 5k per year if you pay for the whole year upfront. I have the pleasure of playing with 64cores, 256gb RAM and 2xV100 for data science projects every now and then. That turns out to be roughly 32k per year.
- Culonavirus 4y ago
  I don't know, 32k per year is pretty steep... I mean ~35k, that's the ballpark number for the entire system (if bought new).
  - DHolzer 4y ago
    I share your perspective on pricing. I had a discussion with my team lead about why we haven't taken on the task of running our own machines. The rationale behind it is that while server management may seem easy, ensuring its security can be complex. Even in a sizable company, it's worth considering whether you want to shoulder the responsibility or simply pay a higher cost to have Microsoft handle it. Personally, I love hosting my own infrastructure. It's fun, potentially saves me some cash, allows me to learn a lot, and gives me full control. However, I understand now that on a company scale, others may see it differently. --edit-- I forgot to add the following: that's 32k if you run the system 24/7. Usually it's up for a few hours per month, so you end up paying maybe 2k for the whole year.
ftth_finland 4y ago
About 150€ per month as a dedicated server from Hetzner.
- belter 4y ago
  So the Cloud is the cheaper option, if you factor in energy cost and hardware depreciation.
  - ftth_finland 4y ago
    The monthly cost of a dedicated server includes everything: bandwidth, power and hardware. How do you figure a $5k annual cloud spend is cheaper than ~150€ per month?
    - belter 4y ago
      I do not understand your comment.
    - hiddencost 4y ago
      I think you're confused. ~150€ in cloud costs is cheaper than $5k cost to buy the hardware the guy has in his living room.
      - Nextgrid 4y ago
        Most people when talking about "cloud" refer to the likes of AWS/GCP/Azure rather than old-school bare-metal server hosts.
    - dizhn 4y ago
      I think parent is talking about colo.
- marginalia_nu 4y ago
  Something like €200/mo if you factor in the need for disk space as well. This is also Hetzner we're talking about. They're sort of infamous for horror stories of arbitrarily removed servers and having shitty support. They're the cheapest for a reason. But with dedicated servers, are we really talking cloud?

I love this!! :) How much does 24 real cores + 126GB ram cost on the cloud? A million dollars?

by dizhn 4y ago

I love this!! :) How much does 24 real cores + 126GB ram cost on the cloud? A million dollars?

Yeah I wouldn't try to run this in the cloud. Would be broke as a joke in a week. This is $5000 worth of consumer hardware, give or take.

TremendousJudge 4y ago
nice

alex_sf 4y ago

More than the cores and RAM, you have bigger issues with I/O (both throughput and latency) to disk and the network from cloud providers. Physical hardware, even when comparing cores/RAM 1:1, is outrageously faster than cloud services.

nine_k 4y ago
Don't use EBS, use the local SSD which are an option for most cloud VMs.
- deleted 4y ago
  [deleted]
- Nextgrid 4y ago
  Problem is that those volumes are ephemeral and may not provide the reliability guarantees that the EBS volumes do, so they're only really good as cache and not for any persistent data you care about.
  - amluto 4y ago
    I would argue that a rack-mounted chassis with lots of disks is also ephemeral, just less so: most failures in a server can be fixed by swapping some parts, and your data is still there. But AWS and its competitors don’t have an offering even close to comparable to what you can get in a commodity server. A 1U server with one or two CPU sockets and 8-12 hot-swap NVMe bays is easy to buy and not terribly expensive, and you can easily fill it with 100+ TiB of storage with several hundred Gbps of storage bandwidth and more IOPS than you are likely able to use. EC2 has no comparable offering at any price. (A Micron 9400 drive supposedly has 7GBps / 56Gbps of usable bandwidth. 10 of them gives 560Gbps, and a modern machine with lots of PCIe 5.0 lanes may actually be able to use a lot of this. As far as I can tell, you literally cannot pay AWS for anywhere near this much bandwidth, but you can buy it for $20k or so.)
    - Nextgrid 4y ago
      > I would argue that a rack-mounted chassis with lots of disks is also ephemeral, just less so: most failures in a server can be fixed by swapping some parts, and your data is still there. True but this also depends on design decisions AWS made with regards to those volumes. Indeed it could be that the volume is internally (at the hypervisor level) redundant (maybe with something like ZFS or other proprietary RAID), but there's no way to know. Furthermore, AWS doesn't allow you to really keep a tab or reservation on the physical machine your VM is on - every time a VM is powered up, it gets assigned a random host machine. If there is a hardware failure they advise you to reboot the instance so it gets rescheduled on another machine, so even though technically your data may still be on that physical host machine, you have no way to get it back. AWS' intent with these seems to be to act as transient cache/scratchpad so they don't seem to offer much durability or recovery strategies for those volumes. Their hypervisor seems to treat them as disposable which is a fair design decision considering the planned use-case, but it means you can't/shouldn't use it for any persistent data. Being in control of your own hardware (or at the very least, renting physical hardware from a provider as opposed to a VM like in AWS) will indeed allow you to get reliable direct-attach storage.
- amluto 4y ago
  Like this? https://instances.vantage.sh/aws/ec2/im4gn.8xlarge?region=us-east-1&os=linux&cost_duration=annually&reserved_term=Standard.noUpfront https://instances.vantage.sh/aws/ec2/im4gn.8xlarge?region=us... I can buy a rather nicer 1U machine with substantially better local storage for something like half the 1-year reserved annual cost of this thing. If you buy your own servers, you can mix and match CPUs and storage, and you can get a lot of NVMe storage capacity and bandwidth, and cloud providers don’t seem to have comparable products.

DHolzer 4y ago

in Azure that's roughly 5k per year if you pay for the whole year upfront. I have the pleasure of playing with 64cores, 256gb RAM and 2xV100 for data science projects every now and then. That turns out to be roughly 32k per year.

Culonavirus 4y ago
I don't know, 32k per year is pretty steep... I mean ~35k, that's the ballpark number for the entire system (if bought new).
- DHolzer 4y ago
  I share your perspective on pricing. I had a discussion with my team lead about why we haven't taken on the task of running our own machines. The rationale behind it is that while server management may seem easy, ensuring its security can be complex. Even in a sizable company, it's worth considering whether you want to shoulder the responsibility or simply pay a higher cost to have Microsoft handle it. Personally, I love hosting my own infrastructure. It's fun, potentially saves me some cash, allows me to learn a lot, and gives me full control. However, I understand now that on a company scale, others may see it differently. --edit-- I forgot to add the following: that's 32k if you run the system 24/7. Usually it's up for a few hours per month, so you end up paying maybe 2k for the whole year.

ftth_finland 4y ago

About 150€ per month as a dedicated server from Hetzner.

belter 4y ago
So the Cloud is the cheaper option, if you factor in energy cost and hardware depreciation.
- ftth_finland 4y ago
  The monthly cost of a dedicated server includes everything: bandwidth, power and hardware. How do you figure a $5k annual cloud spend is cheaper than ~150€ per month?
  - belter 4y ago
    I do not understand your comment.
  - hiddencost 4y ago
    I think you're confused. ~150€ in cloud costs is cheaper than $5k cost to buy the hardware the guy has in his living room.
    - Nextgrid 4y ago
      Most people when talking about "cloud" refer to the likes of AWS/GCP/Azure rather than old-school bare-metal server hosts.
  - dizhn 4y ago
    I think parent is talking about colo.
marginalia_nu 4y ago
Something like €200/mo if you factor in the need for disk space as well. This is also Hetzner we're talking about. They're sort of infamous for horror stories of arbitrarily removed servers and having shitty support. They're the cheapest for a reason. But with dedicated servers, are we really talking cloud?