3 ms·

I would love to see the comparison between running this and just buying cloud compute time. Given its a constant operation which probably requires high speed i

I would love to see the comparison between running this and just buying cloud compute time. Given its a constant operation which probably requires high speed interconnects. However it would be interesting to see how close in cost cloud is to a bare metal build.

philjohn 12y ago
here's a hint - it's nowhere near close in cost to a bare metal build, at least not in my experience. We run a small clustered database across 8 nodes. Total cost for the hardware was ~ $3000 per node. Cost for a comparable node in EC2 per month? $1500. Even with co-lo costs for power and bandwidth, and buying 10gige switches it's far cheaper for us to host it in house than it is to use EC2.
- JoeAltmaier 12y ago
  How about the cost of bricks and mortar? The cost of an administrator? The cost of spares and disaster recovery? Sure the purchase cost looks good, but what about the total cost of ownership?
  - lucian1900 12y ago
    It's still tiny compared to AWS, which is really expensive. That doesn't mean it's not useful, but it's still tens of times more expensive than using your own hardware.
    - JoeAltmaier 12y ago
      Often total cost of ownership is about the same as the equipment cost. So they start to look comparable actually.
      - lucian1900 12y ago
        That's not what I've found. Even just renting dedicated physical machines has been significantly cheaper overall.
  - micro_cam 12y ago
    Former engineer on large research grants (in the us) here. At least in the US a lot of those recurring costs are covered by the standard overhead your institution tacks onto all grants for administrative and facilities costs so you buy a bunch of machines with extended warranties (which should be longer then the terms of most grants) and that covers almost all of your costs. Once they go out of warranty you either use them until they die, scrounge for funds to keep them going or hopefully get a new grant. Also, things are changing, but it is/was hard to ear mark funds for the cloud in a grant so spending the money up front was often what needed to happen.
mscman 12y ago
Cloud HPC Solutions Architect here. In this use case, cloud computing wouldn't really suffice. For a company who needs 24x7 access to compute power of that scale, they really will benefit from buying a standalone system like this. While some cloud providers now have very high-speed, low-latency interconnects, they only scale so wide in a virtual environment. This is due to limitations in the physical datacenter space and the way node provisioning works on the back-end. If you truly need extremely wide, low-latency, high bandwidth interconnect for a large period of time, on-prem is still (probably) your best option.

philjohn 12y ago

here's a hint - it's nowhere near close in cost to a bare metal build, at least not in my experience. We run a small clustered database across 8 nodes. Total cost for the hardware was ~ $3000 per node. Cost for a comparable node in EC2 per month? $1500. Even with co-lo costs for power and bandwidth, and buying 10gige switches it's far cheaper for us to host it in house than it is to use EC2.

JoeAltmaier 12y ago
How about the cost of bricks and mortar? The cost of an administrator? The cost of spares and disaster recovery? Sure the purchase cost looks good, but what about the total cost of ownership?
- lucian1900 12y ago
  It's still tiny compared to AWS, which is really expensive. That doesn't mean it's not useful, but it's still tens of times more expensive than using your own hardware.
  - JoeAltmaier 12y ago
    Often total cost of ownership is about the same as the equipment cost. So they start to look comparable actually.
    - lucian1900 12y ago
      That's not what I've found. Even just renting dedicated physical machines has been significantly cheaper overall.
- micro_cam 12y ago
  Former engineer on large research grants (in the us) here. At least in the US a lot of those recurring costs are covered by the standard overhead your institution tacks onto all grants for administrative and facilities costs so you buy a bunch of machines with extended warranties (which should be longer then the terms of most grants) and that covers almost all of your costs. Once they go out of warranty you either use them until they die, scrounge for funds to keep them going or hopefully get a new grant. Also, things are changing, but it is/was hard to ear mark funds for the cloud in a grant so spending the money up front was often what needed to happen.

mscman 12y ago

Cloud HPC Solutions Architect here. In this use case, cloud computing wouldn't really suffice. For a company who needs 24x7 access to compute power of that scale, they really will benefit from buying a standalone system like this. While some cloud providers now have very high-speed, low-latency interconnects, they only scale so wide in a virtual environment. This is due to limitations in the physical datacenter space and the way node provisioning works on the back-end. If you truly need extremely wide, low-latency, high bandwidth interconnect for a large period of time, on-prem is still (probably) your best option.