21 ms·

AWS doesn't make sense for scientific computing

• bsenftner 4y ago
  This is the case for a large class of big data + high compute applications. Animation / simulation in engineering, planning, forecasting, not to mention entertainment require pipelines the typical cloud is simply too expensive to use.
• idiot900 4y ago
  This rings true for me. I have a federal grant that prohibits me from using its funds for capital acquisitions: i.e. servers. But I can spend it on AWS at massive cost for minimal added utility for my use case. Even though it would be a far better use of taxpayer funds to buy the servers, I have to rent them instead.
  • adgjlsfhk1 4y ago
    Can you get your university to buy some servers for unrelated reasons and have them rent them to you?
    • lostmsu 4y ago
      Doesn't have to be a university either. Depending on the amount of compute needed any capable IT guy can do it for you from their garage with a contract.
    • chrisseaton 4y ago
      Well that’s just rebuilding AWS badly. I’ve used academic-managed time-sharing setups and have some horror stories.
  • giantrobot 4y ago
    I'm not saying AWS is automatically the best option but the question isn't just servers. It's servers, networking hardware, HVAC, a facility to put them all in, and at least a couple people to run and maintain it all. The TCO of some servers is way higher than the cost of the hardware.
  • boldlybold 4y ago
    Lots of places (Hetzner for example) will rent you servers at 10-25% the cost of AWS if you want dedicated hardware, without the ability to autoscale. You can even set up a K8s cluster there if the overhead is worth it.
    • intelVISA 4y ago
      Fond memories of Hetzner asking for my driving license as ID for renting a $2 VPS. Lost a customer for life with that nonsense.
      • perihelions 4y ago
        Also lost a customer for life here. In my case they asked to let an AI scan my face through a webcam. Nah.
  • testplzignore 4y ago
    > prohibits me from using its funds for capital acquisitions What is a legitimate reason for this restriction?
    • blep_ 4y ago
      I can think of a few ways to abuse it while still spinning it as "for research". The obvious one is to buy a $9999 gaming machine with several of whatever the fanciest GPU on the market is at the time, and say you're doing machine learning. So my guess is it's an overly broad patch for that sort of thing.
      • Fomite 4y ago
        Not really - this is also true for things with no particular "civilian" use.
        • blep_ 4y ago
          Yes, that's why I described it as "overly broad".
          • Fomite 4y ago
            But it's also not why the law was made. There are other regulations to keep people from installing Steam on their ML workstations (which also cover machines below the threshold). It's entirely about one grant-giving entity not wanting to pay for a piece of capital equipment that will have use beyond the project they're funding. It's a federal regulation, and it comes up far more commonly with lab equipment than it ever does with computers.
    • Fomite 4y ago
      Basically, the granting organization doesn't want to pay for the full cost of capital equipment that will - either via time or capacity - not be fully used for that grant. There are other grant mechanisms for large capital expenditures. The problem is the thresholds haven't shifted in a long time, so you can easily trigger it with a nice workstation. But then, the budget for a modular NIH R01 was set in 1999, so thats hardly a unique problem.
• dastbe 4y ago
  using on-demand for latency insensitive work, especially when you’re also very cost sensitive, isn’t the right choice. spot instances will get you somewhere in the realm of the hetzner/on-prem numbers.
  • Sebb767 4y ago
    But, as the article points out, you are still paying a lot of money for features that you don't need for scientific computing. Also, AWS is notoriously easy to undercut with on-prem hardware, especially if your budget is large and your uptime requirements aren't - you'll save a few hundred thousand a year alone by not having to hire expert engineers for on-call duty and extreme reliability.
  • lebovic 4y ago
    Even spot instances on AWS are still over 2x more expensive per month than Hetzner. The cheapest c5a.24xlarge spot instance right now is $1.5546/hr in us-east-1c. That's $1134.86/mo, excluding data transfer costs. If you transfer out 10 TB over the course of a month, that's another $921.60/mo – or now 4x more expensive than Hetzner. Using the estimate from the article, spot instances are still over 8x more expensive than on-prem for scientific computing.
  • dekhn 4y ago
    Even more importantly, if you have any reasonable amount of spend on cloud, you can get preferred pricing agreements. As much as I hate to talk to "salespeople", I did manage to cut millions in costs per year with discounts on serving and storage. Personally, when I estimate the total cost of ownership of scientific cloud computing versus on prem (for extremely large-scale science with both significant server, storage, and bandwidth requirements) the cloud ends up winning for a number of reasons. I've seen a lot of academics who disagree but then I find out they use their grad students to manage their clusters.
• mbreese 4y ago
  I completely agree for most cases. In many scientific computing applications, compute time isn’t the factor you prioritize in the good/fast/cheap triad. Instead, you often need to do things as cheaply as possible. And your data access isn’t always predictable, so you need to keep results around for an extended period of time. This makes storage costs a major factor. For us, this alone was enough to move workloads away from cloud and onto local resources.
• jerjerjer 4y ago
  Sure? I mean, if you have: 1) A large enough queue of tasks 2) Users/donstream willing to wait using your own infrastructure always wins (alsuming free labor) since you can load your own infrastructure to ~95% pretty much 24/7 which is unbeatable.
  • mrweasel 4y ago
    It might also depend on how long you're actually willing to wait. There's nothing stopping you from having a job queue in AWS, and you can setup things up so that instances are only running if the price is low enough. Otherwise completely agree, there might be some cases where the cost of labour means that you're better off running something in AWS, even if that requires someone to do the configuration as well.
• aschleck 4y ago
  This is sort of a confusing article because it assumes the premise of "you have a fixed hardware profile" and then argues within that context ("Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers".) Of course if you're getting 100% utilization then you'll find better raw pricing (and this article conveniently leaves out staffing costs), but this model misses one of the most powerful parts of cloud providers: autoscaling. Why would you want to waste scientist time by making them wait in a queue when you can just instead autoscale as high as needed? Giving scientists a tight iteration loop will likely be the biggest cost reduction and also the biggest benefit. And if you're doing that on prem then you need to provision for the peak load, which drives your utilization down and makes on prem far less cost effective.
  • secabeen 4y ago
    Generally, scientists aren't blocked while they are waiting on a computational queue. The results of a computation are needed eventually, but there is lots of other work that can be done that doesn't depend on a specific calculation.
    • jefftk 4y ago
      It's good to learn how not to be blocked on long-running calculations. On the other hand, if transitioning to a bursty cloud model means you can do your full run in hours instead of weeks, that has real impact on how many iterations you can do and often does appreciably affect velocity.
      • secabeen 4y ago
        It can, if you have the technical ability to write code that can leverage the scale-out that most bursty-cloud solutions entail. Coding for clustering can be pretty challenging, and I would generally recommend a user target a single large system with job that takes a week over trying to adapt that job to a clustered solution of 100 smaller systems that can complete it in 8 hours.
• secabeen 4y ago
  The general rule of thumb in the HPC world is if you can keep a system computing for more than 40% of the time, it will be cheaper to buy.
• zatarc 4y ago
  Why does no one consider colocation services anymore? And why do people only know Hetzner, OVH and Linode as alternatives to the big cloud providers? There are so many good and inexpensive server hosting providers, some with decades of experience.
  • lostmsu 4y ago
    Any particular you could recommend for GPU?
    • theblazehen 4y ago
      https://www.genesiscloud.com/ https://www.genesiscloud.com/ is pretty decent
    • dkobran 4y ago
      There are several tier-two clouds that offer GPUs but I think they generally fall prey to the many of the same issues you'll find with AWS. There is a new generation of accelerator native clouds e.g. Paperspace (https://paperspace.com https://paperspace.com) that cater specifically to HPC, AI, etc. workloads. The main differentiators are: - much larger GPU catalog - support for new accelerators e.g. Graphcore IPUs - different pricing structure that address problematic areas for HPC such as egress However, one of the most important differences is the lack of unrelated web services related components that pose a major distraction/headache to users that don't have a DevOps background (which AWS obviously caters to). AWS can be incredibly complicated. Simple tasks are encumbered by a whole host of unrelated options/capabilities and the learning curve is very steep. A platform that is specifically designed to serve the scientific computing audience can be much more streamlined and user-friendly for this audience. Disclosure: I work on Paperspace.
    • sabalaba 4y ago
      Lambda GPU Cloud has the cheapest A100s of that group. https://lambdalabs.com/service/gpu-cloud https://lambdalabs.com/service/gpu-cloud Lambda A100s - $1.10 / hr Paperspace A100s - $3.09 / hr Genesis A100s - no A100s but their 3090 (1/2 the speed of 100) is - $1.30 / hr for half the speed
• prpl 4y ago
  Actually computing is fine for most use cases (spot instances, preemptible VMs on GCP) and have been used in lots of situations, even at CERN. Where it also excels is if you need any kind of infrastructure, because no HPC center has figured a reasonable approach to that (some are trying with k8s). Also, obviously, you get a huge selection of hardware. Where cloud/aws doesn’t make sense is storage, especially if you need egress, and if you actually need IB
• twawaaay 4y ago
  On the other hand it makes sense if you just need to borrow their infrastructure for a while to calculate something. A lot of scientific computing isn't happening continuously, and a lot of it is one time experiment or maybe couple of times after which you would have to tear down and reassign. Another fun fact people forget is our ability to predict future is still pretty poor. Not only that, we are biased towards thinking we can predict it when in fact this is complete bullshit. You have to buy and set up infrastructure before you can use it and then you have to be ready to use it. What if you are not ready? What if you will not need as much resources? What if you stop needing it earlier than you thought? When you borrow it from AWS you have flexibility to start using it when you are ready and drop it immediately when you no longer need it. Which has value on its own. At the company I work for we found out and basically banned signing long term contracts for discounts. We found that, on average, we pay many times more for unused services than whatever we gained through discounts. Also when you pay for the resources there is incentive to improve efficiency. When you have basically prepaid for everything that incentive is very small and is basically limited to making sure you have to stay within limits.
• KaiserPro 4y ago
  Its much more complex than described. The author is making a brilliant argument for getting a secondhand workstation and shoving under their desk. If you are doing multi machine batch style processing, then you won't be using ondemand, you'd use the spot pricing. The missing argument in that part is storage costs. Managing a high speed, highly available synchronous file system that can do a sustained 50gb/sec is hard bloody work (no S3 isnt a good fit, too much management overhead) Don't get me wrong AWS _is_ expensive if you are using a machine for more than a month or two. however if you are doing highly parallel stuff, Batch and lustre on demand is pretty ace. If you are doing a multi-year project, then real steel is where its at. Assuming you have factored in hosting, storage and admin costs.
  • lostmsu 4y ago
    Isn't 50GB sec like 5 NVMe Gen 5 SSDs + 1 or 2 for redundancy? Actually, you are right. Consumer SSDs I've seen only do about 1.5GB/s sustained.
    • pclmulqdq 4y ago
      Even (high-end) consumer SSDs can saturate a PCIe gen 4 x4 link if you are doing sequential reads. Non-sequential hurts on even enterprise SSDs.
    • davidmr 4y ago
      Not in the context the person you responded to meant it. Yes, you can very easily get 50GB/s from a few NVMe devices on a single box. Getting 50GB/s on a POSIX-ish filesystem exported to 1000 servers is very possible and common, but orders of magnitude more complicated. 500GB/s is tougher still. 5TB/s is real tough, but real fun.
  • bushbaba 4y ago
    Checkout Apache Iceberg which makes it fairly trivial to get high throughput from S3 without much fine-tuning. Bursts from 0 to 50Gbps should be possible from S3 without much effort, just have object sizes that are in the NN+ MiB range. Personally, Lustre is a mess, it's expensive and even more pain to fine-tune.
• julienchastang 4y ago
  I’ve also been skeptical of the commercial cloud for scientific computing workflows. I don’t think this cost benefit analysis mentions it, but the commercial cloud makes even less sense when you take into account brick and mortar considerations. In other words, if your company/institution has already paid for the machine rooms, sys admins, networks, the physical buildings, the commercial cloud is even less appealing. This is especially true with “persistent services” for example data servers that are always on because they handle real-time data, for example. Another aspect of scientific computing on the commercial cloud that’s a pain if you work in academia is procurement or paying for the cloud. Academic groups are much more comfortable with the grant model. They often operate on shoe-string budgets and are simply not comfortable entering a credit card number. You can also get commercial cloud grants, but they often lack long-term, multiyear continuity.
  • mattkrause 4y ago
    It's often not that they're "not comfortable"; it's that we're often flat-out not allowed to.
    • Fomite 4y ago
      This. It's got nothing to do with "comfort". I use cloud computing all the time in the rest of my life, but the rest of my life isn't subject to university policies and state regulations.
• aBioGuy 4y ago
  Furthermore, scientific computing often (usually?) involves trainees. It can difficult to train people when small mistakes can have five figure bills.
  • Moissanite 4y ago
    This is the biggest un-addressed problem, IMO. Getting more scientific computing done in the cloud is where we are inevitably trending, but no-one yet has a good answer for completely ad-hoc, low value experimentation and skill building in cloud. I see universities needing to maintain clusters to allow PhDs and postdocs to develop their computing skills for a good while yet.
  • atrettel 4y ago
    I agree that this is a big thing to consider here too. I set up a computing cluster in grad school and it was much less costly to make a mistake there than it would have been in a cloud service. Re-running something only wasted wall time and not any money. That said, money is not the only scarce resource here. Researchers can get allocations at university and government HPC systems, but you then have to be quite careful with your allocation of computing time. I remember keeping track of the number of SUs (core-hours) I was burning quite carefully when I used university clusters, since once it is gone, you might not get any more time.
• walnutclosefarm 4y ago
  Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almost invariably leave major costs out of their calculation - assuming they can put their servers in a closet,maintain them themselves, do all the security infrastructure, provide redundancy and still get to shared compute when they have an overflow need. When the closet starts to smoke because they stuffed it with too many cheaply sourced, hot-running cores and GPUs, or gets hacked by one of their postdocs resulting in an institutional HIPAA violation, well, that's not their fault. Put like for like in a well managed data center against negotiated and planned cloud services, and the former may still win, but it won't be dramatically cheaper, and figured over depreciable lifetime and including opportunity cost, may cost more. It takes work to figure out which is true.
  • mattpallissard 4y ago
    I also managed HPC data centers and I agree with you. I feel like the term data center is a key word there. There is a point of scaling where it's just cheaper to manage a data center with a dedicated in house team. I think that holds true in other industries as well. As far as HPC goes specifically, we could get some of the financial numbers to make sense in the cloud (cpu intensive jobs), but couldn't make it work for others (data intensive jobs shipping PB's around on the reg) That and HPC has a lot of grant funding. It can be quite advantageous for an org to use an on prem data center almost like a slush fund. Can keep key projects running that would otherwise die when they are having a rough funding year.
  • W-Stool 4y ago
    Let me echo this as someone who once was responsible for HPC computing in a research intensive public university. Most career academics have NO IDEA how much enterprise computing infrastructure costs. If a 1 terabyte USB hard drive is $40 at Costco we (university IT) must be getting a much better deal than that. Take this argument and apply it to any aspect of HPC computing and that's what you're fighting against. The closet with racks of gear and no cooling is another fond memory. Don't forget the AC terminal strips that power the whole thing, sourced from the local dollar store.
• rpep 4y ago
  I think there are some things this misses about the scientific ecosystem in Universities/etc. that can make the cloud more attractive than it first appears: * If you want to run really big jobs e.g. with multiple multi-GPU nodes, this might not even be possible depending on your institution or your access. Most research-intensive Universities have a cluster but they’re not normally big machines. For regional and national machines, you usually have to bid for access for specific projects, and you might not be successful. * You have control of exactly what hardware and OS you want on your nodes. Often you’re using an out of date RHEL version and despite spack and easybuild gaining ground, all too often you’re given a compiler and some old versions of libraries and that’s it. * For many computationally intensive studies, your data transfer actually isn’t that large. For e.g. you can often do the post-processing on-node and then only get aggregate statistics about simulation runs out.
• 0xbadcafebee 4y ago
  AWS is fantastic for scientific computing. With it you can: - Deploy a thousand servers with GPUs in 10 minutes, churn over a giant dataset, then turn them all off again. Nobody ever has to wait for access to the supercomputer. - Automatically back up everything into cold storage over time with a lifecycle policy. - Avoid the massive overhead of maintaining HPC clusters, labs, data centers, additional staff and training, capex, load estimation, months/years of advance planning to be ready to start computing. - Automation via APIs to enable very quick adaptation with little coding. - An entire universe of services which ramp up your capabilities to analyze data and apply ML without needing to build anything yourself. - A marketplace of B2B and B2C solutions to quickly deploy new tools within your account. - Share data with other organizations easily. AWS costs are also "retail costs". There are massive savings to be had quite easily.
  • Fomite 4y ago
    One thing to consider: I don't control my AWS account. I don't even have an AWS account in my professional life. I tell my IT department what I want. They tell the AWS people in central IT what they want. It's set up. At some point I get an email with login information. I email them again to turn it off. Do I hate this system? Yes. Is it the system I have to work with? Also yes. "AWS as implemented by any large institution" is considerably less agile than AWS itself.
    • deleted 4y ago
      [deleted]
• pclmulqdq 4y ago
  Even as a big cloud detractor, I have to disagree with this. A lot of scientific computing doesn't need a persistent data center, since you are running a ton of simulations that only take a week or so, and scientific computing centers at big universities are a big expense that isn't always well-utilized. Also, when they are full, jobs can wait weeks to run. These computing centers have fairly high overhead, too, although some of that is absorbed by the university/nonprofit who runs them. It is entirely possible that this dynamic, where universities pay some of the cost out of your grant overhead, makes these computing centers synthetically cheaper for researchers when they are actually more expensive. One other issue here is that scientific computing really benefits from ultra-low-latency infiniband networks, and the cloud providers offer something more similar to a virtualized RoCE system, which is a lot slower. That means accounting for cloud servers potentially being slower core-for-core.
  • lebovic 4y ago
    Author here. I agree with your points! I use AWS for a computational biology company I'm working on. A lot of scientific computing can spin up and down within a couple hours on AWS and benefits from fast turnaround. Most academic HPCs (by # of clusters) are slower than a mega-cluster on AWS, not well utilized, and have a lot of bureaucratic process. That said, most of scientific computing (by % of total compute) happens in a different context. There's often a physical machine within the organization that's creating data (e.g. a DNA sequencer, particle accelerator, etc), and a well-maintained HPC cluster that analyzes that data. The researchers have already waited months for their data, so another couple weeks in a queue doesn't impact their cycle. For that context, AWS doesn't really make sense. I do think there's room for a cloud provider that's geared towards an HPC use-case, and doesn't have the app-inspired limits (e.g data transfer) like AWS, GCP, and Azure.
    • primax 4y ago
      Honestly spot instances sound perfect for those kind of requirements
  • davidmr 4y ago
    This is tangential to your point, but I’ll just mention that Azure has some properly specced out HPC gear: IB, FPGAs, the works. You used to be able to get time on a Cray XC with an Ares interconnect, but I never have occasion to use it, so I don’t know if you still can. They’ve been aggressively hiring top-notch HPC people for a while.
• citizenpaul 4y ago
  No one seems to consider colo data centers anymore as even an option?
  • remram 4y ago
    My university owns hardware in multiple locations, plus uses hardware in a collocation, and still uses the cloud for bursting (overflow). You can't beat the provisioning time of cloud providers which is measured in seconds.
• captainmuon 4y ago
  A former colleague did his PHD in particle physics with a novel technique (matrix element method). I can't really explain it, but it is extremely CPU intensive. That working group did it on CERN's resources, and they had to borrow quotas from a bunch of other people. For fun they calculated how much it would have cost on AWS and came up with something ridiculous like 3 million euros.
  • rirze 4y ago
    I would imagine CERN's resources are essentially a data center comparable to a small cloud provider's resources.
  • wenc 4y ago
    I can’t specifically to CERN and the exact workload. But bear in mind that the 3MM euros is non negotiated sticker pricing. In real life, negotiated pricing can be much much less depending on your org size and spend. This is a variable most people neglect.
    • captainmuon 4y ago
      That is true, and a large part of the theoretical cost was probably also traffic, and the use of nonstandard nodes. They could have gotten a much more realistic price. I guess the point is also that scientists often don't realize computer costs money, when the computers are already bought
  • dguest 4y ago
    The bigger experiments will routinely burn through tens of millions worth of computing. But 10 million euros isn't much for these experiments. The issue is that they are publicly funded: any country is much happier to build a local computing center and lend it to scientists than to fork the money over to an American cloud provider. (The expensive part of these experiments is simulating billions of collisions and how the thousands of outgoing particles propagate though a detector the size of a small building. Simulating a single event takes around a minute on a modern CPU, and the experiments will simulate billions of events in a few months. If AWS is charging 5 cents a minute it works out to tens of millions easy.)
• deleted 4y ago
  [deleted]
• bee_rider 4y ago
  Is genomic code typically distributed-memory parallel? I'm under the impression that it is more like batch processing, not a ton of node-to-node communication but you want lots of bandwidth and storage. If you are doing a big distributed-memory numerical simulation, on the other hand, you probably want infiniband I guess. AWS seems like an OK fit for the former, maybe not great for the latter...
  • pclmulqdq 4y ago
    The fastest way to do a lot of genomics stuff is with FPGA accelerators, which also aren't used by most of the other tenants in a multi-tenant scientific computing center. The cloud is perfect for that kind of work.
    • bee_rider 4y ago
      That's interesting. It is sort of funny that I was right (putting genomics in the "maybe good for cloud" bucket) for the wrong reason (characterizing it as more suited for general-purpose commodity resources, rather than suited for the more niche FPGA platform).
    • chrchang523 4y ago
      This is a bit overstated. Yes, FPGA accelerators can be used effectively in some common genomics workflows. However, my experience is that the right software running on regular Intel/AMD/ARM processors is very competitive with FPGA-using solutions.
• slaymaker1907 4y ago
  Cloud worked really well for me when I was in school. A lot of the time, I would only need a beefy computer for a few hours at a time (often due to high memory usage) and you can/could rent out spot instances for very cheap. There are about 730 hours per month so the cost calculus is very different for a student/researcher who needs fast turnaround times (high performance), but only for a short period of time. However, I know not all HPC/scientific computing works that way and some workloads are much more continuous.
  • a2tech 4y ago
    Thats how my department uses the cloud--we have an image we store up at AWS geared towards a couple of tasks and we spin up a big instance when we need it, run the task, pull out the results, then stop the machine. Total cost sub-100 dollars. If we had to go the HPC group we'd have to fight with them to get the environment configured, get access to the system, get payment setup, teach the faculty to use the environment, etc. Its just a pain for very little gain.
• fwip 4y ago
  The killer we've seen is data egress costs. Crunching the numbers for some of our pipelines, we'd actually be paying more to get the data out of AWS than to compute it.
  • bhewes 4y ago
    Data movement has become the number one cost in system builds energy wise.
    • boldlybold 4y ago
      As in, the networking equipment consumes the most energy? Given the 30x markup on AWS egress I'm inclined to say it's more about incentives and marketing, but I'd love to learn otherwise.
• renewiltord 4y ago
  > Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers – that's the API equivalent of storing your inbound API requests, and then responding to them months later Makes sense if the jobs are all low urgency. We have a similar problem in trading so we have a composite solution with non-cloud simulation hardware and additional AWS hardware. That's because we have the high utilization solution combined with high urgency.
  • Fomite 4y ago
    I did have to chuckle a bit because, working on HPC simulations of the pandemic during the pandemic, there was an awful lot of "This needs to be done tomorrow" urgency.
• thayne 4y ago
  > Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers That sounds very much like an argument for a cloud. Instead of waiting months to do your processing, you spin up what you need, then tear it down when you are done.
  • withinboredom 4y ago
    The queue then just turns into the bank account. The queue doesn’t magically go away.
    • adolph 4y ago
      Classic iron triangle, pick two: * cheap * fast * available https://en.wikipedia.org/wiki/Project_management_triangle https://en.wikipedia.org/wiki/Project_management_triangle
• thesausageking 4y ago
  I'm suspicious of the author's actual experience. The fact that scientific computing has a different pattern than the typical web app is actually a good thing. If you can architect large batch jobs to use spot instances, it's 50-80% cheaper. Also this bit: "you can keep your servers at 100% utilization by maintaining a queue of requested jobs" isn't true in practice. The pattern of research is the work normally comes in waves. You'll want to train a new model or run a number of large simulations. And then there will be periods of tweaking and work on other parts. And then more need for a lot of training. Yes, you can always find work to put on a cluster to keep it >90% utilization, but if it can be elastic (and has compute has budget attached to it), it will rise and fall.
  • lebovic 4y ago
    Author here! I worked for the computing infrastructure for a DNA sequencing facility, and I run a computational biology infrastructure company (trytoolchest.com, YC W22). Both are built on AWS, so I do think AWS in scientific computing has its use-cases – mostly in places where you can't saturate a queue or you want a fast cycle time. Spot instances are still pretty expensive for a steady queue (2x of Hetzer monthly costs, for reference), and you still have to pay AWS data transfer egress costs – which are at least 30x more expensive than a colo or on-prem, if you're saturating a 1 Gbps link. This post was born from frustration at AWS for their pricing and offerings after trying to get people to switch to AWS in scientific computing for years :)
• wenc 4y ago
  Calculating costs based on sticker price is sometimes misleading because there’s another variable: negotiated pricing, which can be much much lower than sticker prices, depending on your negotiating leverage. Different companies pay different prices for the same product. If you’ve ever worked at a big company or university (any place where you spend at scale), you’ll know you rarely pay sticker price. Software licensing is particularly elastic because it’s almost zero marginal cost. Raw cloud costs are largely a function of energy usage and amortized hardware costs — there’s a certain minimum you can’t go under but there remains a huge margin that is open to be negotiated on. Startups/individuals rarely even think about this because they rarely qualify. But big orgs with large spends do. You can get negotiated cloud pricing.
  • racking7 4y ago
    This is definitely true for cloud retail prices. However, this becomes not true in cases I've seen when there is an existing discount. Reserved instances, for example.
• wistlo 4y ago
  Database analyst for a large communication company here. I have similar doubts about AWS for certain kinds of intensive business analysis. Not API based transactions, but back-office analysis where complex multi-join queries are run in sequence against tables with 10s of millions of records. We do some of this with SQL servers running right on the desktop (and one still uses Excel with VLOOKUP). We have a pilot project to try these tasks in a new Azure instance. I look forward to seeing how it performs, and at what cost.
  • AyyWS 4y ago
    Do you have disaster recovery / high availability requirements? SQL server on a desktop has a lot of single points of failure.
• latchkey 4y ago
  I read this as a thinly veiled advertisement for the authors service, toolchest.
  • lebovic 4y ago
    Toolchest actually runs scientific computing on AWS! I'm just frustrated by what we can build, because most scientific compute can't effectively shift to AWS
    • latchkey 4y ago
      As others have noted, there are many other providers out there. I think your essay would have had more value if it didn't end with an advertisement.
      • deleted 4y ago
        [deleted]
• danking00 4y ago
  I think this post is identifying scientific computing with simulation studies and legacy workflows, to a fault. Scientific computing includes those things, but it also includes interactive analysis of very large datasets as well as workflows designed around cloud computing. Interactive analysis of large datasets (e.g. genome & exome sequencing studies with 100s of 1000s of samples) is well suited to low-latency, server-less, & horizontally scalable systems (like Dremel/BigQuery, or Hail [1], which we build and is inspired by Dremel, among other systems). The load profile is unpredictable because after a scientist runs an analysis they need an unpredictable amount of time to think about their next step. As for productionized workflows, if we redesign the tools used within these workflows to directly read and write data to cloud storage as well as to tolerate VM-preemption, then we can exploit the ~1/5 cost of preemptible/spot instances. One last point: for the subset of scientific computing I highlighted above, speed is key. I want the scientist to stay in a flow state, receiving feedback from their experiments as fast as possible, ideally within 300 ms. The only way to achieve that on huge datasets is through rapid and substantial scale-out followed by equally rapid and substantial scale-in (to control cost). [1] https://hail.is https://hail.is
  • jessfyi 4y ago
    I've followed Hail and applaud the Broad Institute's work wrt establishing better bioinformatics software and toolkits so I hope this doesn't come as rude, but I can't imagine an instance in a real industry or academic workflow where you need 300ms feedback from an experiment to "maintain flow" considering how long experiments on data that large (especially exome sequencing!) take overall? My (likely lacking) imagination aside I guess what I'm really saying is that I don't know what's preventing the usecase you've described from being performed locally considering there'd be even less latency?
    • danking00 4y ago
      300ms is my ideal latency, but we don’t achieve that under all circumstances. Even for blob storage, I see as much as 100ms latency. That said, my laptop has maybe 8 cores. Even if I had 0ms reads from an SSD, I’m compute bound for some tasks. Moreover, I think we have differing definitions of “experiment”. In the context of a sequencing study, I think an “experiment” can be as simple as answering the hypothesis: does the missingness of a genotype correlate with any sample metadata (e.g. sequencing platform). You might try to test that hypothesis by looking at a PC1-PC2 plot with points colored by sequencing platform where the PCA is conducted on the 0/1 indicator matrix of missingness. In the dry lab, that is what I mean by experiment. By that definition, a scientist does many experiments a day. Particularly for sequencing studies, these experiments are data-intensive, I need to run a simple computation on a lot of data to confirm the hypothesis.
• bgro 4y ago
  When I was looking at AWS for personal use, I first thought it was oddly expensive even when factoring in not having to buy the hardware. When I looked at just what the electricity cost to run it myself would be, I think that addition alone turned out AWS was actually cheaper. This is without factoring in cooling / dedicated space / maintenance.
• snorkel 4y ago
  Buying your own fleet of dedicated servers seems like a smart move in the short term, but then five years from now you’ll get someone on the team insisting that they need the latest greatest GPU to run their jobs. Cloud providers give you the option of using newer chipsets without having to re-purchase your entire server fleet every five years.
  • lebovic 4y ago
    In HPC land, most hardware is amortized over five years and then replaced! If you keep your service in life for five years at high utilization, you're doing great. For example, the Blue Waters supercomputer at UIUC was originally expected to last five years, although they kept it in service for nine; it was considered a success: https://www.ncsa.illinois.edu/historic-blue-waters-supercomputer-ceasing-operations/ https://www.ncsa.illinois.edu/historic-blue-waters-supercomp...
• gammarator 4y ago
  Astronomy is moving more and more to cloud computing: https://www.nature.com/articles/d41586-020-02284-7 https://www.nature.com/articles/d41586-020-02284-7 https://arxiv.org/abs/1907.06320 https://arxiv.org/abs/1907.06320
• Moissanite 4y ago
  This has been my exact field of work for a few years now; in general I have found that: When people claim it is 10x more expensive to use public cloud, they have no earthly idea what it actually costs to run a HPC service, a data centre, or do any of the associated maintenance. When the claim is 3x more expensive in the cloud, they do know those things but are making a bad faith comparison because their job involves running an on-premises cluster and they are scared of losing their toys. When the claim is 0-50% more to run in the cloud, someone is doing the math properly and aiming for a fair comparison. When the claim is that cloud is cheaper than on-prem, you are probably talking to a cloud vendor account manager whose colleagues are wincing at the fact that they just torched their credibility.
  • lebovic 4y ago
    Author here! I think running an HPC service that has a steady queue in AWS can be more than 3x as expensive. What type of HPC do you work in? Maybe I'm over-indexing on computational biology.
    • Moissanite 4y ago
      All types of HPC; I'm a sysadmin/consultant. I don't think the problem with the cost gap is overestimating cloud costs but rather underestimating on-prem costs. Also, failing to account for financing differences and opportunity-costs of large up-front capital purchases.
  • johnklos 4y ago
    This is oversimplifying things a bit. It can categorically be stated that for a year's worth of CPU compute, local will always be less than Amazon. Of course, putting percentages on it doesn't work - there are just too many variables. There are many admins out there who have no idea what an Alpha is who'll swear that if you're not buying Dell or HP hardware at a premium with expensive support contracts, you're doing things wrong and you're not a real admin. Visit Reddit's /r/sysadmin if you want to see the kind of people I'm talking about. The point is that if people insist on the most expensive, least efficient type of servers such as Dell Xeons with ridiculous service contracts, the savings over Amazon won't be large. It's a cumulative problem, because trying to cool and house less efficient hardware requires more power and that hardware ultimately has less tolerance for non-datacenter cooling. Rethink things. You can have AMD Threadripper / EPYC systems in larger rooms that require less overall cooling, that have better temperature tolerance, that're more reliable in aggregate, which cost less and for which you can easily keep around spare parts which would give better turnaround and availability than support contracts from Dell / HP. Suddenly your compute costs are halved, because of pricing, efficiency, overall power, real estate considerations... So percentages don't work, but the bottom line is that when you're doing lots of compute, over time it's always cheaper locally, even if you do things the "traditional" expensive and inefficient way, so arguing percentages with so many variables doesn't make any sense - it's still cheaper, no matter what.
• COGlory 4y ago
  >a month-long DNA sequencing project can generate 90 TB of data Our EM facility generates 10 TB of raw data per day, and once you start computing it, that increases by 30%-50% depending on what you do with it. Plus, moving between network storage and local scratch for computational steps basically never ends and keeps multiple 10 Gbe links saturated 100% of the time.
• adamsb6 4y ago
  I've never worked in this space, but I'm curious about the need for massive egress. What's driving the need to bring all that data back to the institution? Could whatever actions have to be performed on the data also be performed in AWS? Also while briefly looking into this I found that AWS has an egress waiver for researchers and educational instiutions: https://aws.amazon.com/blogs/publicsector/data-egress-waiver-available-for-eligible-researchers-and-institutions/ https://aws.amazon.com/blogs/publicsector/data-egress-waiver...
  • COGlory 4y ago
    Well for starters, if you are NIH or NSF funded, they have data storage requirements you must meet. So usually this involves something like tape backups in two locations. The other is for reproducibility - typically you need to preserve lots of in-between steps for peer review and proving that you aren't making things up. Some intermediary data is wiped out, but usually only if it can be quickly and easily regenerated.
  • jpeloquin 4y ago
    Regarding the waiver—"The maximum discount is 15 percent of total monthly spending on AWS services". Was very excited at first. As for leaving data in AWS, data is often (not always) revisited repeatedly for years after the fact. If new questions are raised about the results it's often much easier to check the output than rerun the analysis. And cloud storage is not cheap. But yes it sometimes makes sense to egress only summary statistics and discard the raw data.
  • jltsiren 4y ago
    I've worked the last few years in a research institute well-funded by academic standards. During that time, people around me have used self-hosted hardware, at least three different cloud providers, and HPC clusters hosted by several different organizations. I've also worked with people from many other institutions, each of which has made its own infrastructure choices. You may use AWS yourself, but your collaborators have made their own choices for their own reasons. The data goes where it's needed, often crossing institutional boundaries.
• Mave83 4y ago
  I agree with the article. We at croit.io support customers around the globe to build their clusters and save huge amounts. For example, AWS S3 compared to Ceph S3 in any data center of your choice is around 1/10 of the AWS price.
• ordiel 4y ago
  Having worked for 2 of the largest cloud providers (1 of them beimg the largest) i have to say "The Cloud" just doesnt makes sense (maybe with the exception of cloud storage) yet for most use cases, this including start ups, small and, mid size companies its just way to expensive for the benefits it provides, it moves your hardware acquisitions /maintainance cost to development costs, you just think better/cheaper because that cost comes in small monthly chunks rather than as a single bill, plus you add all security risks either those introduced by the vendor or those introduced by the masive complexity and poor training of the developers which if you want to avoid will have to pay by hiring a developer competent in security for that particular cloud provider
  • manv1 4y ago
    Having worked in 3 startups that were AWS-first, I can say that you've learned the completely wrong lessons from your time at your cloud providers. Building on AWS has provided scale, security, and redundancy at a substantially lower cost than doing any on-prem solution (except for a shitty one strung together with lowendbox machines). The combined AWS bill for the three startups is less than the cost of an F5, even on a non-inflation adjusted basis. The cloud doesn't mean that you can be totally clueless. I've had experience in HA/scalability/redundancy/deployment/development/networking/etc. It means that if you do know what you're doing you can deliver a scalable HA solution at a ridiculously lower price point than a DIY solution using bare iron and colo.
    • ordiel 4y ago
      "The combined bill" during which time period? 1 Month, for sure. What about 1 year? Also did those companies required to provide any training or hiring to achieve that? Because you also need to add that to the cost comparison If you are comparing one month bill agains 1 time purchase (which if is correctly chosen should not happen but once every 10 years at the earliest) for sure it will be cheaper. When it comes down to scalability, development and deployment, you should check your tech stack rather than your infrastructure. Kubernetees and containerization should easily take care of those with on premise hardware while also reducing complexity + you will no longer have to worry for off the chart network transit fees
• betolink 4y ago
  I see both sides of the argument, there is a reason why CERN is not processing their data using EC2 and lambdas.
  • harunurhan 4y ago
    The cost isn't the only reason - CERN started planning its computing grid before AWS was launched. - It's pretty complicated (politics, mission, vision) for CERN to use external proprietary software/hardware for its main functions (they have even started to MS Office like products.) - [cost] CERN is quite different than a small team researchers doing few years research. the scale is enormous and very long lived, like for decades continue - and more... HPC and scientific computing aside, I would have loved to be able to use AWS when I worked there, internal infra for running web apps and services wasn't nearly good & reliable, neither had a wide catalog of services offered.
    • betolink 4y ago
      I think the spirit of the article is to put the cloud in perspective of the organization size and the workload type. There is a sweet spot where the cloud is the only option that makes sense, definitely with variable loads and capacity to basically scale on demand as big as our budget, there is no match for that. However... there are organizations with certain type of workloads that could afford to put infrastructure in place and even with the costs of staffing, energy etc they will save millions in the long run. NASA, CERN etc are some. This is not limited to HPC, the cloud at scale is not cheap either see: https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
  • thamer 4y ago
    The vast majority of researchers don't need anywhere close to the amount of resources that CERN needs. The fact that CERN doesn't use EC2 and lambdas shouldn't be taken as a lesson by anyone who's not operating at their scale. This feels like a similar argument to the one made by people who use Kubernetes to ensure their web app with 100 visitors a day is web scale.
• somesortofthing 4y ago
  The author makes a convincing argument against doing this workload on on-demand instances, but what about spot instances? AWS explicitly calls out scientific computing as a major use case for scientific computing in its training/promotional materials. Given the advertised ~70-90% markdown on spot instance time, it seems like a great option compared to paying almost the same amount as the workstation but not having to pay to buy, maintain, or replace the hardware.
  • lebovic 4y ago
    Author here! Spot instance pricing is better than on-demand, but it doesn't include data transfer, and it's still more expensive than on-prem/Hetzner/etc. Data transfer costs exceed the cost of the instance itself if you're transferring many TB off AWS. For of our more popular AWS instance types I use – a c5a.24xlarge, used for comparison in the post – the cheapest spot price over the past month in us-east-1 was $1.69. That's still $1233.70/mo: above on-prem, colo, or Hetzner pricing. Data transfer is still extremely expensive. That said, for bursty loads that can't be smoothed with a queue, spot instances (or just normal EC2 instances) do make sense! I use them all the time for my computational biology company.
    • timmclean 4y ago
      FWIW, spot prices for c5a.24xlarge in us-east-2b and us-east-2c seem to have been under $0.92/hr for most of the last 3 months. So, assuming some flexibility on the choice of region, that would adjust your estimate to $0.92 / $1.69 * $1233.70/mo = $671.60/mo, which looks a lot more reasonable. Hopefully I did that math right. Data egress prices are definitely still ridiculous, I agree.
      • lebovic 4y ago
        True! It sometimes drops even more, which definitely makes spot instances attractive. The r5.16xlarges had a ~80% discount and a <5% termination rate for while. Also, if all of scientific computing switched to AWS in order to exploit spot instance pricing, I don't think those market dynamics would stay the same. As an aside, I've had trouble created large clusters of high memory instances in us-east-2. They might have increased capacity recently, though.
• jrm4 4y ago
  I imagine what makes this especially hard is you have (at least) three parties in play here: - the people doing the research - the institution's IT services group - the administrator who writes the checks And in my experience, "actual knowledge of what must be done and what it will or could cost" can vary greatly across these three groups; frequently in very unintuitive ways.
  • Fomite 4y ago
    This is the biggest point of friction. I spent the better part of a year trying to get a postdoc admin access to his machine.
• bluedino 4y ago
  We have 500-node cluster at a chemical company, and we've been experimenting with "hybrid-cloud". This allows jobs to use servers with resources we just don't have, or couldn't add fast enough. Storage is a huge issue for us. We have a petabyte of local storage from big name vendor that's bursting at the seams, and expensive to upgrade. A lot of our users leave big files laying around for a large time. Every few months we have to hound everyone to delete old stuff. The other thing that you get with the cloud is there's way more accountability for who's using how much resources. Right now we just let people have access and roam free. Cloud HPC is 5-10x more in cost and the beancounters would shut shit down real quick if the actual costs were divvied up. We also still have a legacy datacenter so in a similar vein, it's hard to say how much not having to deal with physical hardware/networking/power/bandwidth would be worth. Our work is maybe 1% of what that team does.
  • adolph 4y ago
    I can relate to these problems. Cloud brings positive accountability that is difficult to justify onprem. I have some hope that higher level tools for project/data/experiment management (as opposed to a bash prompt and a path) will bring some accountability without stifling flexibility.
• Fomite 4y ago
  One of the aspects not touched on for this is PII/confidential data/HIPAA data, etc. For that, whether it makes sense or not, a lot of universities are moving to AWS, and the infrastructure cost of AWS for what would be a pretty modest server are still considerably less than the cost of complying with the policies and regulations involved in that. Recently at my institution I asked about housing it on premise, and the answer was that IT supports AWS, and if I wanted to do something else, supporting that - as well as the responsibility for a breach - would rest entirely on my shoulders. Not doing that.
• CreRecombinase 4y ago
  These MPI-based scientific computing applications make up a bulk of the compute hours on hpc clusters, but there is a crazy long tail of scientists who have workloads that can’t (or shouldn’t) run on their personal computers. The other option is HPC. This sucks for a ton of reasons, but I think the biggest one is that it’s more or less impossible to set up a persistent service of any kind. So no databases; if you want spark, be ready to spin it up from nothing every day (also no HDFS unless you spin that up in your SLURM job too). This makes getting work done harder but it also means that it makes integrating existing work so much harder because everyone’s workflow involves reinventing everything, and everyone does it in subtly incompatible ways; there are no natural (common) abstraction layers because there are no services.
• deleted 4y ago
  [deleted]
• kortex 4y ago
  What does the landscape look like now for "terraform for bare metal"?. Is ansible/chef still the main name in town? I just wanna netboot some lightweight image, set up some basic network discovery on a control plane, and turn every connected box into a flexible worker bee I can deploy whatever cluster control layer (k8s/nomad) on top of and start slinging containers.
  • nwilkens 4y ago
    I really like this description of how baremetal infrastructure should work, and this is where I think (shameless self promotion) Triton DataCenter[1] plays really well today on-prem. PXE booted lightweight compute nodes with a robust API, including operator portal, user portal, and cli. Keep an eye out for the work we are doing with Triton Linux + K8s. Very lightweight Triton Linux compute node + baremetal k8s deployments on Triton. [1] https://www.tritondatacenter.com https://www.tritondatacenter.com
    • kortex 4y ago
      Thanks! This is pretty sleek! I'm going to have to dust off my homelab and play around with this. What is the stack written in? Looks like a lot of javascript and makefiles from the github side of things but idk if that's the whole kit and caboodle.
      • nwilkens 4y ago
        As you've seen, we're open source -- https://github.com/tritondatacenter https://github.com/tritondatacenter. There are tons of repos.. Mostly Nodejs and some golang/rust!
• hellodanylo 4y ago
  [retracted]
  • Marazan 4y ago
    It says 0.09 per GB on that page.
  • philipkglass 4y ago
    Where do you see that? On your link I see: Data Transfer OUT From Amazon EC2 To Internet First 10 TB / Month $0.09 per GB Next 40 TB / Month $0.085 per GB Next 100 TB / Month $0.07 per GB Greater than 150 TB / Month $0.05 per GB Which means if you transfer out 90 TB in one month, it's $0.09 * 10000 + $0.085 * 40000 + $0.07 * 40000 = $7100.
    • hellodanylo 4y ago
      Sorry, you are right. I need another coffee today.
• lowbloodsugar 4y ago
  >Even 2.5x over building your own infrastructure is significant for a $50M/yr supercomputer. Can’t imagine you are paying public prices on any cloud provider if you have a $50M/yr budget. In addition, if, as the article states, the scientists are ok to wait some considerable time for results, then one can run most, if not all, on spot instances, and that can save 10x right there. If you don’t have $50M/yr there are companies that will move your workload around different AWS regions to get the best price - and will factor in the cost of transferring the data too. I was architect at large scientific company using AWS.
  • lebovic 4y ago
    Author here. I agree that pricing is highly negotiable for any large cloud provider, and there are even (capped) egress fee waivers that you can negotiate as a part of your contract. There's also a place for using AWS; I used it for a smaller DNA sequencing facility, and I use it for my computational biology startup. That said, I'll repeat something that I commented somewhere else: most of scientific computing (by % of compute) happens in a context that still doesn't make sense in AWS. There's often a physical machine within the organization that's creating data (e.g. a DNA sequencer, particle accelerator, etc), and a well-maintained HPC cluster that analyzes that data. Spot instances are still pretty expensive for a steady queue (2x of Hetzer monthly costs, for reference), and you still have to pay AWS data transfer egress costs – which are at least 30x more expensive than a colo or on-prem, if you're saturating a 1 Gbps link. Data transfer to optimize for spot instance pricing becomes prohibitive when your job has 100 TB of raw data.
    • lowbloodsugar 4y ago
      > 100 TB of raw data Surely the raw data is input, so ingress costs, which is free? The problem is if you have large amounts of intermediate data, and you want to transfer that somewhere else to continue analysis. Then it's "expensive". So the logical conclusion is to do all the work on the cloud, so you never have egress costs. That causes anxiety, sure. That said, 100TB costs $1000 to egress, assuming that your $50M/yr covers a Verizon 10GB/s line. These opinions are my own and not those of my employer or former employer.
• timeu 4y ago
  As an HPC sysadmin for 3 research institutes (mostly life sciences & biology) I can't see how cloud HPC system could be any cheaper than an on-prem HPC system especially if I look at the resource efficiency (how much resources were requested vs how much were actually useed) of our users SLURM jobs. Often the users request 100s of GB but only use a fraction of it. In our on-prem HPC system this might decrease utilization (which is not great) but in the case of the cloud this would result in increased computing costs (because bigger VM flavor) which would be probably worse (CapEx vs OpEx) Of course you could argue that the users should do and know better and properly size/measure their resource requirements however most of our users have lab background and are new to computational biology so estimating or even knowing what all the knobs (cores, mem per core, total memory, etc) of the job specification means is hard for them. We try to educate by providing trainings and job efficency reporting however the researchters/users have little incentive to optimize the job requests and are more interested in quick results and turnover which is also understandable (the on-prem HPC system is already payed for). Maybe the cost transparancy of the cloud would force them or rather their group leaders/institute heads to put a focus on this but until you move to the cloud you won't know. Additionally the typical workloads that run on our HPC system are often some badly maintained bioinformatics software or R/perl/pythong throwaway scripts and often enough a typo in the script causes the entire pipeline to fail after days of running on the HPC system and needs to be restarted (maybe even multiple times). Again on the on-prem system you have wasted electricity (bad enough) but in the cloud you have to pay the computing costs of the failed runs. Again cost transparency might force to fix this but the users are not software engineers. One thing that the cloud is really good at, is elasticity and access to new hardware. We have seen for example a shift of workloads from pure CPUs to GPUs. A new CryoEM microscope was installed where the downstream analysis is relying heavily on GPUs, more and more resaerch groups run Alpafold predictions and also NGS analysis is now using GPUs. We have around 100 GPUs and average utlizations has increased to 80-90% and the users are complaining about long waiting/queueing times for their GPU jobs. For this bursting to the cloud would be nice, however GPUs are prohibitively expensive in the cloud unfortunately and the above mentioned caveats regarding job resource efficiencies still apply. One thing that will hurt on-prem HPC systems tough are the increased electricity prices. We are now taking measures to actively save energy (i.e. by powering down idle nodes and powering them up again when jobs are scheduled). As far as I can tell the big cloud providers (AWS, etc) haven't increased the prices yet either because they cover elecriticity cost increase with their profit margins or they are not affected as much because they have better deals with elecricity providers.
• tejtm 4y ago
  Cloud never has made sense for scientific computing. Renting someone else's big computer makes good sense in a business setting where you are not paying for your peak capacity when you are not using it, and you are not losing revenue by underestimating whatever the peak capacity the market happens to dictate. For business, outsourcing compute cost center eliminates both cost and risk for a big win each quarter. Scientists never say, Gee it isn't the holiday season, guess we better scale things back. Instead they will always tend to push whatever compute limit there is, it is kinda in the job description. As for the grant argument, that is letting the tool shape the hand. business-science is not science, we will pay now or pay later.
• avereveard 4y ago
  > Hardware is amortized over five years hardware running 100% won't last five years if hardware is not needed to be running 100% at full steam for five years, you can turn down instances on the cloud and you don't pay anything in 2 years you'll be stuck with the same hardware, while on the cloud you follow cpu evolution as it arrives to the provider all in all the comparison is too high level to be useful
  • icedchai 4y ago
    I think you underestimate how long modern hardware can last. I have 8 to 12 year old PCs running non-stop, in a musty and damp basement.
    • avereveard 4y ago
      they don't just die, thermal paste dry up, fans gum up, gpu will live, but thermal throttling will mean it'll run at, say, 80%.
  • e63f67dd-065b 4y ago
    > hardware running 100% won't last five years Five year is a pretty typical amortisation schedule for HPC hardware. During my sysadmin days, of CPU, memory, cooling, power, storage, and networking, the only things that broke were hard disks and a few cooling fans. Disks were replaced by just grabbing a space and slotting it in, and fans were replaced by, well, swapping them out. Modern CPUs and memory last a very long time. I think I remember seeing Ivy Bridge CPUs running in Hetzner servers in a video they put out, and they're still fine.
    • avereveard 4y ago
      if you expect downtime in the 5 year to replace fan and whatnot, you're not getting 100% of your money/perf back - and I didn't see that in the article. if you have spares, spares need to be in the cost, and value lost to downtime stay minimal. but you have to include spares in the expenses. if you don't have spares, 1-2 day downtime is going to be a decent hit to value.
• didip 4y ago
  No way. I vehemently disagree. When a company reached a certain mass, hardware cost is a factor that is considered but not a big factor. The bigger problems are lost opportunity costs and unnecessary churns. Businesses lose a lot when the product launch is delayed by a year simply because the hardware arrived late or have too many defects (Ask your hardware fulfillment people how many defective RAM and SSD they got per new shipment). Churn can cost the business a lot as well. For example, imagine the model that everyone been using is trained in a Mac Pro under XYZ desk. And then when XYZ quit, s/he never properly backup the code and the model. Bare metal allows for sloppiness that the cloud cannot afford to allow. Accountability and ownership is a lot more apparent in the cloud.
  • esalman 4y ago
    Research labs are not businesses.
• nharada 4y ago
  I'd love to buy my own servers for small-scale (i.e. startup size or research lab size) projects, but it's very hard to be utilizing them 24x7. Does anyone know of open-source software or tools that allow multiple people to timeshare one of these? A big server full of A100s would be awesome, with the ability to reserve the server on specific days.
  • didip 4y ago
    This is just the cloud with extra steps.
  • jpeloquin 4y ago
    > the ability to reserve the server on specific days In an environment where there are not too many users and everyone is cooperative, using Google Calendar to reserve time slots works very well and is very low maintenance. Technical restrictions are needed only when the users can't be trusted to stay out of each other's way.
• xani_ 4y ago
  It always was for load that doesn't allow for autoscaling to save you money; the savings were always from convenience of not having to do ops and pay for ops. Then again a part of ops cost you save is paid again in dev salary that have to deal with AWS stuff instead of just throwing a blob of binaries and letting ops worry about the rest.
• milesward 4y ago
  Article lost me at "they'll just wait longer". Get out of line. Run what you need, when you need it, and get back to the science, the good part!
• jiggawatts 4y ago
  I've just spent the last week investigating cloud compute options for a lab that needs to run bioinformatics / genomics algorithms. First off, the pricing in the article is so disingenuous as to be outright deception. Here is the Spot price for Azure HB120rs_v2, a popular HPC size with 120 AMD EPYC cores and 456 GB of RAM: https://azureprice.net/vm/Standard_HB120rs_v2?tier=spot&currency=USD https://azureprice.net/vm/Standard_HB120rs_v2?tier=spot&curr... This is less than $300/month for 2.5x the compute capacity he's referencing! The author's estimate is $200/month for an on-prem server with just 48 cores. Scaled down to that level, the equivalent in cloud spot pricing would be $120. That's assuming on-prem is 100% utilised and the cloud compute is not auto-scaled. If those assumptions are lifted, the cloud is much cheaper. The cloud makes sense in several other ways also: - Once the data is in cloud storage like S3 or Azure Storage Accounts, sharing it with government departments, universities, or other research institutes is trivial. Just send them a SAS URL and they can probably download it at 1GB/s without killing the Internet link at the source. - Many of these processes have 10 GB inputs that produce about 1 TB of output due to all the intermediate and temporary files. These are often kept for later analysis, but they're of low value and go cold very quickly. Tiered storage in the cloud is very easy to set up and dirt cheap compared to on-prem network attached storage. These blobs can be moved to "Cold" storage within a few days, and then to "Archive" within a month or two at most. - The algorithms improve over time, at which point it would be oh-so-nice to be able to re-run them over the old multi-petabyte data sets. But on-prem, this is an extravagance, and needs a lot of justification. In the cloud, you can just spin up a large pool of Spot instances with a low price cap, and let it chunk through the old data when it can. Unlike on-prem, this can read the old data in much faster, easily up to 30-100 Gbps in my tests. Good luck building a disk array that can stream 100 Gbps and also have good performance for high-priority workloads! - The hardware is evolving much more rapidly than typical enterprise purchase cycles. We have a customer that is about to buy one (1) NVIDIA A100 GPU to use for bioinformatics. In a matter of months, it'll be superseded by the NVIDIA "Hopper" H100 series, which is 7x faster for the same genomics codes. In the cloud, both AWS and Azure will soon have instances with four H100 cards in them. That'll be 28 times faster than one A100 card, making the on-prem purchase obsolete years before the warranty runs out. A couple of years later when then successor to H100 is available in the cloud, these guys will still be using the A100! - The cloud provides lots of peripheral services that are a PitA to set up, secure, and manage locally. For example, EKS or AKS are managed Kubernetes clusters that can be used to efficiently bin-pack HPC compute jobs and restart jobs on Spot instances if they're deallocated. Similarly, Azure CycleCloud provides managed Slurm clusters with auto-scale and spot pricing. For Docker workloads there are managed container registries, and both single-instance and scalable "container apps" that work quite well for one-off batch jobs, Jupyter notebooks, and the like. - In the cloud, it's easy to temporarily spin up a true HPC cluster with 200 Gbps Infiniband and a matching high-performance storage cache. It's like a tiny supercomputer, rented by the hour. On-prem, just buying a single Infiniband switch will set you back more than $30K, and it'll be just the chassis. No cables, SFPs, or host adapters. A full setup is north of $100K. Good luck buying "cheap" storage that can keep up with that network! Etc, etc...
• zmmmmm 4y ago
  A missing elements for me is that with a lot of exploratory scientific work we (often half intentionally) have no idea what we are doing. We can easily run a giant job that uses 100x more compute than expected. Yes you can limit cloud compute resources if you are smart but its much better is the default is that you run out of compute and your job takes longer than you get a 100x bill back from our cloud provider. And if you limited your cloud resources to a fixed amount, didn't you just eliminate half the benefit of using cloud in the first place? Then the problems of data management, transfer and egress are huge. Again the "no idea what we are doing" factor comes into play. If you have a really good idea up front what is going to happen you can plan out a strategy that minimises costs. But if you have no idea at all - genuinely, because this is science and we are doing new things - then you could end up blowing huge amounts of money on unnecessary egress and storage costs. And at the small end we can be talking about experiments run on a shoestring where a few thousand dollars is a big deal. The way I see it, we need everything - powerful individual workstations / laptops for direct analysis, then a tier of fixed HPC style compute for this kind of work that is poorly matched to cloud, and then for specific projects where it makes sense (massive scaleout, exotic hardware needs - GPUs, FPGAs etc) you embrace cloud resources for that.
• epberry 4y ago
  I can tell you that NASA is in the midst of a multi-year effort to move their computing to AWS and that yes.. downloading 324 terabytes of data is very expensive but very soon all this data will just remain in the cloud and be accessed virtually.
• osigurdson 4y ago
  There is a lot of discussion about supercomputers in this article. I don't think public cloud providers can compete easily with traditional super computers because they are built for optimal processing of extremely large scale MPI workloads. Such workloads are not common so I expect that public cloud providers wouldn't bother optimizing for this niche use case (though I know they all have offerings). Also when you are only optimizing for a single variable (i.e. speed), you can make design choices that would be impossible to make in a more general situation. Of course, not all scientific computing workloads require a traditional supercomputer. In fact, I suspect most do not.
• wenbin 4y ago
  A trend I've seen on HN over past few years is that people love showing off how they are able to save money by spending more of their own time, especially on infra/cloud things - if you calculate your own hourly rate correctly, it's oftentimes more costly to DIY than outsourcing to experts (e.g., managed cloud).
• jupp0r 4y ago
  Having worked in the high performance computing field and in cloud hosted commercial applications, I can agree with the article but for entirely different reasons. The reason why some scientific computing shouldn't be done on AWS has to do with networking and latency between compute nodes. Supercomputers often use specialized networking hardware to get single digit microsecond latencies for data transfer between compute nodes and much higher network bandwidth than what you would normally find between EC2 nodes. This allows simulations to efficiently operate on really large data sets that span hundreds or thousands of nodes. The network topology between these nodes is often denser than a tree (think a 2D or 3D grid topology) and offers shorter paths between nodes. All of this allows you to run code that you can't run in AWS unless it fits on one computer only. It's also way more expensive than clusters of commodity hardware. For problems that are trivially parallelizable without much communication between nodes - I don't think that most universities can actually operate those cheaper than renting them from cloud computing services. A lot of these calculations don't take the staff to operate data centers, the cost of the building itself or the opportunity cost of using lots of space for this purpose vs something else into account. Economics of scale also kick in here. It's way cheaper per computer for AWS to admin a data center because they do this for orders of magnitude bigger data centers than your typical university.
  • shadycuz 4y ago
    https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placemen...
    • jupp0r 4y ago
      This is using IP over ethernet in a tree, which has an order of magnitude more latency and less bandwidth than what you would see in a supercomputer, even if all your instances end up in the same rack.
• Helmut10001 4y ago
  I almost got a Tenure Track position at a Data Science Faculty in Virginia and I think them not having a HPC was the single issue that blew this move (from both sides). During interviews, I was asking the dean how they set up their HPC - turned out, they hadn't. I then asked a Professor in the next review round how they teach their students without a HPC: > "I buy all resources on AWS - it's painfull because I have to contact AWS almost monthly for accidental over-billing, but we don't have a solution". All of this made me really sceptical, since coming from a big University in Germany, we have unlimited HPC resources for free. I have 16 VMs, the biggest one 125 GB Memory, I can set those up or move around how I want. No space limitations - in need 10 TB of space for 3 month? Open a service ticket, 3 hours later it's available. Ports need to be opened worldwide to the web? No problem. Need a Jupyter Hub Cluster on Kubernetes? Here you go. This has really improved my work (quality, performance, and convenience) so much. I was once coordinator of a research project where we had 30k EUR left and didn't know what to do with it. I contacted our HPC and asked if they want the money - answer: "30k really isn't worth the effort, we don't know what to do with it atm."
  • m_mueller 4y ago
    I second this, would also ring my alarm bells in an interview. Back as a master student I went from a top 10 research university that didn't have a strong HPC infra at the time (ETH Zurich) to a top 100 one that did (Tokyo Tech), at least as a guest. Difference was staggering - all students just got handed a login on their big (top 100) cluster to be able to work with the actual hardware, even if limited to a couple of nodes. This much increased the likelihood that someone would be able to come up with an interesting project just together with their academic supervisor (no huge group projects with tons of extra funding needed).
• Areading314 4y ago
  AWS is one tool but its a lot like the proprietary computing ecosystems that have existed for a long time (remember the Micro$oft days?). It offers convenience in return for lock in and very high margins. Theres no clear answer but its definitely not a clear cut decision where AWS is guaranteed to save money. There are 2 major costs that are overlooked by the in-house crowd, which are operational maintenance cost (an increasingly rare and expensive skillset), and also the cost of downtime -- how much does it cost you when your team of data scientists are blocked because of a failed OS update etc. That being said, hiring competent people to maintain AWS properly isn't cheap either -- and it is quite easy to start running up very wasteful AWS bills on things you don't need. As always there's a tradeoff -- the key is to choose a path and to execute it well.
• fulafel 4y ago
  Most of the comments get fixated on the most and least expensive options in this (AWS where you pay through the nose / own DC where you'll get bad service from your institution and have to fight with hw procurement stuff). What about the middle ones presented, the more reasonably priced cloud/rental server providers?
• stuntkite 4y ago
  If you pay $500 to form an LLC with Stripe Atlas you get $10,000 worth of AWS credits that can be used any way you want. It's a pretty solid way to cost effectively do scientific computing even if you need like five companies. If a policy change is made because of this comment. I'm sorry. For sure let me know though. I'll put it on my resume.
• myuzio 4y ago
  I'm surprised no-one mentioned Amazon Lightsail by now. Anyway, yes, AWS can be super expensive especially if you don't know what you're doing, regardless of type of processing.
• alpineidyll3 4y ago
  We'll put. My company is current suffering through this...
• pacerier 4y ago
  Very odd article (to be written by a scientist)—Shouldn't it be comparing with GCE? Doesn't make sense to compare on cost against AWS instead of GCP, except.. for wow numbers and moar clicks. "Linkedin doesn't make sense for connecting with friends".