27 ms·

Is a billion dollars worth of server lying on the ground?

• KingOfCoders 6y ago
  No one has been fired for choosing AWS.
• deleted 6y ago
  [deleted]
• mst 6y ago
  With respect to varying EBS performance, I remember somebody once mentioning that they always provisioned 10 EBS volumes at once, wrote garbage to them all for 24h or so to warm them up, then benchmarked them and deprovisioned all but the fastest one. How useful this is in practice isn't something I've tried to measure myself, but given the performance variance people report it might not be a bad idea to try for yourself.
  • vegardx 6y ago
    Seems like a strange approach. It's easier to just continuously benchmark performance and kill machines that don't perform as expected. It's very easy to automate. The article also doesn't mention spot pricing and per-second billing. Being able to burst up to hundreds of machines in minutes is super nice. I did some back of the envelope calculations for some build servers and it came out favorably for $cloud providers. Having idle machines also cost money. For a highly optimized and predictable work load I'm sure providers like OVH can provide better bang for bucks. But that also assumes you don't use any of the cost reduction methods that are available with $cloud, like using spot pricing for variable load and reserved capacity for base load.
    • twic 6y ago
      > It's easier to just continuously benchmark performance and kill machines that don't perform as expected. It's very easy to automate. It's also work you simply don't have to do if you rent hardware.
      • cthalupa 6y ago
        You don't think there is variance in performance when you rent hardware? You've never had an HDD or SSD that underperformed vs. others of the same type or model? You've never had a stick of RAM that underperformed, or threw lots of CEs? I'll be frank: If you haven't seen performance variance with physical hardware, you either have been incredibly lucky, not paying attention, or not working at a very large scale.
        • twic 6y ago
          Ah, that delightful cloud propaganda move, where you assert that because some undesirable thing happens to both VMs and physical machines, that they're the same, elegantly glossing over the orders of magnitude difference in severity.
          • cthalupa 6y ago
            I'm not sure how it's "cloud propaganda" to say that performance variance is a very real thing in response to someone making the claim that it doesn't exist when you rent servers.
  • viraptor 6y ago
    Historically EBS volumes were not initialised. The space was allocated, but until you touched the right blocks the first access time was terrible. This is not the case anymore with modern classes. Also neighbours will change over time so testing for 24h and committing to a volume long-term doesn't sound optional.
• aliceryhl 6y ago
  I run my personal website and other small projects on VPSs I bought from OVH, and they're really cheap and work very well. Can recommend.
  • ls65536 6y ago
    I agree. They're really great if you need well-priced, no-hassle infrastructure. As others have already mentioned, they may not have some of the higher-level features that AWS comes with, but if you don't need these, then there's no reason to be paying more for it. I'm really fascinated by their approach to building datacenters, which seems to include taking over disused industrial sites and converting them for datacenter use. For example, their datacenter just outside of Montreal is on the site of an old aluminum smelter [0]. In this case, I'm sure the proximity to plentiful and cheap hydroelectric power nearby (as would have been beneficial for aluminum smelting in the past) was a major factor in the choice of location as well. [0] https://baxtel.com/data-center/ovh-beauharnois-quebec-bhs https://baxtel.com/data-center/ovh-beauharnois-quebec-bhs
• mlthoughts2018 6y ago
  Something the article misses is complementary services. A lot of companies don’t care at all about X vs 2X for server cost, but if you lack certain features around say managed Spark job execution or serverless container deployments, it’s a total deal-breaker. AWS, Azure and GCP tend to have the widest coverage of complementary services, along with all the other stuff like volume discounts and credits. I believe in many cases it could be perfectly rational to pay 2X more on just the server portion. I also think support is another big issue, but GCP makes me second-guess myself, since they are the third largest provider but their support system is (literally) to just say “fuck you, read our upsell-laden docs.” A good example is the way the GKE SLA page mentions a bunch of beta features of kubernetes will void your SLA credits, when in reality Anthos is not remotely close to being ready for production use by most teams and they have no choice but to rely on beta features in GKE. Multi-cluster ingress is a good example of this - “just switch to Anthos” as a proposed solution is literally equivalent to saying, “fuck you.”
• oriesdan 6y ago
  I think the main reason they can afford pricing their services that high is because of peer pressure - probably itself the result of clever marketing, or that would be a really happy coincidence. I've worked in many startups now, several of them where I was the first (and for a while, only) developer and had to decide on the infrastructure. Each time I was going with OVH, and each time the CEO was trying to push for moving to AWS instead, despite having no clue what the difference may be. Their problem was that "startups are supposed to use AWS". They were having impostor syndrome. One would come to tell me every month or so how "all his friends use AWS, and they say it's very good". An other one was afraid what possible investors may say when he tells them we're not on AWS. If people will pay overpriced services to be with the cool kids, why bother competing on price?
  • KingOfCoders 6y ago
    CEO changes his opinion when money runs out and a new CFO comes in to fix the costs - at least from my experience.
    • srtjstjsj 6y ago
      If you need a CFO to look at your IT bill and cut costs, your problem is likely BS title inflation crowding out real work.
  • Cthulhu_ 6y ago
    I've worked as a contractor for a CEO for two companies, in both he pushed for a full migration to AWS. Would not be surprised if he got a kickback from AWS. Amazon is pushing AWS pretty hard in the C-level, I don't know if you've ever followed one of their certifications or landing pages, but they do their marketing really well. Anyway, I do think a platform like App Engine / Beanstalk and other quick / easy / no setup deployment tools have a benefit, if you're not good at setting up servers.
    • WrtCdEvrydy 6y ago
      AWS allows you to shift your costs from CapEx to OpEx. Companies with low CapEx are valued higher since "theoretically" you could remove that bill by moving to another provider. Financial Engineering is just another part of software engineering and the cloud enables it.
• baybal2 6y ago
  So... The news is that AWS is overpriced, and overpriced by quite a lot. This is not news.
• jdub 6y ago
  Are there tens of regions around the world? Are there multiple AZs within a region that are single-digit milliseconds apart? Can I autoscale? Is there a managed load balancer? Do I have to manage my own database instances and availability? Do I have access to hundreds of other services for things that I don't want to spend time managing myself? Aren't we years beyond the 1:1 cost comparison for a single server argument?
  • legulere 6y ago
    The question is wether that is worth the lock in.
    • owenmarshall 6y ago
      I worked at a job that had a multi year contract with a hosting company. We paid eight figures annually to lease MIPS on a mainframe. That’s vendor lock in. AWS “lock in” isn’t, it’s “I could terminate for convenience any day I wanted to, but the ROI isn’t there”.
      • legulere 6y ago
        With cloud providers either you use their proprietary services and thus end up with code that runs only in their cloud. Just like with MIPS code that runs only on MIPS. Only that you probably can more easily port code to another CPU architecture.
        • Cullinet 6y ago
          Not MIPS the RISC CPU arch / vendor and embedded survivor, but IBM MIPS, the only feature added to brilliant hardware designed to turn customers away, e.g. : "Turning our attention back to IBM’s announcement, this new server offers five hardware models and well over 250+ unique software capacity settings, providing a highly granular and scalable system. The base single-engine speed of 98 MIPS is found on the A01; the same full speed unit (Z01) climbs to 1761 MIPs, up from 1570 MIPs on the prior generation" From : https://www.evolvingsol.com/2020/04/14/ibmz15-mainframe/ https://www.evolvingsol.com/2020/04/14/ibmz15-mainframe/
• viraptor 6y ago
  He kind of touches on this in point IV, but never mentioned specifically: the cost comparison is not AWS vs ovh. It's fragment-of-EC2 vs ovh. If you need a predefined small number of VMs and no other functionality, it would be silly to go with AWS. But on the other hand, if you want a set of servers of a given class spawning on demand, with traffic coming in via load balancers, with integrated certificate and DNS management, with programmable lifecycle hooks, with integrated authn/authz, with full audit logs and account management, with configurable private networking to other services, etc. etc. ... You'll pay more than the price difference for someone to implement all of that from scratch. Compared to this many other points sound like conspiracy theories. Meanwhile people either don't know they can do better, or use AWS because they want more features.
  • KingOfCoders 6y ago
    Many people use AWS because everyone uses AWS. Many of my clients have no need for AWS but still use it, at least until the VC money runs out. Usually then I have to go first, then servers are moving to somewhere cheaper when a new CFO comes in.
    • lvh 6y ago
      It's possible that "many" firms do this, but given AWS' growth numbers that would imply they don't really spend meaningful amounts to begin with. Counterpoint: I've seen a great many false economies with people trying to go on-prem and do alternative hosting because they don't think the AWS premium for e.g. GPU instances is worth it. I don't think that has generally worked out well.
      • KingOfCoders 6y ago
        "It's possible that "many" firms do this, but given AWS' growth numbers that would imply they don't really spend meaningful amounts to begin with." I could spend a million EUR a year on AWS without the need for most of the services of AWS? But it YMMV and 1M EUR/y is not meaningful, perhaps we differ there.
      • 6y ago
• tyingq 6y ago
  I love OVH, and have several servers there. But it's not AWS. They don't have the same ecosystem of services. Consider, IAM, for example. Premium pricing for a server that has access to that seems normal to me. On the other hand, he didn't include egress charges. That makes the gap even wider for many use cases.
• ndomer 6y ago
  One big thing not mentioned here: the massive collection of managed services AWS gives you at no cost. The biggest example is probably IAM, which makes it easy to control which servers can take what actions on which other services. And can also be integrated directly into external tools like Vault. Want to use service discovery? No need to set up Consul/Istio/etc., just use the managed service. Same with load balancers, and VPCs, and SSM, and route53, etc. Sure, in 2020 none of those services are that hard to replicate pieces of, and open source tools abound. But setting up those tools all takes time. Only other nit is that the article makes it sound like Terraform and IaC is meant to abstract away AWS vs GCP such that one terraform module _could_ be deployed to either just by changing some string value from "AWS" to "GCP". I don't believe any serious (and good) efforts are being made in that space, and you won't find any popular modules like that on https://registry.terraform.io/ https://registry.terraform.io/.
  • smolder 6y ago
    > At no cost That's so ridiculous I'm not sure how to reply or even interpret your comment.
    • deegles 6y ago
      Maybe they mean that if you're already using some services, you don't need to implement another in the same way you would when self-hosting.
    • throwaway894345 6y ago
      I can't tell if your comment is a particularly rude way of picking the nit that "nothing is free, the cost (e.g., of IAM) is built into other services" or if you really find it absurd that the nominal price of many AWS services is $0 or something else entirely.
      • freeone3000 6y ago
        The nominal price of $0 absolutely is absurd, since every individual service has a separate pricing chart down to the ELB.
• kuon 6y ago
  With the price of pro fiber (redundant with SLA) I recently moved some apps back to our own servers in house. This did cut the price down dramatically. I would not recommend it for super critical apps (except if you have your own state of the art data center, but I am not speaking about that), but having 5-10 servers in a secure cabinet will give you infinite flexibility for little cost. We pay around 5000$/y to keep running about 50k$ worth of hardware. It's 10-20 times less that what we would pay on AWS. Of course this approach has limits, but it can work in some scenarios and should not be "de facto" dismissed.
  • wrkronmiller 6y ago
    How do you deal with data egress costs w/r/t AWS though?
    • kuon 6y ago
      We don't use AWS.
  • ronyfadel 6y ago
    I get it that you’re indie and running web services from home? If you’re willing to share, I’d love to see what kind of apps (that aren’t “super critical”) one can from a cabinet from home.
    • cosmie 6y ago
      "in house" doesn't necessarily mean in a residential home; it refers to "on premise" more generally. A previous company of mine did the same thing - they converted a maintenance closet into a server closet. Even with renovation costs to improve ventilation and electrical load to support the use case, it worked out substantially cheaper than cloud hosting. A few things we ran on it: - A large data infrastructure. We had an EDI[1] side of the business, and egress bandwidth costs would have eaten the product margins and then some. A lot of traditional EDI happens over (S)FTP, and customers only periodically accessed the system (daily/weekly/monthly/quarterly, depending on the customer). Most enterprise EDI systems have retry logic built in, so minor amounts of downtime weren't relevant. If the downtime were for more than several hours, we could cut over to a cloud-based backup (which was fairly cheap to maintain, since ingress bandwidth is generally free). - Our analytics environment. In addition to standard reporting, we also used our analytics toolset to create "data utilities", allowing powerusers on the business teams to be able to access bulk datasets for their own downstream processes. The bandwidth usage would have again been cost prohibitive to cloud-host, plus the data was co-located on-premise as well. - Our B2B website. Traffic volumes were minimal, and it was primarily a static website. So hosting it behind Cloudflare added enough uptime guarantees for our needs. - Dev environments. Both dev environments for all of the above, as well as something similar to LocalStack[2] (it's been a while, not sure if that was the tool used or something else) to mimic our AWS environment For all of those, less than a day of downtime had negligible financial impact. And downtime more than a day was a non-issue, as we had off-site fail-over plans to handle contingencies longer than that. We also operated several services and applications where every single minute of downtime created a visible impact on our financials. Those were all hosted on AWS, and architected with redundant and fault-tolerance built in. [1] https://en.wikipedia.org/wiki/Electronic_data_interchange https://en.wikipedia.org/wiki/Electronic_data_interchange [2] https://localstack.cloud/ https://localstack.cloud/
• drchiu 6y ago
  We tried running things on AWS and digital ocean. Yes, AWS offers a lot of extra value with all the other services. But at the same time, we had a relatively simple application. We ended up managing our own sets of dedicated servers for a fraction of the cost. Peformance is way higher. In the end, the cost difference was just too big to justify it when we were trying to be scrappy. We also realized that all the extra features offered weren’t being utilized as our infrastructure requirements were simple.
  • XCSme 6y ago
    So for you DO was cheaper, it's funny that I'm actually looking to move from DO to OVH, as DO pricing is around 50% more expensive than OVH. My main concern right now is the worse UI and the lack of more pre-built images that allow you to spin-up VPSs quicker.
    • drchiu 6y ago
      Nope, it's OVH for us as well as this point. We have about 8 or 9 dedicated servers at OVH right now. Things we've noticed: - Yes, UI sucks - No autobilling - Reboot times are slow (as the actual machines are reboooted, not a VPS) - You need to upgrade to higher bandwidths for great speeds - Some routing problems, very rarely - You really need to know what your server needs are 6-12 months in advance - Some product offerings way too expensive (eg. Loadbalancer) Great things: - Price. Net-net, still cheaper to buy 2-3 dedicated servers than a single beefy VPS - A lot of sales, like every month - You get the full resources of the whole server - Reasonable support for network or hardware related issues - Unlimited traffic (at least you don't need to worry about normal usage overages) - DDoS protection Other thoughts: In terms of performance, dedi's work great. I've found their OVH Public Cloud instances (not the cheap one, but the more expensive ones) have not as great performance. Might as well buy a dedicated. The only pro is that you only pay for the time you use.
      • XCSme 6y ago
        Thanks a lot for the information! I was mostly looking to get some VPSs from them, the $10.58 2vCPU, 4GB RAM ones to replace the current DO $15/mo 2vCPU 2GB RAM I'm using, as I would get better specs for a cheaper price. It's a bit hard to tell without actual benchmarks if the performance will be better or worse than DO.
• sneak 6y ago
  Few large organizations pay the on-demand price that this article uses for comparison. By reserving the instance, or using spot, the costs of these instances come way, way down, usually 50-80%. I’m not trying to defend AWS here, but for an accurate comparison, it’s best to use the numbers that people are actually paying in practice.
  • sokoloff 6y ago
    The article did reference the (1-year, upfront) reserved instance pricing. It quoted a price 17.5% higher than what I see in US-East2 (Ohio), but they didn't ignore it entirely. My guess is that most people who would be buying such a server would be using a 3-year RI, which is (ballpark) "buy two years of RI, get the third year free".
• patchtopic 6y ago
  Now go stick some Dell servers in a colocation joint and run the numbers again :-)
• jwr 6y ago
  When determining what to use for development of my SaaS, I did a comparison of what you actually get from providers. The full article is at https://jan.rychter.com/enblog/cloud-server-cpu-performance-comparison-2019-12-12 https://jan.rychter.com/enblog/cloud-server-cpu-performance-... My takeaways were that many cloud provider offerings make no sense whatsoever, and that Xeon processors are mostly great if you are a cloud provider and want to offer overbooked "vCPUs". I haven't tested those specific setups, but I strongly suspect a dedicated server from OVH is much faster than a 4.16xlarge from AWS.
  • ti_ranger 6y ago
    > When determining what to use for development of my SaaS, I did a comparison of what you actually get from providers. The full article is at https://jan.rychter.com/enblog/cloud-server-cpu-performance- https://jan.rychter.com/enblog/cloud-server-cpu-performance-... Your results (e.g. that z1d.xlarge with 4 vCPUs is only 10% slower than z1d.2xlarge with 8 vCPUs) shows that the "performance" you were testing was disk IO throughput (probably dominated by disk latency), not vCPUs. > My takeaways were that many cloud provider offerings make no sense whatsoever, and that Xeon processors are mostly great if you are a cloud provider and want to offer overbooked "vCPUs". > I haven't tested those specific setups, but I strongly suspect a dedicated server from OVH is much faster than a 4.16xlarge from AWS. You seem to be implying that AWS/EC2 does CPU over-provisioning on all instance types; this is incorrect, only T-family instance types use CPU over-provisioning.
    • jwr 6y ago
      > the "performance" you were testing was disk IO throughput In part, yes, but not entirely. I was very clear that my load isn't embarrassingly parallel, so it is not expected to scale linearly with the number of processors. > You seem to be implying that AWS/EC2 does CPU over-provisioning on all instance types; this is incorrect, only T-family instance types use CPU over-provisioning. If you think you are getting a Xeon core when paying for a "vCPU" at AWS, I have a bridge to sell you.
• arpinum 6y ago
  These aren't the same product category. OVH sells a specific server, AWS is selling managed compute and storage. I won't buy OVH because the failure scenarios and recovery are incompatible with my needs without a lot of extra work A better comparison would be Rackspace and AWS.
• Semaphor 6y ago
  > and a somewhat worst machine from AWS I’m not a native speaker, but I see this usage of "worst" pretty often. Shouldn’t it be "worse"? Is this a mistake by someone or is this an actual construct I simply don’t know about?
  • lvh 6y ago
    Yep, that's not grammatical.
• prussian 6y ago
  Like I said on another site: ... people know AWS and know how to be productive with the services and frameworks for AWS. that alone is a figure hard to quantify. Sure I could save money bringing all the servers back internally or using cheaper datacenters, but I worked at a company that worked that way. You end up doing a lot of busy work chucking bad drives, making tickets to the infrastructure group and waiting for the UNIX Admin group to add more storage to your server. WIth AWS I can reasonably assume I can spin up as many c5.12xlarge machines as I want, whenever I want with whatever extras I want. It costs an 1/8 of a million a year, roughly. I see that 1/8 of a million that cuts out a lot of busy work I don’t care about doing and an 1/8 of a million that simplifies finding people to do the remaining work I don’t care about doing. The author says money wasted, I see it as money spent so i don’t have to care, and not caring is something I like; hell it isn’t even my money.
  • varispeed 6y ago
    I had this mindset for a long time that we would save money running our own infrastructure. I remember talking days with our CTO about getting dedicated servers and spinning up one of the Citrix or VMWare offerings. I was sure that we could run our backend on one dedicated server for a fraction of the equivalent on AWS. I based my assumptions on running my own infrastructure for years without issues. However, once we started growing I understood that adding resources, changing network policies, spinning test environments would be extremely difficult and would stall our growth. I am so glad they didn't listen to me.
    • crisper78 6y ago
      You just didn't have anyone skilled enough to automate those things, or you couldn't manage that. If you are colo you aren't swapping disks/network your colo provider is. Im not advocating for doing it ALL yourself, making network cables does save money, but its impractical. Just because you couldn't manage the whole thing while having the ear of the CTO, just means you failed it doesn't mean it doesn't work for lots of other companies that make lots of money.
• blunte 6y ago
  Comparisons of providers is about like comparisons of programming languages - the best choice will depend very much on the circumstance, and likely there will still be at least two possible choices which cannot be accurately differentiated based on overall cost. There are just so many variables...
• deleted 6y ago
  [deleted]
• alex_young 6y ago
  Interesting, but somewhat incomplete analysis IMO. If you are running a billion + dollar budget for compute spending, you’re surely going to negotiate pricing with your vendor, and while that won’t bring things to parity, it will bring them much closer together. If you are spending this kind of money, you’re likely doing a lot to get your workloads into some reasonably geographically aligned areas, and if you’re peering with other services, they tend to be running on AWS, which means choosing another provider can significantly increase those latency costs. While we’re on the subject of bandwidth, what do you think data transfer pricing will be between your other cloud provider and this high compute instance you picked up on the cheap? Odds are they will more than negate your cost savings, and again at a latency cost. Let’s say you really want to save huge amounts of money though. The easy answer is probably in moving to a spot instance targeted architecture where you can typically buy the same server from AWS for less than 20 percent of the on demand price. You can always fall back to on demand when they are not available.
• deleted 6y ago
  [deleted]
• prohor 6y ago
  GCP also has sustained-usage discounts, which are very convenient, as even without reservation as in AWS, you get up to 30% discount if you have more constant usage, but you still have the full flexibility of on-demand. When it comes to pricing comparisons of cloud see Cloudorado: https://www.cloudorado.com/ https://www.cloudorado.com/
• karlkatzke 6y ago
  Just migrated an ecommerce consulting client from a major VPS provider in the US to AWS. There's two main reasons. One is that the traffic to this client is heavily driven by ad spend, and the site fell on it's face hard this past year any time a big spike got sent our way by a single or combination of ad vendors (which we can't really control in fine enough detail) -- facebook, by the way, is by far the worst about this. They can and will cause a thundering herd. The second reason is that there's a bunch of scale up/down to services in AWS if you're doing it right. You're not buying a .16xlarge server in AWS to host your ecommerce site, that would be stupid, you don't need that 24x7. You're paying for a pair of .xlarge servers at a Reserved Instance rate, which is half the published rate. When you need to, your instance count (and your aws bill) goes up. We couldn't do that at the VPS host we were on, so we kicked them to the curb. With AWS, we can handle the load spikes -AND- the total lower bill for a year came out much lower.
  • disgruntledphd2 6y ago
    >facebook, by the way, is by far the worst about this. They can and will cause a thundering herd. From a marketer's perspective, (as long as the site doesn't go down), this is actually facebook doing their job.
• api 6y ago
  Bare metal destroys AWS and other big managed clouds and even smaller cloud companies like Digital Ocean if all you need is compute, storage, and bandwidth and want to manage it yourself. Bandwidth is even more extreme than CPU/RAM. There are high bandwidth bare metal vendors that basically sell like rack vendors by capacity, not quantity, and offer bandwidth at stupidly cheap rates compared to any cloud. ZeroTier uses these for root server clusters and get tens of gigabits for <$1000/server... and you can actually use that and it's fine. Vendors include datapacket.com, reliablesite.net, fdcservers.net, and others. Totally no frills though. They provision it, turn it on, and give you access. Also their TOS may have specific prohibitions, so read it. The problem is that you have to manage it all yourself, which can be challenging and distracting, and these are no frills services with no add-ons like managed or specialized databases or S3. That being said the cost savings are so extreme that you should consider it. It's ultimately a spreadsheet decision. Compare cost of hosting vs cost of DIY hosting plus labor for your specific application. Some apps benefit more from managed cloud than others. There's also the multi-cloud option. You could host things that benefit from AWS or GCP there, and host the bandwidth or CPU-intensive stuff on bare metal.
  • smilliken 6y ago
    Aside: thank you for ZeroTier. Fantastic software that I've been recommending to everyone.
• jonstewart 6y ago
  As a developer at a big company, if I try to buy a server, then I have to deal with my IT department. I don’t get to buy what I want, have to deal with particular overpriced vendors, and the process slows to a crawl. If I want to use AWS, however, I get instant gratification. And of course I can experiment with different models and then refine into cheaper service mixes as scale increases. As base load emerges, I can use reserved instance pricing and shift that onto our capital budget. And of course I can control everything with code using AWS CDK. At a smaller company where I was able to control the IT process more, it might be worthwhile. But the speed and optionality I get with AWS is vastly more cost effective than dealing with legacy enterprise IT processes.
  • MrBuddyCasino 6y ago
    > As a developer at a big company, if I try to buy a server, then I have to deal with my IT department. This. AWS is a way to put control back into the hands of developers and work around the internal IT mafia. People are quick to forget how that was like. For startups, AWS is either a really dumb idea or a valuable tool to go faster than the competition, depending on funding and growth rate.
    • user5994461 6y ago
      >>> People are quick to forget how that was like. I would postulate that the author of the article has no idea what's it like. The stories are happening in small companies that are too small to have an IT mafia or any sort of procurement process.
  • someguydave 6y ago
    This is probably the real reason for bundled cloud provider popularity. It’s way easier for competent engineers to cut out the brigade of uninformed and cheap managers of the IT budget when you distill your specs into an AWS credit card charge. For a small startup of competent people this is unneeded, but those companies typically succeed or fail into mediocrity eventually.
    • 6y ago
• andrenotgiant 6y ago
  > On the flip side of the coin, there are server providers such as digital oceans, GC, and Azure that can be more expensive than AWS. What gave the OP the impression that Digital Ocean is more expensive than AWS? AFAIK there is no configuration of Droplets vs EC2 where Digital Ocean is more expensive. In most cases, especially with any outbound bandwidth, it is cheaper by a large amount.
  • hansvm 6y ago
    Off-topic: Bandwidth is still pretty costly at Digital Ocean (DO). You can rent an entire droplet along with the additional bandwidth that comes with said droplet for 50% less than just buying the extra bandwidth (and even with that 50% discount it's still prohibitive for bandwidth-intensive apps). More off-topic: I still generally like and use DO, but I dislike the policy of billing for one thing (e.g. 1vCPU) and adding hidden, fuzzy terms and conditions like "don't consume excessive CPU cycles." IMO the marketing ought to make it crystal clear that you're allowed to _burst_ to 100% use and also put in a well-defined threshold on any other resource limits like total CPU cycles so that people can plan accordingly and not be hit with surprise outages. It's similar to the complaint people have with Comcast selling XX00 Mbps packages and then tucking a miniscule bandwidth cap in the terms and conditions. That's potentially a fine policy, but it's extremely misleading to sell one thing and use fine print to shape it into a completely different offering.
• vaccinator 6y ago
  Why not just buy the hardware and save even more then?
  • speedgoose 6y ago
    It depends on your needs. Buying hardware still make sense in some scenarios where quality of service is not a thing, you have space and some time.
    • vaccinator 6y ago
      yeah if you have a little time, you can save a lot
• iampims 6y ago
  I’m surprised no one has mentioned the number of data centers that AWS offers. Good luck using OVH anywhere but France/US east coast.
  • bogwog 6y ago
    If all you need is a few VMs, then there's no real reason you have to be stuck on a single provider. If OVH only has servers on the east coast, then you find another provider for the west coast. I've never used OVH, but I doubt creating an account with them (and other providers) is difficult. Shopping around is the only way to get the best prices. Maybe it's mildly annoying to have to deal with multiple hosting accounts, but that's a small price to pay for potentially huge savings, IMO.
  • hbogert 6y ago
    Why? They have 8 PoPs stretching from US west coast to Singapore and Australia. That seems pretty okay.
• treis 6y ago
  The article and the other posts in this thread miss the killer feature. Data security via backups. If I put my data in a manager DB and use S3 or equivalent I am more or less guaranteed to never lose data. Running bare metal in OVH can't provide that guarantee. The security of that data depends on the thoroughness and correctness of my back up policy. And those are almost guaranteed to be much worse than what AWS et al provide.
• Fumtumi 6y ago
  You actually get encryption out of the box on GCP. Your data is encrypted also at rest. Do not compare apples with oranges, or compare them more fair. Go to whatever provider you want. Make sure you know why you have choosen AWS or someone else. For a lot of companies, the quality they get from GCP might be overkill, but don't get me wrong, infrastructure cost are often enough, in comparision to how critical they are, very cheap. It might just be that it is easier for you to get another company to manage your AWS Account while they don't know anything about some other cloud provider and the additional cost is then just worth it.
• emrehan 6y ago
  What if there were Airbnb for servers, where you could rent your spare computing resources in a marketplace? Many non-confidental processes could run on some other people's computers.
  • coding_unit_1 6y ago
    AWS spot pricing meets SETI@home?
• rektide 6y ago
  A lot of people pointing out the complementary services in this thread. Another place that I hope Kubernetes creates a consistent operational environment, with wide service offerings, such that we aren't so strongly reliant on Big Cloud forever.
• known 6y ago
  Capex + Opex = $32,000 in AWS https://archive.is/rav98 https://archive.is/rav98
• nraynaud 6y ago
  A pet peeve of mine is also the "bullshit cloud". For example OnShape is basically running a CAD software on a virtual machine. Fusion 360 is saving files on the networks or launching batch jobs on a farm, 1990 banking style. The cloudiness provides very little value to the user. Whereas computing the fillets concurrently on a bunch of server and the client and accepting the fastest answer would be useful, instead of locking the UI for 2 minutes, same for toolpath generation.
• tormeh 6y ago
  Has anyone tried the Scaleway Elements? Looks pretty feature-complete (hosted DB, Kubernetes, object storage). Just wondering if there's a reason it's not talked about, i.e. is it too good to be true?
• NoOneNew 6y ago
  Um, the real reason CFOs go for cloudserver solutions is tax reasons. The tax breaks for "leasing" instead of owning, are a cheap way of an exec looking good to a board (plus it's easier to do creative accounting). I promise you, this accounts for easily 70% of equipment leasing/renting agreements, no matter the industry. Construction, printing, food industry, etc. They get a full tax break on the lease and on the service agreement tied into it. This looks good in the short term, but is obviously shit in the long term. I've been apart of too many arguments with clients about this. The person who signs the cheques always mentions the tax incentives and the lower initial yearly costs. But then they lose their ass 5 years down the line. You can argue tech reasons all you want, but it's ego driven money that drives the cloud industry. Wile I agree mostly with the author... wtf is all the hate with C# I see from people? lol "whatever the heck people use C# for". This pops up so often. C# is the F-350, Catipillar, Kubota of the tech world. Yea, it ain't sexy, but it's meant to just get work done. When it works, no one notices. But when people screw with it, yea, it's easy to laugh at like the videos of crane construction fails (mostly operator error is to blame). While other tech might be the Bentley and Ferrari, they're either all looks and no muscle or catch on fire when something minor goes wrong (looking at you MongoDB :P).
  • disgruntledphd2 6y ago
    I've assumed this for a while, because so many companies appear to be doing this - do you have any links to the tax savings as I'd like to understand this in more detail?
    • rbanffy 6y ago
      If you don't need what cloud providers offer you, from hosted databases, access control, blue-green deployment, load balancing, auto-scaling, multiple detacenters, etc, there is little point in going with something like AWS. OTOH, if you decide to do it yourself, you'll need to engineer a lot of what you need. That costs time and money. In the very low-end, it's cheaper to go bare VM. As size and complexity grows, it'll be cheaper to go with a cloud provider for a while, until you reach a point where you have so much infrastructure and so many services running that moving parts of it to on-prem will be the cheapest option. At this point, you will be your own little AWS.
• ddevault 6y ago
  Yes. > So, SourceHut is not hosted in anyone's cloud. I own all of the hardware outright and colocate most of it in a local datacenter. > I just built a new server for git.sr.ht, and boy is she a beaut. It cost me about $5.5K as a one-time upfront cost, and now I just pay for power, bandwidth, and space, which runs about $650/mo for all of my servers (10+). > Ran back of the napkin numbers with AWS's price estimator for a server of equivalent specs, and without even considering bandwidth usage it'd cost me almost TEN GRAND PER MONTH to host JUST that server alone on AWS. > AWS is how techbro startups pile up and BURN their investor money. https://cmpwn.com/@sir/103496073614106505 https://cmpwn.com/@sir/103496073614106505
• specialist 6y ago
  My last gig spent way too much on AWS, as percentage of own revenue. Years earlier, old brick & mortar company brought in tech consultants to rapidly pivot to e-commerce, who then farmed out most of the work to InfoSys. My team's own spend was ridiculous. Our hottest data set, which could easily fit in RAM, was on DynamoDB. So much "event sourcing" and CloudWatch. Because you needed all those logs for troubleshooting such a brittle system, right? And since our core function was the recommendation engine, of course we hoarded and munged data like gnomes, with almost no value add, result negative ROI. (+70% of our "lift" was from the customer's own "recently viewed" list. The highest cost recommendations accounted for less than 3% of "lift".) There was some director level push to migrate from AWS to Google. Which meant first Kubernetes and then Google. Such fun. There was next to zero consideration of questioning assumptions. Like unwinding the unholy tar pit of mainframe era batch processing mentality fork lifted onto cloud era web services hotness. And I don't think they could question their assumptions. The skills and experience of the traditional brick & mortar types simply couldn't. This "consultants ruin everything" story has played out across all industries.
• discodave 6y ago
  I would put the estimate of "server lying on the ground" at more like tens of billions of dollars. Here's how I back in my math: 1. AWS is a $44 billion run-rate business. 2. This estimate from the Duckbill group has EC2 at 60% of the AWS bills of "large accounts". https://www.duckbillgroup.com/products/public-cloud-analysis/ https://www.duckbillgroup.com/products/public-cloud-analysis... 3. I used to work in EC2 Networking and looked at the graphs with the server counts for all of EC2 on a weekly basis, so I can't say the actual numbers here, but I have a very good idea how big the EC2 fleet is. 4. 60% of 44 billion gives us an estimate of 26 billion run rate for just EC2. 5. In my time at Amazon where I was on multiple teams that ran large fleets (thousands of machines), and I heard about utilization numbers for services like Lambda and Fargate, 50% utilization would be a very, very good number for VMs in the cloud. 6. 50% of 26 billion is over 10 billion per year of wasted server capacity (that people are paying AWS for) So yeah, tens of billions of dollars of AWS and the other cloud providers revenue are likely "waste" from the POV of customers.
• lbriner 6y ago
  There appear to be a few other important factors missing from the article. 1) It isn't just that AWS (and others) provide loads of services, it's that I can run everything I need in the same data centre. SQL Azure might be a good pricing model for your system but if it makes sense for everything else to be in AWS (features, cost whatever) then I am more likely to spend more on AWS RDS or whatever. Might cost £1000s more but I cannot simply split my load across two providers, that is two points of failure. 2) The level of support you get from OVH is nowhere near as good as AWS, which is to be expected when comparing a "budget" provider with the "gold standard". OVH can rent stuff to you cheaper because their support team is presumably much smaller. 3) Even the UI for OVH gives me the heebie jeebies. I use them for personal stuff because they are cheap but the UI is far too slow to use for fast work. I see notifications that I have previously cleared and I get random emails occasionally about things I don't quite understand. I can live with that for cheaper personal sites but if I was using that every day, I would quit! Of course, for many people, there are issues with marketing (I haven't of some of the other providers) and the basic, and probably fair, assumption that with AWS you know the sort of quality you expect - a bit like buying Toyota. The time and risk involved in evaluating other providers is not worth it in many cases, especially when you only learn 6 months later that OVH don't have a configurable load-balancer or they don't have proper zone redundancy or you cannot upgrade certain hardware without literally copying your files into storage and trying to copy them back to a larger machine.
• varispeed 6y ago
  I went into OVH and AWS rabbit hole. I think person needs to understand the concept of alternative cost when approaching such problem. Sure, you will save money on the server itself, but you don't get all other things that AWS is adding and you would have to one way or another do them by yourself. Even if your time is extremely cheap, I don't think one is able to beat their price as much to justify doing all this work. They've been at it for years and their solution is battle tested and you may be adding issues you won't be aware of that will bite you once you get growing.
• johnghanks 6y ago
  god this is written so poorly it's hard to follow
• sparrc 6y ago
  AWS is not optimized or built for people who want to reserve their own personal server for a year or more. Another cost option that the author didn't mention are spot instances. An r4.16xlarge spot instance in Paris for one year would cost $8,900, about half the price of OVH and that is assuming that the customer needs the server 24x7. Having a server 24x7 obviously wouldn't work on spot, but the point is that AWS is built more for customers who are frequently creating, destroying, and scaling their clusters based on need.
• closeparen 6y ago
  Hypothesis: the big three US cloud providers are not really trying to be competitive in the European market. Their EU datacenters price in the convenience of a consistent dev/ops story when a US company extends its primarily US footprint into Europe. They are a bad deal for entirely European ops, but this story doesn’t generalize to the US. US dedicated server pricing is much closer to cloud pricing.
• agilob 6y ago
  Amazon hides a lot of extra paid features that are free in dedicated servers. I got cut by their practices of hiding prices while trying to experiment with "free-tried" someone recommended me. Amazon made it really easy to subscribe to paid features. After first $50 bill I moved to kimsufi where I have more memory, disk, transfer and public addresses for €5/monthly.
• boublepop 6y ago
  This is giving too much credit to organizations. In some Cloud providers arechosen plainly because some executive asked “So are we cloud?” And someone said “err I don’t think so” and then some other middle manager was charged with “make us cloud please” and it was never a question of what actually makes sense just a “do we side with Amazon, Google or Microsoft?”.
• hinkley 6y ago
  I wonder how many billions of dollars worth of unsold cars are just 'lying around'. How many billions of dollars worth of fire trucks are parked right now doing nothing? Given redundancy and inventory logistics, once your industry gets to some number of billions of dollars of equipment, you're goddamn right there's going to be a billion dollars worth of equipment lying around. It's not the magnitude, it's the ratio that you should get upset about, if there's even anything to get upset about. There's also rates of consumption versus rates of production. If consumption rates are variable (eg, ramp-up to Black Friday), there is only so much variability I can manage in production. I can produce at a mostly-stable rate all year and let inventory accumulate during slow months, or I can improve my variability, but the cost is more complicated management of people, and increased likelihood of late surprises. Also, don't data centers collide with public sector timelines? If you have to start building a data center by November 1 and you try to get your approvals all to happen on October 31, you're going to fail, and possibly get fired. You try to line those up ahead of time, maybe break ground before the next election so they can't stop it, or someone else sucks up the surplus grid power in that substation. If it's built early, it's sitting around waiting for demand to use it. But at least you have it.
  • antsar 6y ago
    The article does discuss ratios. "Billion" is just the attention-grabber. > If the first server, the one that is better in literally every way, costs ~16k/year... how much should the other one cost? Well, maybe 10, maybe 12, maybe 14? > I don't know, but the answer certainly shouldn't be "Almost twice as much at 26k/year", that's the kind of answer that indicates something is broken. > In a worst-case scenario, AWS is ~1.6x times as expensive, but again, that's paid yearly. If we compare paid monthly to paid hourly (not exactly fair) we get 37k vs 16k, if we do some napkin math calculations for equivalent storage cost (with equivalent speed via guaranteed iops) we easily get to ~3k/year extra. We have a 40k vs 16k difference, the AWS machine with the worst specs is 250% more expensive. > But whether the worst AWS machine is 160% or 250% as expensive as the OVH one is not the relevant question here, the question is why are we seeing those astronomical differences in terms of cost to being with.
  • throwaway1777 6y ago
• ohazi 6y ago
  If I get a three-year reserved instance of a t3.small, it works out to $6/month. A t3.micro works out to $3/mo. Storage and bandwidth, for my use-cases, amount to less than a dollar a month. Are there any other providers that compete on the low end like this? The common suggestions (DO, Linode, etc.) all cost at least twice as much. I buy the author's argument for "real" use-cases where you need one or more expensive servers. But I also see this argument made to people using AWS for personal use, and I've never understood it. Am I missing something? I don't need a massive server, I need something that can run code, and that's always on. AWS seems really cheap on the low end, and those machines are more than powerful enough to host your own email, files, projects, etc.
  • qeternity 6y ago
    OVH, Hetzner, and a small army of very reputable but smaller providers (Ramnode, BuyVM, etc).
  • zootboy 6y ago
    There's plenty of small VPS providers out there who will give you slivers of a server for very cheap: https://lowendbox.com/ https://lowendbox.com/
• ksec 6y ago
  I felt there are several important things missing in the article, and things that not mentioned in the 160 comments so far. AWS listed Rate are literally for small fish. Which is the stage you get your $100K / 1% of your Investment. In the 10x example, let say $1M, you should start asking for discount. And it could range anywhere from 20% to much higher. Compared to OVH which is already offering at a very low price. The price stays the same at scale. Amazon are also going full steam ahead with their own ARM instances, and those offering are 20% cheaper listed already. And in many cases they even perform better per vCore than Intel. On x86 instances vCore is a Thread, while on ARM instances vCore is an actual Core. ( Assuming your Apps works with ARM. ) Network quality - OVH, DO, Linode, none of them are any where near as good as AWS in terms of Network connection speed, routing and capacity. And this is something you can recreate with other IaaS such as OVH. All this brings AWS to a much lower multiple of OVH. And then you add up all the extra benefits such as easier to hire, Resume Driven Development, Asset vs Lease Tax Break / Financial Engineering etc. I really do wish Google and Microsoft brings in more competition though. As the author mentioned he was surprised with no single monopoly. Because the market is growing faster than all of these HyperScaler can handle. Intel has been selling as many Server CPU as they could Fab them.
• perlgeek 6y ago
  I work at a company that markets itself as a "premium IT service provider", and I think our cheapest VM is maybe 3x to 5x what the equivalent OVH or Digital Ocean VM costs. What you get, is: * actual humans to talk to, when things go wrong * hosting in central Europe, without dubious ownership structures * you actually get the VM specs you payed for, no overbooking of hypervisors * colocation in the same datacenters as the VMs. * custom deals. If you pay enough money, you can install cages with your own access control in our datacenters. * architecture support and consulting * you can visit our datacenters if you really want * lots of certifications that will make your compliance department happy * if the auditors come to you, and request access protocols to your servers, we can provide those ... and so on. Why do I list all that? Because cost is just one aspect to consider, and some business have good reasons to optimize for other aspects. Some value flexibility in their service provider, some value physical proximity, some value constant contact persons over multiple years.
• ttul 6y ago
  I run a large infrastructure SaaS on AWS. Our bill is frightening and so we work diligently to find ways to cut it down. Yet, despite other cloud providers offering much lower prices on CPU, storage, and bandwidth, we stick with AWS for the following reason: If we were to switch to a cheaper provider, we would have to do a lot of building ourselves, which would require hiring talent and keeping that talent employed continuously. Any time we look at the switch, it doesn’t make sense because of the up front costs of staffing that we would need to absorb.
  • jopsen 6y ago
    Also if you build it yourself it carries risk -- each project could fail, or take extra time. Where as buying AWS, has low risk.
    • ttul 6y ago
      Yes. And risk here means my business fails potentially. AWS has never once had an outage in the region we operate in.
    • crisper78 6y ago
      Servers go down in AWS too, they are a colo shop with an open source services arm to implement multi tenant services of which you are one tenant yet paying for that multi tenancy service. You may not notice it having 1 but have a few thousand. Everything goes down in AWS just like it does on normal servers, no one likes to mention that though so often its blamed on something else like software that is running on the cloud that just went down hah.
      • ttul 6y ago
        Okay fair comment. But in terms of the entire data center being unreachable, that has never happened - not in ten years.
• krob 6y ago
  what you're paying for is the capability of scalable architecture, on-demand launch, and heavy redundancy. this very easily acts as a multiplying cost factor and building out your own infrastructure as opposed to leveraging one that can dynamically scale as your needs require. Yes you can get away with paying for your own infrastructure at a third the cost, but then when you compute the cost of all the individual requirements. colocation ISP prices, space, people to manage all of these machines, the software to then scale yourself in equivalence, you eventually end up at w similar cost apples to apples, oranges to oranges if you replicate for yourself exactly what you get through AWS.
• ponker 6y ago
  The fact is that cost of compute is only a serious fraction of revenue for a small percent of companies. Those should worry about compute — the rest should focus their attention elsewhere.
• whatsmyusername 6y ago
  I can think of one reason why OVH is that much cheaper. OVH doesn't bother policing their network. At all. You don't even need to take my word for it, Cisco Umbrella has written numerous papers on the subject. They're one of three hosting companies I instantly blacklist across the board when I take custodianship of a network. We don't want their customers business.
• oneplane 6y ago
  Bad comparison: - on-demand vs upfront (AWS has upfront with slimilar cost reduction) - scalable vs. fixed (when you stop using it on AWS you also stop paying) Then there is all of the functionality and integration everyone else already has touched on.
• pontifier 6y ago
  Personal observation: I purchased a company that was locked into cloud infrastructure. They had received over $10M in venture capital several years ago, and had been running their own servers at some point because I found some of them in their warehouse. At some point they had switched to AWS, and then at a later point switched to GC. When I encountered them they had shut down due to lack of funding. Their servers were running in zombie mode racking up over $6k per month in charges. Data export would have cost over $10k. They were also renting warehouse and office space. They had very few assets, and not enough income to cover expenses. If their server costs had been reasonable, they might have survived, but they couldn't bring those costs down.
  • annoyingnoob 6y ago
    > had been running their own servers at some point Wonder why they stopped?
• karmakaze 6y ago
  I've worked at a shop that used both bare-metal servers from OVH as well as VMs from AWS. Using the EC2 instances were infinitely simpler, we could easily automate so much of it and manage using load balancing and scaling services. AWS instances were the default choice as the scaling up and down was a huge benefit as well as the simpler management and additional services available. The only places we used OVH bare metal servers was for bulk. Our ElasticSearch cluster had so much RAM that the EC2 cost was very prohibitive. For our DB servers we couldn't at the time get the same performance raid SSDs/controllers and vertical scaling (256GB, 512GB). For our disposable webapp servers we could have used either and since the ops was already worked out having used OVH before AWS kept it that way with a bit more ops and a smaller infra bill. When you have hundreds of hardware hosts, they're failing monthly and it's up to you to image a new one and add it to the cluster. To have only one way to do things, it also meant that Chef was used instead of immutable instance creation which was a pain to keep everything in sync. My thought process is basically: (1) small number of instances -> cloud VMs, (2) medium instances w/ founder(s) time but no capital -> bare metal, (3) mature app or huge number -> bare metal
• simonebrunozzi 6y ago
  > If I thought the whole "beat out the big guys by using cheap compute" thing was so easy, I'd be doing it instead of writing about it. Very wise.
• kiwijimm 6y ago
  Seems to me that this sort of thinking is predicated on the the idea that infrastructure should still work like it did 10-20 years ago. Cost is not the driving factor, and in any case, cost is often calculated wrong. Putting one set of prices 'for stuff' in one column and another set in another column and looking at the difference tells you almost nothing. Forget about start-ups for a minute. There are certainly arguments to be made both ways there. Focus on big enterprises for a second. Infrastructure complexity isnt getting smaller over time and neither is demand. Delivering on demand and managing complexity with a bounded number of people requires a change in thinking when it comes to infrastucture writ-large. We cannot sustain the old on-premises, dinosaur pen style data centres and deliver and grow our core businesses. It just wont work... So you either go cloud or create something very cloud like and do it on-premises. Heck that is how AWS came about in the first place. Anyone that thinks going cloud is a good way to reduce headcount is going to get a shock. Going to cloud (or lets say changing the way you do infrastructure) is a way to continue to do business with the headcount you have. It's not a question of carrying on with the 'old school' infrastructure you have, you just want keep up doing that. Do nothing and your headcout requirements are unbounded. Anyone that thinks cloud infrastructure requires you to hire only a bunch of cloud experts is wrong. Chances are you have the bulk of the infra people you need right now. All those people that have been doing 'old school' infrastructure for years are still your most valuable resource. The mechanics of the infrastructure are fairly irrelevant (ok they are not, but in the grand scheme of things we can kind of cancel the mechanics out in the equations), oft missed value is the operational knowledge. The operational bit is what gets swept under the rug in the DevOps discussion. I believe the knowledge of how to translate existing infra into a 'new' model doesnt come from hiring DevOps people or cloud people, it comes from the infrastructure people that have been doing it for years. Leverage the intellectual capital you already have. The flip side of that bargain is that 'old school' infrastructure people need to recognise we have to adapt. Those that dont are doomed to be cancelled out in the same equations that cancel out the mechnanics of the infrasturcture itself. Enterprises that fail to recognise the shift are also doomed. Those start-ups we arent talking about? They can scale far more quickly and quickly get to a point where they can deliver at the level a much bigger organisation can. They can eclipse the slow movers. I assume this is why a lot of consolidation happens... The slow dudes only really have one move and that is to buy the little guys before they can get there (the Facebook defence). But thats like doubling your bet on red every time you lose at roulette. Eventually you go bust or the game moves quicker than your bankroll... Thanks, James
• kelp 6y ago
  I'd originally posted this here: https://lobste.rs/s/surdxc/is_billion_dollar_worth_server_lying_on https://lobste.rs/s/surdxc/is_billion_dollar_worth_server_ly... But cross posting in case it's interesting to this audience. Over the past few years of my career, I was responsible for over $20M/year in physical infra spend. Colocation, network backbone, etc. And then 2 companies that were 100% cloud with over $20M/year in spend. When I was doing the physical infra, my team was managing roughly 75 racks of servers in 4 US datacenters, 2 on each cost, and an N+2 network backbone connecting them together. That roughly $20M/year counts both OpEx and CapEx, but not engineering costs. I haven’t done this in about 3 years, but for 6+ years in a row, I’d model out the physical infra costs vs AWS prices, at 3 year reserved pricing. Our infra always came out about 40% cheaper than buying from AWS for as apples to apples as I could get. Now I would model this with savings plan, and probably bake in some of what I know about the discounts you can get when you’re willing to sign a multi-year commit. That said, cost is not the only factor. Now bear in mind, my perspective is not 1 server, or 1 instance. It’s single-digit thousands. But here are a few tradeoffs to consider: Do you have the staff / skillset to manage physical datacenters and a network? In my experience you don’t need a huge team to be successful at this. I think I could do the above $20M/year, 75 rack scale, with 4-8 of the right people. Maybe even less. But you do have to be able to hire and retain those people. We also ended up having 1-2 people who did nothing but vendor management and logistics. Is your workload predictable? This is a key consideration. If you have a steady or highly predictable workload, owning your own equipment is almost always more cost-effective, even when considering that 4-8 person team you need to operate it at the scale I’ve done it at. But if you need new servers in a hurry, well, you basically can’t get them. It takes 6-8 weeks to get a rack built and then you have to have it shipped, installed, bolted down etc. All this takes scheduling and logistics. So you have to do substantial planning. That said, these days I also regularly run into issues where the big 3 cloud providers don’t have the gear either, and we have to work directly with them for capacity planning. So this problem doesn’t go away completely, once your scale is substantial enough it gets worse again, even with Cloud. If your workload is NOT predictable, or you have crazy fast growth. Deploying mostly or all cloud can make huge sense. Your tradeoff is you pay more, but you get a lot of agility for the privilege. Network costs are absolutely egregious on the cloud. Especially AWS. I’m not talking about a 2x, or 10x, markup. By my last estimate, AWS marks up their egress costs by roughly 200-300x their costs! This is based on my estimates of what it would take to buy the network transit and routers/switches you’d need to egress a handful of Gbps. I’m sure this is an intentional lockin strategy on their part. That said, I have heard rumors of quite deep discounts on the network if you spend enough $$$. We’re talking 3 digits million multi-year commits to get the really good discounts. My final point, and a major downside of cloud deployments, combined with a Service Ownership / DevOps model, is you can see your cloud costs grow to insane levels due to simple waste. Many engineering teams just don’t think about the costs. The Cloud makes lots of things seem “free” from a friction standpoint. So it’s very very easy to have a ton of resources running, racking up the bill. And then a lot of work to claw that back. You either need a set of gatekeepers, which I don’t love, because that ends up looking like an Ops team. Or you have to build a team to build cost visibility and attribution. On the physical infra side, people are forced to plan, forced to come ask for servers. And when the next set of racks aren’t arriving for 6 weeks, they have to get creative and find ways to squeeze more performance out of their existing applications. This can lead to more efficient use of infra. In the cloud world, just turn up more instances, and move on. The bill doesn’t come until next month. Lots of other thoughts in this area, but this got long already. As an aside, for my personal projects, I mostly do OVH dedicated servers. Cheap and they work well. Though their management console leaves much to be desired.
• luord 6y ago
  Great write-up, and I've got to say that the biggest takeaway from it, for me, is that the cloud pie has more sections than I thought it did. Well, I guess in the back of my mind I knew there were quite a few providers, but I never paid serious attention. From now on, I'll make a conscious effort of at least taking a look beyond the most popular providers (and Heroku), and not just for the sake of cutting costs.
• some_random 6y ago
  Honestly this is a terrible article, the only real point on display is "if you use AWS for EC2 reserved instances only, then you are overpaying". If your application can be interrupted, spun down, and spun up then congratulations you can use spot instances[1] at a fraction of the price (depending on your region and instance size). r4.16xlarge isn't available in France, so I'll use us-east-1 Reserved: $36,365 Spot: $9,665 UVH (Still France): $25,771 So if your system can tolerate nodes going up and down every once in a while (and some other caveats), it seems pretty dumb to pay for the dedicated server. [1]https://aws.amazon.com/ec2/spot/pricing/ https://aws.amazon.com/ec2/spot/pricing/
• jopsen 6y ago
  How fast can you reset an instance on OVH? I personally wouldn't want a server to live longer than 48 hours :) (Unless it's a managed service)
  • dpedu 6y ago
    Does it matter? I'm sure the machines backing EC2 live quite a long time. And that's OK depending on what they're used for.
  • crisper78 6y ago
    Can you please tell us why? What are you afraid of? I have seen servers up much longer than 2 days, still humming along no memory leaks no whatever you can think of that "rots" on a server that doesn't change. Maybe your app has a lot of memory leaks or is not well coded where it has to be restarted every few days, or as you say completely brought up on a new server why?
    • jopsen 6y ago
      2 days isn't a strict limit :) But I wouldn't want to maintain instances that aren't regularly cycled. Then you have to install security updates and things like that. I think many people today, only want to own the inside of their container, and only have the container exposed through a load balancer that has some sanity check on incoming traffic.. Most LBs won't forward non-HTTP requests, etc..
• AtlasBarfed 6y ago
  The lawyer/legal department bottlenecks on cloud usage is a pretty maddening aspect of corporate cross-shopping of clouds. At this point, every semi-established cloud provider must have sufficient paperwork to cover the various legal permutations. And it's not like the lawyers are doing any technical research to verify any security aspects.
• GhostVII 6y ago
  Or maybe cost just doesn't really matter to many companies. If using AWS helps you scale easier, and lets developers spend less time managing servers, that can be extremely valuable, especially at a company that is growing quickly. Who really cares if you are spending too much, the goal isn't profitability it's getting as many users as possible. For many tech companies, I think money is pretty much a non-issue, as evidenced by the insane salaries they offer - the real challenge is scaling up an organization without completely paralyzing all progress, and AWS can help do that.
• newintellectual 6y ago
  Completely unmentioned is reliability of infrastructure and scope of integrated systems for OVH vs AWS. OVH looks good on paper but wait until there’s a problem. Downtime cost is usually far more costly than just server cost.
• JacobSuperslav 6y ago
  pretty sure trillions of dollars in unused houses are lying on the ground
• csense 6y ago
  I know someone who needed to run a large-ish distributed workload. The project had a pretty respectable budget (I think more than $10k, less than $1M). They were going to rent DigitalOcean servers, because benchmarks showed they were the cheapest hosting provider around. Then their account got banned due to "patterns associated with cryptocurrency mining." And they had no way to get in contact with a human, appeal the ban and explain their situation. (Interesting side-note: Even if they were doing cryptocurrency mining, AFAICT it isn't against DigitalOcean ToS.) They ended up switching to another provider (Linode I think?) Anyway, what I'm trying to say is, it's an example of a cheap provider who has good paper specs and even good benchmarks. But when you actually try to use the platform at scale, you'll discover there's some catch and you can't actually use it. (I speculate that maybe DigitalOcean madly oversubscribes physical CPU, so anyone who actually uses the resources they pay for trips the automated system and gets their account banned, because DigitalOcean can't both keep their cheap prices and pay for the engineering effort to figure out how to throttle customers who peg their CPU. So they make enough money to stay in business from websites and non-CPU-heavy workloads, but have to use the backdoor "cryptocurrency mining" banning excuse to turn away customers who basically want to buy CPU-hours.)