14 ms·

The Cloud

244 points by zulgan 7y ago

gist 7y ago
Kind of along the vein of someone telling me I am paying to much for my fancy luxury sports car and completely ignoring the reasons that I find value in it.
ElFitz 7y ago
That's... a point of view. I'm just glad I can do my things without even having to think about, let alone maintain, 1/10th of all this, really. But YMMV, I guess.
- tobib 7y ago
  It's very short sighted to ignore the money being saved by not having to worry about all those things. Their "$5k" example is probably wrong and also irrelevant in most (enterprise) contexts.
- wayoutthere 7y ago
  Part of me wonders if this is a troll or just a green sysadmin.
  - pferde 7y ago
    Or perhaps just an experienced sysadmin who has different opinions and preferences than you do. There is never a single TheRightWayToDoThings(tm), different people have different needs, priorities and/or resources available.
    - kixiQu 7y ago
      And an experienced sysadmin would know that there are plenty of needs, priorities, and resources that play well with the cloud. ;)
- jcastro 7y ago
  This article ends up being a good reason for using cloud. Reads like it was written by "I could write Dropbox in a weekend with rsync" guy.
mktmkr 7y ago
Hard to take seriously and barely rises to the level necessary for rational debate. Is there a workload that is demonstrably 100x slower on AWS or GCE compared to the fully-loaded cost of bare metal?
- pmlnr 7y ago
  Well optimized MySQL masters, probably.
  - PaulHoule 7y ago
    MySQL-RDS is the AWS service which I felt gave me the least value for the dollar. (Except for the OpenVPN server which went nuts and did an obscene number of I/Os against EBS which ran up my bill)
  - mktmkr 7y ago
    "Well optimized MySQL" sounds a bit like having a really attractive liver cancer. Cloud services expand your options and it's worth pondering them. Don't ask what is the best MySQL, ask for a given amount of items I need to store and recall what is the best server OR SERVICE that could do it? MySQL on bare metal? On virtual machines? RDS? Cloud Spanner? Do cloud providers increase your ability to choose scale-up instead of scale-out? Instead of your sharded and replicated MySQL that is almost inoperable and never seems to maintain consistency, would you be better served with one gigantic database? Sure it costs $13k/month to rent one from Google but it costs a quarter million to buy one from Dell. These are all aspects of the decision-making process.
- wayoutthere 7y ago
  The author was largely focused on specific implementations over architecture patterns, so I wouldn’t think they’re operating at a level where TCO is a consideration...
- nostrademons 7y ago
  The TechEmpower benchmarks showed roughly a 10x difference in performance between AWS and bare-metal for basic webserving + DB, across many different languages and frameworks: https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=fortune https://www.techempower.com/benchmarks/#section=data-r18&hw=...
deleted 7y ago
[deleted]
wayoutthere 7y ago
The money you save in the cloud isn’t on hardware, it’s in not having to pay a sysadmin so you can work on business problems. And that benefit only improves with scale.
- SketchySeaBeast 7y ago
  Didn't you see? You're just supposed to learn some sysadmin skills yourself. The trick is to make yourself the linchpin, so that once the site grows large enough you can never leave, as you're the only one able to keep that writhing mass of wires up and running.
  - leetbulb 7y ago
    Yeah, been there, though against my will. Boy was I happy when the system (which actually worked extremely well) was shut down and replaced with a SaaS-based product.
- mey 7y ago
  That job doesn't go away, it just changes focus. See DevOps. Generally it is a net positive in my book, but don't believe that operations goes away.
  - pferde 7y ago
    Indeed. Even despite all the shiny web console controls, someone has to know what all the items on the invoice from the cloud provider do, and how they work together.
- hartator 7y ago
  What's different between managing a VPS and an actual server?
  - LoSboccacc 7y ago
    who drives to the center to change the disk when it fails
PaulHoule 7y ago
I dunno. Going from "lift-and-shift" use of EC2 to Cloud Native (lambda, cloudfront, etc.) lowered my AWS bill from $300 a month to $50 a month. I switched from the dedicated hosting that he advocates to AWS long ago because I was always having things get broken by the people at Softlayer. For instance one time I upgraded my network port and somehow that broke my record in the issue tracking system so I could not longer put issues (or any work reports in.) There was the clunky and expensive backup server (I'm not sure if it would really restore...) and the breaking point was when somebody added a new disk but they did it wrong so that the partition table got overwritten when the machine rebooted. I was able to fix the partition table but then I moved my files to EC2 as fast as I could because i didn[t have time to deal with that. As for queues I wonder what is up with that. When I build systems based on message queues they work OK, but I've got an eye for detail and for architecture that many people don't seem to have because I always see other people get in trouble with them.
- trazire 7y ago
  To be fair, those are cheaper because they want to make you more reliant on AWS and because those are more idyllic for Amazon to run, as with Lyft. Same with AWS Activate and etc. But for most people, it's simpler to just use them and get it over with, and I'm glad it worked for you.
- lliamander 7y ago
  Yeah, I think the kicker with managed services vs. self-hosted is whether you're paying for capacity or by usage. The thing with capacity driven costs is that you will always have some unused capacity (and indeed you should as a safety measure). If you're paying by usage, then that pushes the issue of managing overhead capacity onto the service provider.
trilila 7y ago
“don't use CDNs” good luck with that on high traffic volume websites.
- merb 7y ago
  > CDNs increase your complexity, they creep into your deployments and the way you think..invalidation of objects I've never seen invalidation in decades. you just have unique names.
notwedtm 7y ago
The core benefit of "the cloud" is missed here. I don't think any serious engineers are saying you CAN'T do it this way, just that nobody wants to anymore.
johngalt 7y ago
Respectfully disagree. I'm an ops person, and I have a nostalgic attachment to running my own stuff in boxes that I can touch (or at least visit). However, cloud is just better for most applications. Certainly there are some scenarios where rolling your own is important, but it is uncommon now and only going to increase. Yes cloud is expensive in a raw performance per $ sense, but that isn't it's value proposition. Most workloads aren't limited by the processing power that you can buy anyway. The draw for cloud is that it abstracts away entire classes of problems. Certainly it provides new problems as well, but those can be worked on in concert with everyone else working on the same platform.
freehunter 7y ago
Man I wish HN had a downvote option for articles that are just not worth reading. "Don't go to the cloud, just buy your own servers" completely ignores the reason anyone rents cloud servers in the first place. If I could easily say "I need exactly this much capacity, no more no less, with no unexpected scaling needs and no code/infrastructure changes" then I'd be sitting pretty. Now for a show of hands, how many companies does this describe? And of those companies that raised their hand, how many can say "I'm fine with just having a server in Germany" who will then go on to say "and I don't need a CDN to serve customers in other regions"? >keep running live/live setup...This is very critical Oh I didn't know it was that easy. If all you have to do to keep a server running is say "stay running, it's very critical" then of course no one needs any managed services. Outages are solved forever. Completely worthless advice from a text file written by... I'm sorry who is this person and what authority do they have?
- OnlyRepliesToBS 7y ago
  so much wrong
- ggg3 7y ago
  not buying into aws and not having a cdn are completely different questions. I'd downvote your comment even before reaching the ad hominem attack at the end.
- JMTQp8lwXL 7y ago
  A $200 DIY hardware cost (equivalent cloud price quoted as $5000 in the article) suggests that, for $4800/mo, you too can do Dev Ops yourself. For a lot of people, that proposition is a hard pass. You want enterprise-grade backups. If your availability drops, your business will lose credibility. Also, good luck finding a Dev Ops engineer for $57,600 a year.
  - ElFitz 7y ago
    Not even really 57,600$. Over here, a person costs their company 1.5x the salary written in their contract (and nearly 2x their "take-home" salary). And most developed countries have something similar (more or less). So the salary we're really talking about actually is less than 40k a year. Good luck ^^'
rifung 7y ago
"The cloud (aws, gcloud, azure... whatever) is a piece of shit. 10x price for 1/10th performance (so 100x price haha)" This ignores the pay of the extra people you need to pay to build/maintain your infrastructure..
andrewstuart 7y ago
It's not eloquent prose but the message is valid in some ways. "The Cloud" (presumably AWS/GOOG/AZURE) has become very expensive and very complex.
- jasonvorhe 7y ago
  Compared to pricing, features and UX cloud has gotten so much cheaper and easier to use if compared to just a few years ago, let alone the first steps AWS took back in the day.
patrioticaction 7y ago
Lead times on server orders are non trivial once you are buying a large quantity with specific configurations. Want to scale up? 5 weeks minimum. Your opportunity may have been lost by then.
- dylz 7y ago
  I use the host he mentioned - they have an API, and my average time to deploy a baremetal physical machine is under 60 seconds via API. They don't really do custom specs, iirc part of the low pricing is because of how much useful automation they have. Sure, it's not cloud cloud, but if I have a personal side project that I need a few terabytes of ram for, there is no way I can afford AWS/GCP/etc.
  - SketchySeaBeast 7y ago
    You can grab new metal and have it all up and running in 60 seconds? That's amazing.
    - dylz 7y ago
      Yes. The API call is here: https://robot.your-server.de/doc/webservice/en.html#get-order-server-product https://robot.your-server.de/doc/webservice/en.html#get-orde... The thing is that most baremetal hosts are exactly as the parent described - annoyingly slow, manual, etc. Hetzner is a little gem here, where they have a large selection of pre-specced machines that are ready for automated deployment, and they also show you a pool of cheaper machines that were returned by other customers/terminated/cancelled and automatically wiped with varying specs that you can also automatically pick out via API (this includes random upgrades that the ex-customers added). If you manage to fit in the square hole that their automation occupies, it's a great fit.
      - merb 7y ago
        yeah it works if you only need to serve europe. and if you do not need a private network for your servers (for metal they cost a lot of money, even on hetzner, they have a cloud tough which has a network feature). also keep in mind once you try to scale hetzner, is quickly a bottleneck. > 100 servers per hour. EDIT: oh and also server buying/renting/installing is probably the smallest problem when running your own metal/colo or renting servers. a big use case for clouds is, data sharing between hosts via object storage/moveable disks, etc. which is non trivial on "bare metal/colo/renting" and that is just one use case.
nilkn 7y ago
In many cases, using the cloud is a business and staffing decision more than it is a technical one. It's a decision to accept a monthly cloud bill above a certain threshold instead of maintaining an expensive and perhaps extensive operations staff.
didip 7y ago
Ugh, what kind of opinion is this? He neglected to mention all the drawbacks: * It takes forever for Dell or HP to ship your stuff. * Your co-lo can physically ran out of space. * Your predecessor will inevitable have tons of one-off unversioned changes in prod. * Repetitive manual work when scaling multi DCs. * Making unscalable database choices (non-master-master) because you are not in the cloud mindset. * Don't worry about CDN? AWS S3 comes with CDN automatically. * Too many more bullet points to mention.
- freehunter 7y ago
  >Your co-lo can physically ran out of space The number of times I've run into this at clients is ridiculous. Either there isn't enough space in the rack, or they're out of network switch ports, even down to being out of SFPs or network cables themselves. I've even seen clients run out of power... as in, they couldn't add one of our servers to the data center because the UPS and power delivery systems could not safely handle the load of one more server. Running a datacenter is a lot harder and more costly than most people think.
- quickthrower2 7y ago
  > It takes forever for Dell or HP to ship your stuff. Not a problem for this guy, he uses hetzner's cloud. Did I say cloud, sorry I mean server rental.
Townley 7y ago
Alongside a host of other issues ("Don't use CDNs because they increase your complexity" is a wild idea coming from someone advocating bare metal boxes) I think there's a confusion of terms when it comes to "cloud" I prefer renting virtual private servers from Digital Ocean (or, yes, EC2 instances) and configuring them myself. Doing so is perhaps more involved than buying into the AWS/GCP/Azure ecosystem, but I already know the tools involved, and I value the safeguard against vendor lock-in/buy-in enough to spend more time on them. Those VPSs are "cloud" solutions in my opinion, and a happy medium between AWS and the author's unnecessarily-profanity-laden proposal that everyone should rent/buy metal. You can be "pro-cloud" and "anti-closed-cloud-ecosystem" at the same time.
- burpsnard 7y ago
  a big advantage to shoving containers into a larger vps is saving $$ and Mb/s internode.
jimconstable 7y ago
It seems people are still angry on the internet. These number seem made up and the advice is not realistic. "Avoid big data" What does that even mean?
- zxcmx 7y ago
  He means that in his opinion text files are fine for low value analytics until they don’t fit on an SSD you can afford. In the “roll your own, close to the metal” world this is a pretty sane opinion when the alternative is like, spark or hadoop or something that suck up ops time.
therealmarv 7y ago
He lost it on "don't use CDN". Asia/Europe/America ... if you run a little bit more world wide you will be thankful for every sort of CDN when it's not just text.
huangc10 7y ago
As someone mentioned, YMMV. That's all I have to say. Fun side project and learning experience though.
ekzy 7y ago
heroku create; git push heroku master; and use my mouse to scale up. But yeah, sure, the cloud is a piece of shit. Just a beautiful, magical piece of shit.
diminoten 7y ago
Said like a person whose never been woken up at 3am due to a hardware issue...
streetcat1 7y ago
What the cloud gives you (and thus cloud native), is way to manage infra like you manage software objects. I would say - run everything in kubernetes (cloud or not cloud). Change your infra layer to use kuberentes objects. Then you minimise the problem to : where is my kubernetes cluster, on cloud, on prem, I do not care.
dulldrums 7y ago
10x price for 1/10th performance (so 100x price haha) Don't go to the cloud you're such a good sysadmin aha
neighbour 7y ago
I disagree with the author but I love reading articles like this.
timwaagh 7y ago
Well I guess cloud is just the new reality now. There's no point fighting windmills really. If it's crappy, but fashionable, it's still fashionable.
wang_li 7y ago
I'm glad this person has an opinion, but it would be nice to know if this opinion is based on how things are today, or two years ago. Note to bloggers: put dates in your damned posts.
throwaway082729 7y ago
This post is equivalent to that famous comment on Dropbox about running a fileserver yourself. It's a waste of time and I can't believe it's being upvoted on HN.
- majewsky 7y ago
  I would like to have a browser extension that filters all the comments referencing the HN Dropbox meme. It stopped being funny after the 100,000th time.
tomc1985 7y ago
Oh look, all the cloud cheerleaders have come out to mount their defense! RIP sysadmins, and all the IT knowledge that's slowly disappearing from the web because the sysadmins all work on proprietary stacks at FANG now
- fteem 7y ago
  There was never a better time in the history of the internet where "all the IT knowledge" was more available and easy to get. And the sysadmins are doing pretty well I am sure, they'll pitch in here once they find their way out of Terraform's documentation.
scarejunba 7y ago
This guy has a specific problem that's best solved this way. For ten years, I ran my own SQL server, DNS, web servers, game servers, etc. It was the right thing to do. Because I was a hobbyist. Price was everything. That's not the deal for most startups, though. You're optimizing for the search process. So optimize for the search process. Even after you've found some fit, you're still searching and attempting to grow. Everything that cuts friction from ideation to sale is pure gold. That's why the cloud rules. I don't figure out how to host the right-sized server with the right infra. I put it on GCS and BigQuery that shit. Or I snowplow into S3 and then Redshift Athena it. Two days to discovering product doesn't work. If you've ever worked at a big company, you'll know that they have strict IT ops and then a shadow IT operation that manages to be every growth opportunity that gets late merged into the real thing. You want to bypass that. You want to enable people to build things they want to build and sell those things while having automatic best practices. That's the magic of the cloud: instantaneous infrastructure, instantaneous tooling. $5k is nothing. For the gain I get from that I'd pay out of my own pocket.
MadWombat 7y ago
Back in 2005 I used to work as a systems admin on a relatively high traffic real estate web site. Back at the time, cloud was not a thing, so we had a rented rack in a colo facility and stuffed it full of gear. I remember very well, just how much work and ingenuity it took to keep this thing running, despite the best efforts of our dev team.
andkenneth 7y ago
My summary of this article: Don't go cloud because it will cost you heaps. Instead, spend most of your day sysadmining instead of working on business problems. Or hire a sysadmin. Sounds like this persons job is being made redundant by clients moving to the cloud and he's not adapting to the market.
mcs_ 7y ago
If you manage some legacy systems, as I do, you will find that Cloud is very useful for automation, CI, and dev environments, why? Well, create scripts to build Window servers created with custom images is actually a big deal, so: - disaster recovery is the first thing you should have in mind, where is the second. - the right strategy is to preserve the business, not going to the cloud. I always had fun building my own environments. After the fun came the work and, economically speaking, managing and maintaining those environments has always been an advantage for me. If however, I consider the time it takes to understand and fix problems in VMWare, or SQL optimization, or configure Microsoft AD, Nginx, and others..., I don't know how much I would recommend building everything from scratch by renting some servers.
CyanLite4 7y ago
Surely this is satire...right?
- refrigerator 7y ago
  Haha, did a CMD+F for "satire" as soon as I started reading the comments. This was the giveaway for me: Don't go to the cloud. It will force you to... over-complicate your infrastructure to incredible degree. It is truly a piece of shit and will just force you to design systems in a horrible way."
vmizzle 7y ago
I run my own mini-cloud at home (42u rack with way too much hardware). I do my best to follow best practices with regard to hardware, backups, redundancy, etc. I keep downtime to a minimum (Plex is life, yo). I've been doing it for around 20 years, so I'm fairly confident I know what I'm doing. As a Director of Technology for a real company, I have never, nor would I ever dream of even considering moving to bare metal for ANYTHING mission critical. The sheer number of possible catastrophies that can occur at the infrastructure level would have me shitting my pants 24/7. Could we hire someone (or a team) to do it? Yes. But EC2 and similar infrastructure beats the pants off those salaries any day.
dekhn 7y ago
The reason I run in the cloud to be close to the object storage and average bandwidth between machines. I run big batch jobs and setting up a full cluster filesystem and fast network is non-trivial. Recently, using S3 with VMs that have 25Gbit network, I can easily make 'aws s3 cp' (after tuning) generate 10+Gbit of traffic reliably.
gioscarab 7y ago
Our times demonstrate that the effective social cost of "the cloud" for sure overtake the provided convenience.
cateye 7y ago
Over generalization is the main problem here. It all just depends on the context. Going to the cloud, can really make a lot of sense. For example, if you are a start up that is targeting a mass market and need to scale quickly in multiple regions... Or, if you are the CTO of a large company and don't want to spend time and focus on running your own data center and everything involved with it... There are also many situations where running your own collocated dedicated server is the best choice. Keeping it lean and mean and removing as much as possible dependencies, complexity and layers does have benefits. It is hyperbolic to state that something is just shit for everyone in all circumstances.
bcheung 7y ago
So what's the alternative to not using CDN's for those of us who don't have 100% of our customers in or near Germany?
quickthrower2 7y ago
I find using the cloud too close to the metal for me. I'm using Netlify for hosting. I get impression they might be using AWS under the hood.
- jwp23 7y ago
  I interviewed with them (didn't get the job). If I understood it correctly, they pretty much use every cloud under the sun to get full coverage for their CDN.
awinter-py 7y ago
Would certainly consider a 'fewer bigger boxes, more hands-on' approach, BUT ONLY for clients that had good administration talent in-house and were already debugging the managed services a lot anyway bc they're pushing the envelope. Managed services aren't perfect and I've seen them go bad, esp for things like failover. When they hiccup (either perf-wise or downtime) it's hard to debug effectively depending on what logs / stats the vendor makes available.
emmelaich 7y ago
Does anyone have a good cloud cost calculator that they can recommend? Web searching shows a few but I'd love some recommendations. Ideally an online spreadsheet, but excel would do too.
RandomTisk 7y ago
As expensive as AWS is, at my company (Fortune 500) company with at least 2 major data centers in the US for my division alone, serving around a hundred clients and thousands of web sites and applications, we found that AWS can still be a lot cheaper than a poorly run private data center. My department is technically a cost center, like most of Security, but the charge-backs we get from our own data center for various services are insanely high compared to the same services we can procure from AWS and Azure. We've been steadily moving our services out of our own data centers and into the cloud and so far, my life is a lot better. Before, just getting someone to run down a blinking disk light on a server was a chore. Getting them to rack a new server took several days or weeks. Doing anything, any changes, took several days minimum. Now, we can do it all much faster, and from all accounts of my management, it's cheaper. I don't doubt you can run your own data center cheaper, but not for my company.
deleted 7y ago
[deleted]
namesbc 7y ago
This reads like a parody. Please tell me this is a parody?
geekpowa 7y ago
Part of Friendsters death spiral was the decision to move to AWS, which was a technical disaster. On Cloud vs baremetal arguments. I respect people find cloud works and is valuable for them. This inspite of my own personal experiences, direct or indirect with cloud have been adverse. I agree with author, for me, for dollar and time effort, baremetal is just better economically and performance. Most recent. A web service I use heavily migrated from bare metal to cloud and their performance / reliability dropped significantly with numerous outages which tied me up on the phones to my clients. I gave them 6 months to get their act together, as a loyal client then gave up and moved elsewhere. Being a reasonably big client their sales tried to retain then win me back. Conversations were always the same: Me: Moved because you guys stopped being reliable, reliability is important to me. Ever since you moved to AWS you have been unreliable. Sales: we had to move to scale and grow Me: whats the point in moving if moving weakens your product. there are other ways to scale. Sale: we had to move to scale and grow
Bootwizard 7y ago
"Old man yells at cloud"
013a 7y ago
I really think there's more to this than most people will admit. Our very early, small startup (4 engineers, low six figures ARR) accumulates $4,000+/month in cloud bills. Its very difficult to manage cloud costs on a small engineering team. One of our recent examples is with SQS. Our SQS bill is something like $50/month. We process ~100k messages every month. Why is it so expensive, then? It isn't because its processing tons of messages; its because we use open-source polling libraries that are pretty aggressive with their poll rate (then multiply that out by N instances of the poller and there's your bill). I find it fun to think about "what could I get for $50/month" in regards to a queuing solution. As an extreme counterpoint, Redis running on a raspberry pi in the closet would be a rounding error in our cloud bills over the span of a year. The gray area between these two extremes is where the value lies, and I think that's the author's point: redis on a VPC at Hetzner or DigitalOcean. Billions upon billions of messages processed every month at a fraction of the cost. So, why pay AWS? There's not NO reason. Reliability is the biggest one I think. But herein lies what I believe to be the intrinsic problem with the cloud: No one can put a price on reliability. If I pay an extra $10k a year to be 99.99999 available (~10s/year), versus 99.99 (~1hr/year), how much extra value does my business receive? No one even CARES what that number is. They just know that more nines is better, so we gotta pay it. But we won't pay too much for it. At some point, it crosses some TOTALLY arbitrary threshold in the CTO's head where its "too expensive" and we need to reduce costs. But, then, we're so bought in that we can't. We can tweak poll rates, we can buy reserved instances, maybe see a 20% reduction, but the pattern is set. We're just a serf to Amazon now. The other quoted reason is always "the cloud is easier". "You'll save money on HR." "Your engineers can focus on the product." I think Amazon has shouted this line so much from the rooftops that even very smart engineering managers have begun to believe it. Heroku was really the last platform I've used that felt like it could fulfill this promise to any degree of meaning. It really did everything, but AWS has managed to create a network of services that is so fundamentally complex that even when they release services to "simplify" things they actually create more complexity (beanstalk, amplify). You're not getting around having an engineer spending a decent chunk of her time managing this complexity (building CI pipelines, managing IAM permissions, alerting strategies, centralized log ingestion, autoscaling, cost management, encoding everything into CloudFormation, ugh maybe terraform will be easier, figuring out the madness behind VPCs because some random CF stack you found that you thought could automate something for you just spat out an egress-only internet gateway and what the fuck even is that I'm just one person why did I need that). I've been working in AWS since... the beginning. There are three good AWS services: S3, SQS, and EC2 (in that order). Everything else ranges somewhere between "Fuck it, I guess I'll use cloudwatch" and "what was the rekognition team smoking when they built this, and are they getting a daily dose of it through the vents because its been years and its still not better." The best part about those "good" services is that they're all solved problems available on a wide variety of cloud providers. Block Storage, Queues, and VMs. I think that's what the author is getting at, in a weird way. The cloud is fine. Managed services on top of open source software is probably fine. Even closed-source managed services can be fine, if they're so stupidly simple that you could explain it to your 8 year old and he'd understand it. But the vast majority of AWS (or Azure, or Google Cloud) is none of these things, and that's the stuff that you should avoid. Or, if you want to save some money, head over to Hetzner or Digital Ocean.
nh2 7y ago
This is good advice overall, in my experience. Most of the small/medium sized companies I've consulted for could have been easily served by a triple-redundancy Hetzner dedicated server setup for 200 EUR/month. Yet they they chose AWS and paid thousands per month because "that's what everybody does". I have not regularly encountered actual calculation or other analysis as basis for this decision. Many people here and in industry seem to think that it's either AWS or "build your own datacenter or colocation and maintain machines". Be reminded: This is incorrect. The post recommends renting dedicated servers from a server provider (in this case Hetzner), which includes all hardware handling personnel, hardware replacement, power, and so on, for 50 EUR/month per big-sized server. Others complain about the "too points in the list". In practice, all these points have to be addressed analogously in an AWS deployment as well, and require approximately as much knowledge as the "sysadmin you'd have to hire". It's a myth that you can just use AWS without reasonable sysadmin knowledge. As a "DevOps engineer" building sophisticated projects on AWS, you usually have to have a superset of typical sysadmin knowledge. Regarding scalability: For me, Hetzner has reliably fulfilled orders for new dedicated servers within 15 minutes (!) to ssh. In contrast, I've had cases where the AWS support request to raise the limit of 0 big machines in a region to 1 has taken more than 2 business days ("a request of this size has to be escalated"). With a reasonable dedicated server setup, many companies don't need to scale for a while, because you can go a long way with ~5000 SQL inserts that a modern dedicated machine can achieve on an SSD. Again others highlight the cloud for the available internal bandwidth. Note Hetzner gives you 10 Gbit/s as a 39 EUR/month upgrade. Such bandwidth is reliable; I have measured that I can reliably transfer at Gigabit speed even across the oceans. A 1 Gbit/s dedicated server with unlimited traffic for 50 EUR/month (at Hetzner) can transfer ~260 TB per month egress. Per AWS calculator, this would cost 18700 USD there -- that's a factor of 375x on traffic cost. In general, in my experience, the factors you pay for common cloud providers over rented dedicated servers are: * 10-20x for compute * 2-4x for storage (this computation includes multi-AZ redundancy) * ~300x for traffic The article's point about not being able to use strace/gdb/iostat to debug problems on hosted services is also a good one. For example, we have found AWS EFS to have abysmal performance and could not address this problem. Using CephFS on dedicated infrastructure instead, any bottlenecks could be quickly debugged and performance was great. Similarly, the point about simplicity is good. Having just simple Linux processes running under systemd on a machine, instead of processes-in-docker-on-kubernetes-in-VMs-on-EC2, removes complexity and makes systems many times simpler and easier to debug, and thus much faster to iterate on. The only point of the article I don't quite a agree with is to not use CDNs. If you want fast loading times over HTTPs (multiple roundtrips), you need to use a CDN, because of physics. (But you can build a CDN with dedicated servers very easily.) I have successfully run a triple-redundancy dedicated Hetzner setup over the last 2 years (managed as infrastructure-as-code using NixOS) with a High-Availability database (postgres with Stolon), High-Availability file system (CephFS), systemd and round-robin DNS. So basically, everything the post recommends. It's been the simplest to manage deployment I've used for years, compared to many cloud setups I've worked on. It's had higher uptime than AWS in the last 2 years. It has extremely predictable performance. It costs 200 EUR/month. It's a pleasure for me as a programmer. In summary, I think the article is on-spot (even though it uses odd language). There remains one significant benefit of big cloud providers like AWS: Being able to spawn whatever you need via /one/ unified API in very many locations across the world. If you need something that must be distributed across as many data centers as possible, this is quite a plus. But very few people actually need this.
manigandham 7y ago
So just get a single VM from a cloud provider then. They are all perfectly fine as IaaS if you want to manage it that way, and small VMs are cheap. This seems to be a massive misunderstanding of what "cloud" even is.
valenciarose 7y ago
Even for large organizations, the most compelling argument for the cloud is capital structure and the risks associated with holding assets that may not prove to be productive. We inherently understand the advantage of the cloud for startups. It preserves capital and can be used to links expense to generated value (user engagement, etc) if not actual revenue. What may be less obvious is that the same math applies for at least some activities of larger organizations. What goes into a corporate data center is effectively a distressed asset the instant the capacity isn't needed. Forecast risk is very real, and even conservative estimates of that risk can radically change the risk adjusted return on the capital investment.
jasonvorhe 7y ago
This is the author's take on syntax highlighting: https://txt.black/~jack/syntax-highlighting.txt https://txt.black/~jack/syntax-highlighting.txt At least he's consistent.
adchari 7y ago
I honestly wonder if this is satire. I feel like this is one of those situations where the reason the writer doesn't like it is solely that he doesn't understand the true use case and reasoning behind moving off-prem. It isn't just the cost of servers, it's the fact that AWS or Azure gives you a variety of simple ways to configure and integrate services, without paying an entire team of ops people for putting servers in racks. It's more efficient and reduces overhead, which saves the kind of company that spends 6 figures on AWS costs a huge sum of money too.
lunias 7y ago
I really respect the desire and the ability to do it all yourself. I expect that you'll see real gains over the cloud (up until a point) assuming that you're knowledgeable and your environment permits you to work quickly and independently. https://github.com/jackdoe https://github.com/jackdoe "old man yells at cloud" - I loled

244 points by zulgan 7y ago

gist 7y ago

Kind of along the vein of someone telling me I am paying to much for my fancy luxury sports car and completely ignoring the reasons that I find value in it.

ElFitz 7y ago

That's... a point of view. I'm just glad I can do my things without even having to think about, let alone maintain, 1/10th of all this, really. But YMMV, I guess.

tobib 7y ago
It's very short sighted to ignore the money being saved by not having to worry about all those things. Their "$5k" example is probably wrong and also irrelevant in most (enterprise) contexts.
wayoutthere 7y ago
Part of me wonders if this is a troll or just a green sysadmin.
- pferde 7y ago
  Or perhaps just an experienced sysadmin who has different opinions and preferences than you do. There is never a single TheRightWayToDoThings(tm), different people have different needs, priorities and/or resources available.
  - kixiQu 7y ago
    And an experienced sysadmin would know that there are plenty of needs, priorities, and resources that play well with the cloud. ;)
jcastro 7y ago
This article ends up being a good reason for using cloud. Reads like it was written by "I could write Dropbox in a weekend with rsync" guy.

mktmkr 7y ago

Hard to take seriously and barely rises to the level necessary for rational debate. Is there a workload that is demonstrably 100x slower on AWS or GCE compared to the fully-loaded cost of bare metal?

pmlnr 7y ago
Well optimized MySQL masters, probably.
- PaulHoule 7y ago
  MySQL-RDS is the AWS service which I felt gave me the least value for the dollar. (Except for the OpenVPN server which went nuts and did an obscene number of I/Os against EBS which ran up my bill)
- mktmkr 7y ago
  "Well optimized MySQL" sounds a bit like having a really attractive liver cancer. Cloud services expand your options and it's worth pondering them. Don't ask what is the best MySQL, ask for a given amount of items I need to store and recall what is the best server OR SERVICE that could do it? MySQL on bare metal? On virtual machines? RDS? Cloud Spanner? Do cloud providers increase your ability to choose scale-up instead of scale-out? Instead of your sharded and replicated MySQL that is almost inoperable and never seems to maintain consistency, would you be better served with one gigantic database? Sure it costs $13k/month to rent one from Google but it costs a quarter million to buy one from Dell. These are all aspects of the decision-making process.
wayoutthere 7y ago
The author was largely focused on specific implementations over architecture patterns, so I wouldn’t think they’re operating at a level where TCO is a consideration...
nostrademons 7y ago
The TechEmpower benchmarks showed roughly a 10x difference in performance between AWS and bare-metal for basic webserving + DB, across many different languages and frameworks: https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=fortune https://www.techempower.com/benchmarks/#section=data-r18&hw=...

deleted 7y ago

[deleted]

wayoutthere 7y ago

The money you save in the cloud isn’t on hardware, it’s in not having to pay a sysadmin so you can work on business problems. And that benefit only improves with scale.

SketchySeaBeast 7y ago
Didn't you see? You're just supposed to learn some sysadmin skills yourself. The trick is to make yourself the linchpin, so that once the site grows large enough you can never leave, as you're the only one able to keep that writhing mass of wires up and running.
- leetbulb 7y ago
  Yeah, been there, though against my will. Boy was I happy when the system (which actually worked extremely well) was shut down and replaced with a SaaS-based product.
mey 7y ago
That job doesn't go away, it just changes focus. See DevOps. Generally it is a net positive in my book, but don't believe that operations goes away.
- pferde 7y ago
  Indeed. Even despite all the shiny web console controls, someone has to know what all the items on the invoice from the cloud provider do, and how they work together.
hartator 7y ago
What's different between managing a VPS and an actual server?
- LoSboccacc 7y ago
  who drives to the center to change the disk when it fails

PaulHoule 7y ago

I dunno. Going from "lift-and-shift" use of EC2 to Cloud Native (lambda, cloudfront, etc.) lowered my AWS bill from $300 a month to $50 a month. I switched from the dedicated hosting that he advocates to AWS long ago because I was always having things get broken by the people at Softlayer. For instance one time I upgraded my network port and somehow that broke my record in the issue tracking system so I could not longer put issues (or any work reports in.) There was the clunky and expensive backup server (I'm not sure if it would really restore...) and the breaking point was when somebody added a new disk but they did it wrong so that the partition table got overwritten when the machine rebooted. I was able to fix the partition table but then I moved my files to EC2 as fast as I could because i didn[t have time to deal with that. As for queues I wonder what is up with that. When I build systems based on message queues they work OK, but I've got an eye for detail and for architecture that many people don't seem to have because I always see other people get in trouble with them.

trazire 7y ago
To be fair, those are cheaper because they want to make you more reliant on AWS and because those are more idyllic for Amazon to run, as with Lyft. Same with AWS Activate and etc. But for most people, it's simpler to just use them and get it over with, and I'm glad it worked for you.
lliamander 7y ago
Yeah, I think the kicker with managed services vs. self-hosted is whether you're paying for capacity or by usage. The thing with capacity driven costs is that you will always have some unused capacity (and indeed you should as a safety measure). If you're paying by usage, then that pushes the issue of managing overhead capacity onto the service provider.

trilila 7y ago

“don't use CDNs” good luck with that on high traffic volume websites.

merb 7y ago
> CDNs increase your complexity, they creep into your deployments and the way you think..invalidation of objects I've never seen invalidation in decades. you just have unique names.

notwedtm 7y ago

The core benefit of "the cloud" is missed here. I don't think any serious engineers are saying you CAN'T do it this way, just that nobody wants to anymore.

johngalt 7y ago

Respectfully disagree. I'm an ops person, and I have a nostalgic attachment to running my own stuff in boxes that I can touch (or at least visit). However, cloud is just better for most applications. Certainly there are some scenarios where rolling your own is important, but it is uncommon now and only going to increase. Yes cloud is expensive in a raw performance per $ sense, but that isn't it's value proposition. Most workloads aren't limited by the processing power that you can buy anyway. The draw for cloud is that it abstracts away entire classes of problems. Certainly it provides new problems as well, but those can be worked on in concert with everyone else working on the same platform.

freehunter 7y ago

Man I wish HN had a downvote option for articles that are just not worth reading. "Don't go to the cloud, just buy your own servers" completely ignores the reason anyone rents cloud servers in the first place. If I could easily say "I need exactly this much capacity, no more no less, with no unexpected scaling needs and no code/infrastructure changes" then I'd be sitting pretty. Now for a show of hands, how many companies does this describe? And of those companies that raised their hand, how many can say "I'm fine with just having a server in Germany" who will then go on to say "and I don't need a CDN to serve customers in other regions"? >keep running live/live setup...This is very critical Oh I didn't know it was that easy. If all you have to do to keep a server running is say "stay running, it's very critical" then of course no one needs any managed services. Outages are solved forever. Completely worthless advice from a text file written by... I'm sorry who is this person and what authority do they have?

OnlyRepliesToBS 7y ago
so much wrong
ggg3 7y ago
not buying into aws and not having a cdn are completely different questions. I'd downvote your comment even before reaching the ad hominem attack at the end.
JMTQp8lwXL 7y ago
A $200 DIY hardware cost (equivalent cloud price quoted as $5000 in the article) suggests that, for $4800/mo, you too can do Dev Ops yourself. For a lot of people, that proposition is a hard pass. You want enterprise-grade backups. If your availability drops, your business will lose credibility. Also, good luck finding a Dev Ops engineer for $57,600 a year.
- ElFitz 7y ago
  Not even really 57,600$. Over here, a person costs their company 1.5x the salary written in their contract (and nearly 2x their "take-home" salary). And most developed countries have something similar (more or less). So the salary we're really talking about actually is less than 40k a year. Good luck ^^'

rifung 7y ago

"The cloud (aws, gcloud, azure... whatever) is a piece of shit. 10x price for 1/10th performance (so 100x price haha)" This ignores the pay of the extra people you need to pay to build/maintain your infrastructure..

andrewstuart 7y ago

It's not eloquent prose but the message is valid in some ways. "The Cloud" (presumably AWS/GOOG/AZURE) has become very expensive and very complex.

jasonvorhe 7y ago
Compared to pricing, features and UX cloud has gotten so much cheaper and easier to use if compared to just a few years ago, let alone the first steps AWS took back in the day.

patrioticaction 7y ago

Lead times on server orders are non trivial once you are buying a large quantity with specific configurations. Want to scale up? 5 weeks minimum. Your opportunity may have been lost by then.

dylz 7y ago
I use the host he mentioned - they have an API, and my average time to deploy a baremetal physical machine is under 60 seconds via API. They don't really do custom specs, iirc part of the low pricing is because of how much useful automation they have. Sure, it's not cloud cloud, but if I have a personal side project that I need a few terabytes of ram for, there is no way I can afford AWS/GCP/etc.
- SketchySeaBeast 7y ago
  You can grab new metal and have it all up and running in 60 seconds? That's amazing.
  - dylz 7y ago
    Yes. The API call is here: https://robot.your-server.de/doc/webservice/en.html#get-order-server-product https://robot.your-server.de/doc/webservice/en.html#get-orde... The thing is that most baremetal hosts are exactly as the parent described - annoyingly slow, manual, etc. Hetzner is a little gem here, where they have a large selection of pre-specced machines that are ready for automated deployment, and they also show you a pool of cheaper machines that were returned by other customers/terminated/cancelled and automatically wiped with varying specs that you can also automatically pick out via API (this includes random upgrades that the ex-customers added). If you manage to fit in the square hole that their automation occupies, it's a great fit.
    - merb 7y ago
      yeah it works if you only need to serve europe. and if you do not need a private network for your servers (for metal they cost a lot of money, even on hetzner, they have a cloud tough which has a network feature). also keep in mind once you try to scale hetzner, is quickly a bottleneck. > 100 servers per hour. EDIT: oh and also server buying/renting/installing is probably the smallest problem when running your own metal/colo or renting servers. a big use case for clouds is, data sharing between hosts via object storage/moveable disks, etc. which is non trivial on "bare metal/colo/renting" and that is just one use case.

nilkn 7y ago

In many cases, using the cloud is a business and staffing decision more than it is a technical one. It's a decision to accept a monthly cloud bill above a certain threshold instead of maintaining an expensive and perhaps extensive operations staff.

didip 7y ago

Ugh, what kind of opinion is this? He neglected to mention all the drawbacks: * It takes forever for Dell or HP to ship your stuff. * Your co-lo can physically ran out of space. * Your predecessor will inevitable have tons of one-off unversioned changes in prod. * Repetitive manual work when scaling multi DCs. * Making unscalable database choices (non-master-master) because you are not in the cloud mindset. * Don't worry about CDN? AWS S3 comes with CDN automatically. * Too many more bullet points to mention.

freehunter 7y ago
>Your co-lo can physically ran out of space The number of times I've run into this at clients is ridiculous. Either there isn't enough space in the rack, or they're out of network switch ports, even down to being out of SFPs or network cables themselves. I've even seen clients run out of power... as in, they couldn't add one of our servers to the data center because the UPS and power delivery systems could not safely handle the load of one more server. Running a datacenter is a lot harder and more costly than most people think.
quickthrower2 7y ago
> It takes forever for Dell or HP to ship your stuff. Not a problem for this guy, he uses hetzner's cloud. Did I say cloud, sorry I mean server rental.

Townley 7y ago

Alongside a host of other issues ("Don't use CDNs because they increase your complexity" is a wild idea coming from someone advocating bare metal boxes) I think there's a confusion of terms when it comes to "cloud" I prefer renting virtual private servers from Digital Ocean (or, yes, EC2 instances) and configuring them myself. Doing so is perhaps more involved than buying into the AWS/GCP/Azure ecosystem, but I already know the tools involved, and I value the safeguard against vendor lock-in/buy-in enough to spend more time on them. Those VPSs are "cloud" solutions in my opinion, and a happy medium between AWS and the author's unnecessarily-profanity-laden proposal that everyone should rent/buy metal. You can be "pro-cloud" and "anti-closed-cloud-ecosystem" at the same time.

burpsnard 7y ago
a big advantage to shoving containers into a larger vps is saving $$ and Mb/s internode.

jimconstable 7y ago

It seems people are still angry on the internet. These number seem made up and the advice is not realistic. "Avoid big data" What does that even mean?

zxcmx 7y ago
He means that in his opinion text files are fine for low value analytics until they don’t fit on an SSD you can afford. In the “roll your own, close to the metal” world this is a pretty sane opinion when the alternative is like, spark or hadoop or something that suck up ops time.

therealmarv 7y ago

He lost it on "don't use CDN". Asia/Europe/America ... if you run a little bit more world wide you will be thankful for every sort of CDN when it's not just text.

huangc10 7y ago

As someone mentioned, YMMV. That's all I have to say. Fun side project and learning experience though.

ekzy 7y ago

heroku create; git push heroku master; and use my mouse to scale up. But yeah, sure, the cloud is a piece of shit. Just a beautiful, magical piece of shit.

diminoten 7y ago

Said like a person whose never been woken up at 3am due to a hardware issue...

streetcat1 7y ago

What the cloud gives you (and thus cloud native), is way to manage infra like you manage software objects. I would say - run everything in kubernetes (cloud or not cloud). Change your infra layer to use kuberentes objects. Then you minimise the problem to : where is my kubernetes cluster, on cloud, on prem, I do not care.

dulldrums 7y ago

10x price for 1/10th performance (so 100x price haha) Don't go to the cloud you're such a good sysadmin aha

neighbour 7y ago

I disagree with the author but I love reading articles like this.

timwaagh 7y ago

Well I guess cloud is just the new reality now. There's no point fighting windmills really. If it's crappy, but fashionable, it's still fashionable.

wang_li 7y ago

I'm glad this person has an opinion, but it would be nice to know if this opinion is based on how things are today, or two years ago. Note to bloggers: put dates in your damned posts.

throwaway082729 7y ago

This post is equivalent to that famous comment on Dropbox about running a fileserver yourself. It's a waste of time and I can't believe it's being upvoted on HN.

majewsky 7y ago
I would like to have a browser extension that filters all the comments referencing the HN Dropbox meme. It stopped being funny after the 100,000th time.

tomc1985 7y ago

Oh look, all the cloud cheerleaders have come out to mount their defense! RIP sysadmins, and all the IT knowledge that's slowly disappearing from the web because the sysadmins all work on proprietary stacks at FANG now

fteem 7y ago
There was never a better time in the history of the internet where "all the IT knowledge" was more available and easy to get. And the sysadmins are doing pretty well I am sure, they'll pitch in here once they find their way out of Terraform's documentation.

scarejunba 7y ago

This guy has a specific problem that's best solved this way. For ten years, I ran my own SQL server, DNS, web servers, game servers, etc. It was the right thing to do. Because I was a hobbyist. Price was everything. That's not the deal for most startups, though. You're optimizing for the search process. So optimize for the search process. Even after you've found some fit, you're still searching and attempting to grow. Everything that cuts friction from ideation to sale is pure gold. That's why the cloud rules. I don't figure out how to host the right-sized server with the right infra. I put it on GCS and BigQuery that shit. Or I snowplow into S3 and then Redshift Athena it. Two days to discovering product doesn't work. If you've ever worked at a big company, you'll know that they have strict IT ops and then a shadow IT operation that manages to be every growth opportunity that gets late merged into the real thing. You want to bypass that. You want to enable people to build things they want to build and sell those things while having automatic best practices. That's the magic of the cloud: instantaneous infrastructure, instantaneous tooling. $5k is nothing. For the gain I get from that I'd pay out of my own pocket.

MadWombat 7y ago

Back in 2005 I used to work as a systems admin on a relatively high traffic real estate web site. Back at the time, cloud was not a thing, so we had a rented rack in a colo facility and stuffed it full of gear. I remember very well, just how much work and ingenuity it took to keep this thing running, despite the best efforts of our dev team.

andkenneth 7y ago

My summary of this article: Don't go cloud because it will cost you heaps. Instead, spend most of your day sysadmining instead of working on business problems. Or hire a sysadmin. Sounds like this persons job is being made redundant by clients moving to the cloud and he's not adapting to the market.

mcs_ 7y ago

If you manage some legacy systems, as I do, you will find that Cloud is very useful for automation, CI, and dev environments, why? Well, create scripts to build Window servers created with custom images is actually a big deal, so: - disaster recovery is the first thing you should have in mind, where is the second. - the right strategy is to preserve the business, not going to the cloud. I always had fun building my own environments. After the fun came the work and, economically speaking, managing and maintaining those environments has always been an advantage for me. If however, I consider the time it takes to understand and fix problems in VMWare, or SQL optimization, or configure Microsoft AD, Nginx, and others..., I don't know how much I would recommend building everything from scratch by renting some servers.

CyanLite4 7y ago

Surely this is satire...right?

refrigerator 7y ago
Haha, did a CMD+F for "satire" as soon as I started reading the comments. This was the giveaway for me: Don't go to the cloud. It will force you to... over-complicate your infrastructure to incredible degree. It is truly a piece of shit and will just force you to design systems in a horrible way."

vmizzle 7y ago

I run my own mini-cloud at home (42u rack with way too much hardware). I do my best to follow best practices with regard to hardware, backups, redundancy, etc. I keep downtime to a minimum (Plex is life, yo). I've been doing it for around 20 years, so I'm fairly confident I know what I'm doing. As a Director of Technology for a real company, I have never, nor would I ever dream of even considering moving to bare metal for ANYTHING mission critical. The sheer number of possible catastrophies that can occur at the infrastructure level would have me shitting my pants 24/7. Could we hire someone (or a team) to do it? Yes. But EC2 and similar infrastructure beats the pants off those salaries any day.

dekhn 7y ago

The reason I run in the cloud to be close to the object storage and average bandwidth between machines. I run big batch jobs and setting up a full cluster filesystem and fast network is non-trivial. Recently, using S3 with VMs that have 25Gbit network, I can easily make 'aws s3 cp' (after tuning) generate 10+Gbit of traffic reliably.

gioscarab 7y ago

Our times demonstrate that the effective social cost of "the cloud" for sure overtake the provided convenience.

cateye 7y ago

Over generalization is the main problem here. It all just depends on the context. Going to the cloud, can really make a lot of sense. For example, if you are a start up that is targeting a mass market and need to scale quickly in multiple regions... Or, if you are the CTO of a large company and don't want to spend time and focus on running your own data center and everything involved with it... There are also many situations where running your own collocated dedicated server is the best choice. Keeping it lean and mean and removing as much as possible dependencies, complexity and layers does have benefits. It is hyperbolic to state that something is just shit for everyone in all circumstances.

bcheung 7y ago

So what's the alternative to not using CDN's for those of us who don't have 100% of our customers in or near Germany?

quickthrower2 7y ago

I find using the cloud too close to the metal for me. I'm using Netlify for hosting. I get impression they might be using AWS under the hood.

jwp23 7y ago
I interviewed with them (didn't get the job). If I understood it correctly, they pretty much use every cloud under the sun to get full coverage for their CDN.

awinter-py 7y ago

Would certainly consider a 'fewer bigger boxes, more hands-on' approach, BUT ONLY for clients that had good administration talent in-house and were already debugging the managed services a lot anyway bc they're pushing the envelope. Managed services aren't perfect and I've seen them go bad, esp for things like failover. When they hiccup (either perf-wise or downtime) it's hard to debug effectively depending on what logs / stats the vendor makes available.

emmelaich 7y ago

Does anyone have a good cloud cost calculator that they can recommend? Web searching shows a few but I'd love some recommendations. Ideally an online spreadsheet, but excel would do too.

RandomTisk 7y ago

As expensive as AWS is, at my company (Fortune 500) company with at least 2 major data centers in the US for my division alone, serving around a hundred clients and thousands of web sites and applications, we found that AWS can still be a lot cheaper than a poorly run private data center. My department is technically a cost center, like most of Security, but the charge-backs we get from our own data center for various services are insanely high compared to the same services we can procure from AWS and Azure. We've been steadily moving our services out of our own data centers and into the cloud and so far, my life is a lot better. Before, just getting someone to run down a blinking disk light on a server was a chore. Getting them to rack a new server took several days or weeks. Doing anything, any changes, took several days minimum. Now, we can do it all much faster, and from all accounts of my management, it's cheaper. I don't doubt you can run your own data center cheaper, but not for my company.

deleted 7y ago

[deleted]

namesbc 7y ago

This reads like a parody. Please tell me this is a parody?

geekpowa 7y ago

Part of Friendsters death spiral was the decision to move to AWS, which was a technical disaster. On Cloud vs baremetal arguments. I respect people find cloud works and is valuable for them. This inspite of my own personal experiences, direct or indirect with cloud have been adverse. I agree with author, for me, for dollar and time effort, baremetal is just better economically and performance. Most recent. A web service I use heavily migrated from bare metal to cloud and their performance / reliability dropped significantly with numerous outages which tied me up on the phones to my clients. I gave them 6 months to get their act together, as a loyal client then gave up and moved elsewhere. Being a reasonably big client their sales tried to retain then win me back. Conversations were always the same: Me: Moved because you guys stopped being reliable, reliability is important to me. Ever since you moved to AWS you have been unreliable. Sales: we had to move to scale and grow Me: whats the point in moving if moving weakens your product. there are other ways to scale. Sale: we had to move to scale and grow

Bootwizard 7y ago

"Old man yells at cloud"

013a 7y ago

I really think there's more to this than most people will admit. Our very early, small startup (4 engineers, low six figures ARR) accumulates $4,000+/month in cloud bills. Its very difficult to manage cloud costs on a small engineering team. One of our recent examples is with SQS. Our SQS bill is something like $50/month. We process ~100k messages every month. Why is it so expensive, then? It isn't because its processing tons of messages; its because we use open-source polling libraries that are pretty aggressive with their poll rate (then multiply that out by N instances of the poller and there's your bill). I find it fun to think about "what could I get for $50/month" in regards to a queuing solution. As an extreme counterpoint, Redis running on a raspberry pi in the closet would be a rounding error in our cloud bills over the span of a year. The gray area between these two extremes is where the value lies, and I think that's the author's point: redis on a VPC at Hetzner or DigitalOcean. Billions upon billions of messages processed every month at a fraction of the cost. So, why pay AWS? There's not NO reason. Reliability is the biggest one I think. But herein lies what I believe to be the intrinsic problem with the cloud: No one can put a price on reliability. If I pay an extra $10k a year to be 99.99999 available (~10s/year), versus 99.99 (~1hr/year), how much extra value does my business receive? No one even CARES what that number is. They just know that more nines is better, so we gotta pay it. But we won't pay too much for it. At some point, it crosses some TOTALLY arbitrary threshold in the CTO's head where its "too expensive" and we need to reduce costs. But, then, we're so bought in that we can't. We can tweak poll rates, we can buy reserved instances, maybe see a 20% reduction, but the pattern is set. We're just a serf to Amazon now. The other quoted reason is always "the cloud is easier". "You'll save money on HR." "Your engineers can focus on the product." I think Amazon has shouted this line so much from the rooftops that even very smart engineering managers have begun to believe it. Heroku was really the last platform I've used that felt like it could fulfill this promise to any degree of meaning. It really did everything, but AWS has managed to create a network of services that is so fundamentally complex that even when they release services to "simplify" things they actually create more complexity (beanstalk, amplify). You're not getting around having an engineer spending a decent chunk of her time managing this complexity (building CI pipelines, managing IAM permissions, alerting strategies, centralized log ingestion, autoscaling, cost management, encoding everything into CloudFormation, ugh maybe terraform will be easier, figuring out the madness behind VPCs because some random CF stack you found that you thought could automate something for you just spat out an egress-only internet gateway and what the fuck even is that I'm just one person why did I need that). I've been working in AWS since... the beginning. There are three good AWS services: S3, SQS, and EC2 (in that order). Everything else ranges somewhere between "Fuck it, I guess I'll use cloudwatch" and "what was the rekognition team smoking when they built this, and are they getting a daily dose of it through the vents because its been years and its still not better." The best part about those "good" services is that they're all solved problems available on a wide variety of cloud providers. Block Storage, Queues, and VMs. I think that's what the author is getting at, in a weird way. The cloud is fine. Managed services on top of open source software is probably fine. Even closed-source managed services can be fine, if they're so stupidly simple that you could explain it to your 8 year old and he'd understand it. But the vast majority of AWS (or Azure, or Google Cloud) is none of these things, and that's the stuff that you should avoid. Or, if you want to save some money, head over to Hetzner or Digital Ocean.

nh2 7y ago

This is good advice overall, in my experience. Most of the small/medium sized companies I've consulted for could have been easily served by a triple-redundancy Hetzner dedicated server setup for 200 EUR/month. Yet they they chose AWS and paid thousands per month because "that's what everybody does". I have not regularly encountered actual calculation or other analysis as basis for this decision. Many people here and in industry seem to think that it's either AWS or "build your own datacenter or colocation and maintain machines". Be reminded: This is incorrect. The post recommends renting dedicated servers from a server provider (in this case Hetzner), which includes all hardware handling personnel, hardware replacement, power, and so on, for 50 EUR/month per big-sized server. Others complain about the "too points in the list". In practice, all these points have to be addressed analogously in an AWS deployment as well, and require approximately as much knowledge as the "sysadmin you'd have to hire". It's a myth that you can just use AWS without reasonable sysadmin knowledge. As a "DevOps engineer" building sophisticated projects on AWS, you usually have to have a superset of typical sysadmin knowledge. Regarding scalability: For me, Hetzner has reliably fulfilled orders for new dedicated servers within 15 minutes (!) to ssh. In contrast, I've had cases where the AWS support request to raise the limit of 0 big machines in a region to 1 has taken more than 2 business days ("a request of this size has to be escalated"). With a reasonable dedicated server setup, many companies don't need to scale for a while, because you can go a long way with ~5000 SQL inserts that a modern dedicated machine can achieve on an SSD. Again others highlight the cloud for the available internal bandwidth. Note Hetzner gives you 10 Gbit/s as a 39 EUR/month upgrade. Such bandwidth is reliable; I have measured that I can reliably transfer at Gigabit speed even across the oceans. A 1 Gbit/s dedicated server with unlimited traffic for 50 EUR/month (at Hetzner) can transfer ~260 TB per month egress. Per AWS calculator, this would cost 18700 USD there -- that's a factor of 375x on traffic cost. In general, in my experience, the factors you pay for common cloud providers over rented dedicated servers are: * 10-20x for compute * 2-4x for storage (this computation includes multi-AZ redundancy) * ~300x for traffic The article's point about not being able to use strace/gdb/iostat to debug problems on hosted services is also a good one. For example, we have found AWS EFS to have abysmal performance and could not address this problem. Using CephFS on dedicated infrastructure instead, any bottlenecks could be quickly debugged and performance was great. Similarly, the point about simplicity is good. Having just simple Linux processes running under systemd on a machine, instead of processes-in-docker-on-kubernetes-in-VMs-on-EC2, removes complexity and makes systems many times simpler and easier to debug, and thus much faster to iterate on. The only point of the article I don't quite a agree with is to not use CDNs. If you want fast loading times over HTTPs (multiple roundtrips), you need to use a CDN, because of physics. (But you can build a CDN with dedicated servers very easily.) I have successfully run a triple-redundancy dedicated Hetzner setup over the last 2 years (managed as infrastructure-as-code using NixOS) with a High-Availability database (postgres with Stolon), High-Availability file system (CephFS), systemd and round-robin DNS. So basically, everything the post recommends. It's been the simplest to manage deployment I've used for years, compared to many cloud setups I've worked on. It's had higher uptime than AWS in the last 2 years. It has extremely predictable performance. It costs 200 EUR/month. It's a pleasure for me as a programmer. In summary, I think the article is on-spot (even though it uses odd language). There remains one significant benefit of big cloud providers like AWS: Being able to spawn whatever you need via /one/ unified API in very many locations across the world. If you need something that must be distributed across as many data centers as possible, this is quite a plus. But very few people actually need this.

manigandham 7y ago

So just get a single VM from a cloud provider then. They are all perfectly fine as IaaS if you want to manage it that way, and small VMs are cheap. This seems to be a massive misunderstanding of what "cloud" even is.

valenciarose 7y ago

Even for large organizations, the most compelling argument for the cloud is capital structure and the risks associated with holding assets that may not prove to be productive. We inherently understand the advantage of the cloud for startups. It preserves capital and can be used to links expense to generated value (user engagement, etc) if not actual revenue. What may be less obvious is that the same math applies for at least some activities of larger organizations. What goes into a corporate data center is effectively a distressed asset the instant the capacity isn't needed. Forecast risk is very real, and even conservative estimates of that risk can radically change the risk adjusted return on the capital investment.

jasonvorhe 7y ago

This is the author's take on syntax highlighting: https://txt.black/~jack/syntax-highlighting.txt https://txt.black/~jack/syntax-highlighting.txt At least he's consistent.

adchari 7y ago

I honestly wonder if this is satire. I feel like this is one of those situations where the reason the writer doesn't like it is solely that he doesn't understand the true use case and reasoning behind moving off-prem. It isn't just the cost of servers, it's the fact that AWS or Azure gives you a variety of simple ways to configure and integrate services, without paying an entire team of ops people for putting servers in racks. It's more efficient and reduces overhead, which saves the kind of company that spends 6 figures on AWS costs a huge sum of money too.

lunias 7y ago

I really respect the desire and the ability to do it all yourself. I expect that you'll see real gains over the cloud (up until a point) assuming that you're knowledgeable and your environment permits you to work quickly and independently. https://github.com/jackdoe https://github.com/jackdoe "old man yells at cloud" - I loled