7 ms·
THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at
by sgarland 1y ago
THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at best, they’ll argue that “this way, they don’t have to manage infrastructure.” Except you do - you absolutely do. It’s just been partially abstracted away, and some parts like OS maintenance are handled (not that that was ever the difficult part of managing servers), but you absolutely need to configure and monitor your specific XaaS you’re renting.
- huflungdung 1y agoWhat I say is that we massively underestimate just how fast computers are these days
- ahartmetz 1y agoIndeed - they are incredibly fast, it's just buried under layers upon layers of stuff
- f1shy 1y agoIn 2010 I was managing 100 servers, with many Oracle and Postgres DB, PHP, Apache, all on Solaris and Sun HW. I was constantly impressed by how people were unable to do more or less correct estimations. I had a discussion with my boss, he wanted to buy 8 servers, I argued one was more than enough. The system, after growing massively, was still in 2020 managing the load with just 3 servers. So I would argue, not only today, but 15 years ago already.
- hombre_fatal 1y agoOn the other hand, there is a real crossroad that pops up that HNers tend to dismiss. A common story is that since day one you just have lightweight app servers handling http requests doing 99% I/O. And your app servers can be deployed on a cheap box anywhere since they're just doing I/O. Maybe they're on Google Cloud Run or a small cluster of $5 VPS. You've built them so that they have zero deps on the machine they're running on. But then one day you need to do some sort of computations. One incremental option is to create a worker that can sit on a machine that can crunch the tasks and a pipeline to feed it. This can be seen as operationally complex compared to one machine, but it's also simple in other ways. Another option is to do everything on one beefy server where your app servers just shell out the work on the same machine. This can be operationally simple in some ways, but not necessarily in all ways.
- vidarh 1y agoMost younger devs just have no concept on how limited hardware we ran services on... I used to run a webmail system with 2m accounts on hardware with less total capacity (ram, disk, CPU throughput) than my laptop... What's more: It was a CGI (so new process for every request), and the storage backend spawned separate processes per user.
- robotresearcher 1y agoNo worries, another fifteen layers of software abstraction will soak that up pronto.
- oceanplexian 1y agoIf you know anything about hardware and look at the typical instances AWS is serving up (other than the ludicrously expensive ones) it's Skylake and older. I think people have a warped perception of performance, if only because the cloud providers are serving up a shared VM on equipment I'd practically class as vintage computing. You could throw some of the same parts together from eBay and buy the whole system with less than a few months worth of the hourly on-demand cost.
- skydhash 1y agoI look at what I can do with an old mac mini (2011) and it’s quite good. I think the only issue with hardware is technical maintenance, but at the scale of a small companies, that would probably be having a support contract with Dell and co.
- amluto 1y agoSmall companies should never forget to ask Dell, etc for discounts. The list prices at many of these companies are aspirational and, even at very small scale, huge discounts are available.
- hrimfaxi 1y agoDepending on your regulatory environment, it can be cost-effective to not have to maintain your own data center with 24/7 security response, environmental monitoring, fire suppression systems, etc. (of course, the majority of businesses are probably not interested in things like SOC 2)
- wongarsu 1y agoThis argument comes up a lot, but it feels a bit silly to me. If you want a beefy server you start out with renting one. $150/month will give you a server with 24 core Xeon and 256GB of RAM, in a data center with everything you mentined plus a 24/7 hands-on technician you can book. Preferably rent two servers, because reliablity. Once you outgrow renting servers you start renting rack space in a certified data center with all the same amenities. Once you outgrow that you start renting entire racks, then rows of racks or small rooms inside the DC. Then you start renting portions of the DC. Once you have outgrown that you have to seriously worry about maintaining your own data center. But at that point you have so much scale that this will be the least of your worries
- hrimfaxi 1y ago> This argument comes up a lot, but it feels a bit silly to me. If you want a beefy server you start out with renting one. $150/month will give you a server with 24 core Xeon and 256GB of RAM, in a data center with everything you mentined plus a 24/7 hands-on technician you can book. What's the bandwidth and where can I rent one of these??
- wongarsu 1y agoHetzner [1]. Bandwidth is 1 GBit/s. You can also get 10 GBit/s, that's hidden away a bit instead of being mentioned on the order page [2] 1: https://www.hetzner.com/dedicated-rootserver/matrix-ex https://www.hetzner.com/dedicated-rootserver/matrix-ex 2: https://docs.hetzner.com/robot/dedicated-server/network/10g-uplink/ https://docs.hetzner.com/robot/dedicated-server/network/10g-...
- BobbyTables2 1y agoHave always felt the same. I’ve seen an entire company proudly proclaim a modern multicore Xeon with 32GB RAM can do basic monitoring tasks that should have been possible with little more than an Arduino. Except the 32GB Xeon was far too slow for their implementation...
- MattGaiser 1y agoHow did they implement it? That's horrendous.
- BobbyTables2 1y agoJava - but done so horribly even the devil would be aghast. There were thousands of threads, murmuring an incessant hum fully occupying a few cores when absolutely nothing was happening. Over 20GB RAM actively used at idle. I think booting the application took almost half an hour at one point, using a local SSD. I’m fairly certain at no point in my career could I ever have replicated such a monstrosity.
- wltr 1y agoI swear, before I finished reading your comment, this thought jumped into my mind: ‘oh my, they do host everything with a computer similar to my [pretty old by the way, but still beefy] for-work computer! Impressive!’ Which is, I still believe is perfectly possible to do. Then, I was ‘what?!’
- icedchai 1y agoLet me guess: database tables with no indexes, full scans everywhere?
- gaoshan 1y agoAnyone that says, "they don’t have to manage infrastructure" I would invite them to deal with a multi-environment terraform setup and tell me again that about what they don't have to manage.
- darkwater 1y agoThose are the ones that also usually tell you you can just stitch together a few SaaS products and it's magic.
- galaxyLogic 1y agoIt's much the same mindset as: "Vibe-coding can do it for you so you don't have to program"
- darkwater 1y agoYep. Low-effort, shallow knowledge, risk-taking guys.
- ta12653421 1y agoYou are an outdate Boomer!!! I have 37 agents doing that for me!!!!!11^ LOL
- simianwords 1y agoWhile terraform is not ideal it is much much more easy to deal with managed services in AWS than to deal with on premises baremetal servers. Most are biased because they like dealing with the kind of issues in on premises. They like dealing with the performance regressions, heat maps, kernel issues etc. Because why not? You are a developer and you need some way to exercise your skills. AWS takes that away and makes you focus on the product. Issues arising from AWS only requires you talking to support. Most developers get into this industry for the love of solving these problems and not actually solving product requirements. AWS takes away what devs like and brings in more "actual" work.
- 1y ago
- macNchz 1y agoWorking on various teams operating on infrastructure that ranged from a rack in the back of the office, a few beefy servers in a colo, a fleet of Chef-managed VMs, GKE, ECS, and various PaaSes, what I've liked the most about the cloud and containerized workflows is that they wind up being a forcing function for reproducibility, at least to a degree. While it's absolutely 100% possible to have a "big beefy server architecture" that's reasonably portable, reproducible, and documented, it takes discipline and policy to avoid the "there's a small issue preventing {something important}, I can fix it over SSH with this one-liner and totally document it/add it to the config management tooling later once we've finished with {something else important}" pattern, and once people have been doing that for a while it's a total nightmare to unwind down the line. Sometimes I want to smash my face into my monitor the 37th time I push an update to some CI code and wait 5 minutes for it to error out, wishing I could just make that band-aid fix, but at the end of the day I can't forget to write down what I did, since it's in my Dockerfile or deploy.yaml or entrypoint.sh or Terraform or whatever.
- darkwater 1y agoYou have to remove admin rights to your admins then, because scrappy enough DevOps/platform engineers/whatever will totally hand-edit your AWS infra or Kubernetes deployments. I suffered that first hand. And it's even worse that in the old days, because at least back in the day it was expected.
- jkaplowitz 1y agoOr at least you have to automatically destroy and recreate all nodes / VMs / similar every N days, so that nobody can pretend that any truly unavoidable hand-edits during emergency situations will persist. Possibly also control access to the ability to do hand edits behind a break-glass feature that also notifies executives or schedules a postmortem meeting about why it was necessary to do that.
- rustystump 1y agoOh no it ran out of disk space because of bug! I will run a command on that instance to free it rather than fix bug. Oh no error now happens half of the time better debug for hours only to find out someone only fixed a single instance… I will never understand the argument for cloud other than bragging rights about burning money and saving money which never shoulda been burning to begin with.
- chamomeal 1y agoDocker compose on a couple nice VPS’s can do a LOT
- vidarh 1y agoI do consulting in this space, and I'm torn: I make much more money managing infrastructure from clients who insist on AWS. But it's much more enjoyable to work with people who knows how to keep it simple.
- yoyohello13 1y agoI worked on a project for my company (a low volume basic web app) and I suggested we could just start the whole thing on one server. They brought on some Azure consultants and the project ballooned out to months of work and all kinds of services. I’m convinced most of the consultants were just piling on services so they could make more money.
- treis 1y agoI joke with my boss that all our shit ends up running on a single server in some Amazon data center. It's probably not true but if you add up everything we do it's pretty close to one big server.
- selcuka 1y agoIt's probably not true, but the end result would be the same even if it was true.
- vidarh 1y agoIt's probably true. The biggest challenge with doing the right thing in this space is that the sales job is hard, time consuming and so expensive that it's a lot easier to make it profitable if you make projects balloon like that. The sales effort is much the same. I've been offering to help people cut costs for a while, and it's a shockingly hard sell even with offers of guarantees, so we're deemphasizing it to focus more on selling more complex DevOps assistance and AI advice instead... Got to eat (well, I do much better than that, but anyway), but I refuse to over engineer things just to make more money.
- yoyohello13 1y ago
- immibis 1y agoYou can get a server now with, like, five hundred cores and a fifty terabytes of RAM. It's expensive, but you can get one. A used server with sixty cores and one terabyte of RAM is a lot cheaper. Couple thousand bucks. I mean, that's a lot of bucks, but a terabyte for only four digits?
- immibis 1y agoYou can get a server now with, like, five hundred cores and a fifty terabytes of RAM. It's expensive, but you can get one. A used server with sixty cores and one terabyte of RAM is a lot cheaper. Couple thousand bucks. I mean, that's still a lot of bucks, but a terabyte for only four digits?
- yibg 1y agoI think it depends on what you are optimizing for. If you are a VC funded startup trying to get to product market fit, spending a bit more on say AWS probably makes sense so you can be “agile”. The opportunity cost there might outweigh infrastructure cost. If you are bootstrapped and cost matters a lot, then different story.
- frde_me 1y agoOn the other hand, I know a lot of people who spend more time / salary messing around with their infra than the couple hundred bucks they've saved from not pressing a couple of buttons on vercel / cloudfare There's a time and place for just deploying quickly to a cloud provider versus trying to manage your infra. It's a nuanced tradeoff that rarely has a clear winner.
- citizenpaul 1y agoThe problem with onsite or colo is always the same. You have to keep fighting the same battle again and again and again. In 5 years when the servers need replaced even though you have already proven it saves orders of magnitude in costs. I've never once been rewarded for saving 100k+ a month even though I have done exactly that. I have been punished by having to constantly re justify the decision though. I just don't care anymore. I let the "BIG BRAIN MBA's" go ahead and set money on fire in the cloud. It's easier for me. Now I get to hire a team of "cloud architects" to do the infra. At eye bleeding cost increases for a system that will never ever see more than a few thousand users.
- adamcharnock 1y agoI totally agree. So much complexity for generally no good reason [0]. I saw so much of this that I ended up starting a company doing the exact opposite. I figured I could do it better and cheaper, so that that's now what we do! If anyone wants to bail out of AWS et al and onto a few beefy servers, save some money, and gain a DevOps team in the process, then drop us an email (adam at domain in bio). [0] My pet theory about the real reason: the hyper-scalers hire all the engineers who have the skills to deploy-to-a-few-beefy-servers, and then charge a 10x multiplier for compute. Companies can then choose between impossible hiring, or paying more. Paying more is easier to stomach, and plenty of rationalisations are available.
- sgarland 1y ago> My pet theory about the real reason: the hyper-scalers hire all the engineers who have the skills to deploy-to-a-few-beefy-servers, and then charge a 10x multiplier for compute. This is also my pet theory, and it’s maddening. They’ve successfully convinced an entire generation of devs that physical servers are super scary and they shouldn’t ever have to look at them.
- resonious 1y agoI'll play devil's advocate a little bit here. But to be clear, I hate AWS and all of their crazy concepts and exorbitant pricing, so ultimately I think I'm on your side. OS maintenance honestly is a bit hard for me. I need to know what to install for monitoring, I need to maintain scripts or Ansible playbooks. I need to update these and make sure they don't break my setup. And the big kicker is compliance. I always work under SOC2, ISO27001, PCI-DSS, HIPAA, you name it. These require even more things like intrusion detection, antivirus, very detailed logging, backups, backup testing, web application firewall. When you just use AWS Lambda with DynamoDB, the compliance burden goes down a lot.
- sgarland 1y agoYes, you need to write Ansible initially. But honestly, it’s not that much for your average application server. Turn on unattended-upgrades with anything critical to your application blacklisted, and you won’t have to touch it other than to bump version pins whenever you make a new golden image. Re: compliance, other than SOC2 being a giant theater of bullshit, agreed that it adds additional work. My point is that the claims of “not having to manage infrastructure” is highly misleading. You get to skip some stuff, yes, but you are paying through the nose in order to avoid writing some additional config files.
- digdug2 1y ago[dead]