9 ms·
I agree entirely. I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you bui
by movedx 4y ago
I agree entirely.
I like to call what the author is referring to as, "What-If Engineering". It's the science of thinking you'll be Google next week, so you build for that level of scale today. It involves picking extremely complicated, expensive (both in compute and skilled labour) technologies to deploy a Rails app that has two features. And it all boils down to, "But what if..." pre-optimising.
It happens at all levels.
At the individual unit level: "I'll make these four lines of code a function in case I need to call it more than once later on - you know, what if that's needed?"
It also happens at the database level: "What if we need to change the schema later on? Do we really want to be restricted to a hard schema in MySQL? Let's use MongoDB".
What's even worse, is Helm and the likes make it possible to spin up these kinds of solutions in a heart beat. And, as witnessed and evidenced by several comments below, developers think that's that... all done. It's a perfect solution. It won't fail because K8s will manage it. Oh boy.
Start with a monolith on two VMs and a load balancer. Chips and networks are cheaper than labour, and right now, anyone with K8s experience is demanding $150k + 10% Superannuation here in Australia... minimum!
https://martinfowler.com/bliki/MonolithFirst.html https://martinfowler.com/bliki/MonolithFirst.html
- iso1631 4y agoYou're forgetting that many people will want to use K8s for a project because they want it on their CV to get the high paying jobs. I saw the term on HN a couple of weeks ago -- CVOps
- movedx 4y agoI'm not forgetting that fact, I'm simply choosing to ignore such people. They're not really what the industry is about. That's not in the spirit of a healthy society. That's just leeching. Good luck to them, but they're not going to occupy time and space in my mind.
- iso1631 4y agoMaybe it's just my organization, but I see the behavior across the corporation far more than I'd like, and inevitably these people move on leaving a complex mess in their wake that long term support staff have to deal with. We seem to mostly manage to avoid that in my department, but we have a very low turnover.
- movedx 4y agoSounds rough, buddy! Sorry to hear that. I hope you're being well compensated.
- jjav 4y ago> They're not really what the industry is about. We'd like that (I'd like that), but resume-driven choices are a very large driver of technology direction, unfortunately. It means those of who want to build something very maintainable and very stable using the most boring (stable, secure) technology possible are often outnumbered.
- movedx 4y agoSigh. Tell me about it :-/
- NoGravitas 4y agoAnd underpaid.
- Jistern 4y ago
- danielvaughn 4y agoI've told this story before on HN, but a recent client of mine was on Kubernetes. He had hired an engineer like 5 years ago to build out his backend, and the guy set up about 60 different services to run a 2-3 page note taking web app. Absolute madness. I couldn't help but rewrite the entire thing, and now it's a single 8K SLOC server in App Engine, down from about 70K SLOC.
- javajosh 4y agoWhat's your plan when (not if) Google deprecates GAE?
- gizzlon 4y agoNot OP, but App Engine have actually been around for a long time. Also, there's no inherent lock-in, you can basically just deploy it somewhere else. Data is where the lock-in lies. Moving can be hard if you use proprietary databases. Can still be worth it.
- danielvaughn 4y agoYeah the one good decision by that former dev was to use Mongo Atlas, so the data layer is completely decoupled from GCP.
- javajosh 4y ago>App Engine have actually been around for a long time That means very little, I hope you realize. Reader, Voice, Chat, etc.[0] were all around a long time. >Also, there's no inherent lock-in GAE has plenty of proprietary APIs you can depend on. Whether or not you do is up to the programmer. 0 - A comment below notes that voice and chat aren't deprecated yet. Voice was announced deprecated, and google has had so many chat apps I'm not sure which ones are gone. Anyway, here is a more complete list of things Google has abandoned: https://killedbygoogle.com/ https://killedbygoogle.com/
- joshuamorton 4y ago
- Jistern 4y ago>> I agree entirely. I agree entirely too. >> Start with a monolith on two VMs and a load balancer. Chips and networks are cheaper than labour, Kudos to you! You are a dangerous man for you opine the truth. My advice is generally, "Build something. Then see if you can sell it." or "Sell something and then go build it." Either way, it all starts soooo small that the infrastructure is hardly a problem. If you "get lucky" and things really take off. Sure. Yeah. Then get a DevOpps superstar to build what you need. In reality, your business will very probably fail.
- politician 4y agoYou can’t just hire 1 DevOps superstar though because they need to sleep and not burnout. You’ll need ~7 people on a rotation if you need to really support anything worth really supporting. DevOps is about giving Developers a dedicated System Operations job for some small fraction of their time.
- Jistern 4y agoGood grief. First and foremost I was simplifying to make a point. >> You can’t just hire 1 DevOps superstar Secondly, that assertion is not necessarily true. Obviously, you should just hire 1 DevOps superstar... in some cases. Don't nitpick and don't argue foolishly.
- politician 4y agoI’ve run DevOps and know from experience the pitfalls. I’m sorry that you’ve interpreted my general agreement and elaboration of your comment as nitpicking foolishness.
- roflyear 4y agoAt least in the sense of code you arent doing any real harm I can think of and there are other benefits like testing and organization.
- osigurdson 4y ago>> Helm and the likes make it possible to spin up these kinds of solutions in a heart beat Genuine question, why is this bad? Is it because k8s can spin it up but becomes unreliable later? I think the industry wants something like k8s - define a deployment in a file and have that work across cloud providers and even on premise. Why can't we have that? It's just machines on a network after all. Maybe k8s itself is just buggy and unreliable but I'm hopeful that something like it becomes ubiquitous eventually.
- movedx 4y agoOh it's most certainly NOT bad! It's very, very good. But it's not the end of the story. Imagine if you could press a button in a recipe book and a stove, pan, some oil, and some eggs appeared and the eggs started cooking... amazing! But that's not the end of the story. You don't have scrambled egg yet. There's still work to be done and after that, there's yet more work to be done - the washing up being one of them. It's everything that comes afterwards that gets neglected. Not by everyone, granted, but by most.
- jbverschoor 4y agoWe had J2EE almost 30 years ago. 1 file which described everything and contained everything
- osigurdson 4y agoAccording to the following link, that literally sounds nothing like Kubernetes. Perhaps a more appropriate analogy to older tech is something like LSF or Slurm. https://www.webopedia.com/definitions/j2ee/ https://www.webopedia.com/definitions/j2ee/
- scheme271 4y agoLSF and Slurm is still around and kicking on HPC systems. But they're nothing like kubernetes. Maybe close to k8s batch jobs but that's it.
- preommr 4y ago> It's the science of thinking you'll be Google next week, There's other reasons to use K8s than just thinking of massive scale. Setting up environments becomes a massive PITA when working directly with VMs. The end result is either custom scripts, which is a messier version of terraform, which ends up being messier than just writing a couple of manifest files for a managed k8s. > anyone with K8s experience is demanding $150k + 10% Superannuation here in Australia... minimum! sheds a tear for CAD salaries and poor career decisions
- ryanbrunner 4y ago> Setting up environments becomes a massive PITA when working directly with VMs. The end result is either custom scripts, which is a messier version of terraform, which ends up being messier than just writing a couple of manifest files for a managed k8s. The author advocates using a high-level PaaS. For sure working directly with VMs is the wrong answer to premature optimization, but as an early stage startup, there's plenty of services around that you basically just need to point your Git repo at and you'll have a reasonable infrastructure set up for you.
- tenfourty 4y agoOP here, I agree. You can get super far with a service like fly.io/Heroku/Netlify/Vercel etc. (pick the one that works with your stack). VMs are an anti-pattern as well for the early stages of an application or startup.
- mbonca 4y agoI find those solutions end up being harder once you have to integrate something new you didn't know you needed. And the cost, at least for something like Heroku, is MORE than using something like GKE where I don't need a new dyno for every service. I consider GKE (and DO's and AWS's equivalent k8s solutions) on the same platform level as a fly.io/Heroku/etc.
- movedx 4y ago> Setting up environments becomes a massive PITA when working directly with VMs. I guess I've just never found this to be true. My main goal when engineering a solution is always, "Keep It Super Simple (KISS), so that a junior engineer can maintain and evolve it." Working directly with operating systems, VMs, networking, etc. (purely in Cloud - never on-prem... come on it's 2022!) is the simplest form of engineering and is much easier than most claim.
- morelish 4y ago> I'll make these four lines of code a function in case I need to call it more than once later on - you know, what if that's needed? The code example is not always right. Beware, if you know it will be needed, you might as well make it a function now. Likewise if you think probably it will be needed, why not make it a function now? It’s not a good review comment or rejection to say “yeah but I don’t want to do that because it’s not yet needed”. Sure, but what if you are just being lazy and you don’t appreciate what it should look like long term? The “I don’t want to write a function yet not needed” is not a clear cut example.
- _gabe_ 4y ago> Sure, but what if you are just being lazy and you don’t appreciate what it should look like long term? I wasn't aware that some devs have a side hustle as fortune tellers? On a more serious note, you should take a look at Sandi Metz's "the wrong abstraction". https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction
- woojoo666 4y agoYou're making a gamble either way. The article you linked is correct that duplication is usually cheaper than abstraction. So if you really have no idea what your code will do in the future, then cheaper is the way to go. But an experienced dev can start to fortune-tell, they know what parts tend to be re-used, which abstractions are common and powerful. And if you also plan your architecture before you code, you can also see what abstractions might be needed ahead of time. If you are sure an abstraction is better, then duplication is tech debt. A simple example. If you are making a multi-page website that contains a header on all pages, you can separate the header into a component from the get go, instead of duplicating the header for each page and then abstracting it in the future (where it starts to become more work).
- d23 4y agoThe truth is at scale the last thing you want is a nest of unmanaged complexity, so it’s also the wrong instinct there. It’s usually contact with the real world that dictates what needs the extra engineering effort, and trying to do it ahead of time just means you’ll sink time up front and in maintenance on things that didn’t turn out to be your problem.
- movedx 4y agoI think at scale, K8s is a good choice. I run a Discord server with like, 3,400 members now, and some of them are working at mental scale. They've claimed the same as you: K8s at scale is the only way. I would very likely agree in all cases. However, they only represent 4-5 users out of all 3,400. And that's the issue - only a small fraction of the industry operates at that scale :-)
- deleted 4y ago[deleted]
- mountainriver 4y agoSorry but managed k8s is really simple and wildly a better pattern than just running VMs. You don’t need google scale for it to help you, and spinning things up without understanding the maintenance cost is just bad engineering
- mirceal 4y agoNo, it's not. If going for managed services: A load balancer + an asg is stupid simple to setup and it just works.
- zinclozenge 4y agoHow do you deploy your code in this scenario, ssh into VMs?
- mirceal 4y agoLevel 1) you package your service into a zip/rpm/deb/etc and have an agent on the machine that periodically pulls Level 2) you pack your software into an ami and use the update the asg config. You can periodically "drain" the asg of old instances Level 3) you deploy your stack again with the new stack having the ami that you've build at level 2 referenced. You start shifting traffic between the old stack and the new stack. You monitor and rollback if something is wrong.