9 ms·

> So what have we now? A “mono repo” codebase, because clearly a Git repository per function in your system would be too much, a large deployment descriptor per

by cs702 6y ago

> So what have we now? A “mono repo” codebase, because clearly a Git repository per function in your system would be too much, a large deployment descriptor per fine-grained component, which Spring maybe called “Controller” but is now called “Function”, and instead of combining them all on your desktop, you send them off to someone else’s mega-mainframe [AKA "the cloud"]. You deploy, get an error message, and login to CloudWatch to see what actually happened - it’s all batch-driven, just like the bad old days, so progress is slow. At least we’re not having to walk to the printer on every try, but that pretty much sums up the progress of the last half century. Oh, and a “Function” will be able to handle a single request concurrently, so here’s your AWS hosting bill, we needed a lot of instances, I hope you won’t have a heart attack. Yes, we run workloads that your nephew can run on his Raspberry Pi 4, but this is the future of enterprise. I've mentioned as much to several C-suite executives and professional venture investors I know who tend to wax philosophical about "exponential growth in cloud services and big data" and whatnot. Invariably, their response has been to tell me that I just don't get it, or that I make no sense. They give me puzzled looks and maybe even get a bit exasperated with me. Surely others here on HN have run into similarly minded individuals too. They're not hard to find. Plus ça change, plus c'est la même chose.

mattmanser 6y ago
Everyone's chasing "scalable", mostly for no reason at all. There's also this enduring myth you see repeated even here of how much saving there is by not having "devops", when only very big companies needed them in the first place. A few minutes per month is the most devops any startup would need to do in reality running their own VMs or servers, after an hour or two of setup. And that hour or two is often less than faffing around getting all the labyrinth moving parts of AWS/Azure/etc. working. And the whole thing becomes a transparent farce when you look at providers like Azure, where their own admin interface is so slow and prone to random breaks you have to ask: How is this scalable? It's clearly not scalable for themselves, so how is it "scalable" for everyone else? There are benefits, mainly for me I'll use them for hassle free deploys[1], but the whole thing sometimes feels like a massive con to sell the performance of a potato by advertising it as a jet engine. [1] Like a month back I deployed a demo prototype web app I wrote in 3 hours onto Azure straight from github with like two clicks, and got the contract, can't really beat that.
- waheoo 6y ago
  Makes me think of those old "Webscale" skits on YouTube.
  - amiga-workbench 6y ago
    https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=bzkRVzciAZg https://www.youtube.com/watch?v=bzkRVzciAZg
- empthought 6y ago
  > A few minutes per month is the most devops any startup would need to do in reality running their own VMs or servers, after an hour or two of setup. Right up until something crashes or some hardware fails.
  - jiggawatts 6y ago
    > Right up until something crashes or some hardware fails. Unlike the public cloud where Nothing Ever Goes Wrong (tm). There are never brownouts. There are never latency spikes. No undocumented quota limits. Documented but confusing quota limits. Unexpected performance issues caused by other customers. Forced upgrades. Forced patching. Shock bills. No siree Bob, that kind of thing just doesn't go on in the public cloud! It's all rainbows and unicorns....
    - rpedela 6y ago
      You have all that DIY too, except maybe shock bills, but it's your problem to fix. Like with everything, extremes are bad and all managed or all unmanaged are both bad. My personal guideline is use off-the-shelf services if they are cheaper than DIY, including labor, or if DIY provides no competitive advantage and off-the-shelf is cheap enough. But if DIY is much cheaper or gives a competitive advantage, do it yourself.
    - empthought 6y ago
      I fail to see how this supports your claim that you can get by with "a few minutes per month" of devops. In fact, it's a classic red herring. Your original claim is quite unrealistic and you'd do well to retract it. It's bad advice both for public cloud and for self-managed efforts alike.
tyre 6y ago
I think what’s important to remember is that these kinds of “truth” are not the full story. Yes, a technology probably won’t live up to its marketing. Yes, what works is probably good enough. Yes, changing has switching costs and sacrifices institutional knowledge. Yes, we’ve seen these technologies before and switched away from them from various reasons. That’s all fine and sometimes frustrating as an engineer. But it’s not the only reason decisions are made. Sometimes pitching your exec team on The New Hotness gives cover for changes that actually do need to happen. Sometimes adopting The New Hotness is marketing to recruit engineers who want to try something new. Sometimes adopting The New Hotness is because some people are bored and uninspired in their jobs and just want to believe in something good again. The success of things like MongoDB and blockchain are not because they are better or even very good at most things people use them for. They’ll never live up to the hype. But people need things to get excited about. It’s psychological. Attacking them purely on technical points can lead one to miss the actual point.
- still_grokking 6y ago
  I don't think burning in sum hundreds of billions of Dollars (or even more) because "it's OK being irrational, because emotions" makes sense.
  - travbrack 6y ago
    The business case is that you won't be able to retain good engineers if you refuse to rewrite your AS400 COBOL app.
    - yters 6y ago
      I’ve wondered whether these good engineers are really a net benefit. They crank out something spiffy, move your big systems to it, and then disappear to work on the next hotness, leaving an already burdened tech team to support yet one more spiffy stack that is not quite as reliable or easy to use as we would like.
      - 6y ago
la6471 6y ago
TLDR
FpUser 6y ago
>"Invariably, their response has been to tell me that I just don't get it" I cured this one when my native server running on not so expensive multicore CPU with enough RAM left their cloudy setup in a dust performing more than a 100 times faster and still having huge potential reserve in vertical scalability. They were just astonished at what a single piece of modern hardware in combination with the native code could do.
- rangoon626 6y ago
  We need to meme c++ back into popularity
  - FpUser 6y ago
    I do not think there is any real need. To me it is doing just fine. It does not have to beat every other language into submission.
- vntok 6y ago
  Congratulations, from now on you are on call 24/7 in case anything happens to that machine. It is now your own pet. When RAM fails in unpredictable random ways, you'll spend hours reading the syslogs. When your SSDs fail, you'll spend hours ordering new ones, setting raid up and migrating the backups you've had to provision, automate, supervise and secure yourself. When the server gets even remotely suspected of having been hacked into, you must trash it overnight and never use any of it again.
  - chousuke 6y ago
    You talk like using a physical server somehow prevents building a highly available system or from using automation. Why would the hardware be any more of a pet than any random VM in the cloud? You could also just run a hypervisor on the host and get the benefits of virtualization along with full control of hardware. Sure, if a drive breaks you'd probably want to replace it instead of just replacing the entire machine; it's a thing you don't need to do with VMs, but that doesn't mean it's difficult. It definitely doesn't take "hours". Managing physical setups is different from managing fleets of cloud instances, but it's not necessarily automatically inferior.
mlthoughts2018 6y ago
I find your quote here and the follow up seriously unconvincing. You know what a miserable workflow is? Preparing all my build scripts according to the half baked build tool that our in-house Deployment team cooked up to use fabric to spray my deployment onto a bunch of nodes in our data center, all the tests and script linting pass, so then I go push it through our CI/CD pipeline (which can’t do the above tests for me because again this is all threadbare maintained by in-house tooling teams who could not be less aligned to end product deliverables), and lo and behold it breaks. Good thing I’m not paying so much for CloudWatch so I get to literally go ssh to the worker node of the CI/CD system where my task failed and cross my fingers that it wasn’t reclaimed for the next task and that the log files weren’t rotated off, grep my way to tracebacks that probably involve things no one on my team has ever heard of, spend 1/2 a day chasing them all down (if I’m lucky), and finally kicking the box to restart a build a few times and it goes. Then it goes to the in house homebrewed kubernetes deployment scripts where it promptly chokes on a secret that wasn’t configured right (thanks home made script linters), so then I have to go down a rabbit hole of eye melting Helm config of secrets for an hour but thank god I’m not paying for a simple cloud UI for this. So we get the secret cleared up and everything is great, until the kubernetes scheduler can’t schedule my pods because our data center is at capacity and now I literally can’t get work done until I chase down some probably incorrect and certainly unmaintained Grafana chart of who is using how much excess memory on which nodes, and wrangle someone with authority to force another team to stop eating lunch and downsize pods right now. After all this, 6-7 hours later and 20-30 labor hours later, my workload is finally deployed, but it will die overnight for an uncaught reason that doesn’t alert anyone, and even if it did, the team managing compute resources is in a permanent state of PTSD from never ending pages so they wouldn’t bother caring about why my workload failed and will just assume it’s application logic bugs or Docker bugs on my side for as long as they can just to avoid yet another un-debuggable failure landing on their plate, so that’ll be great for morale in the morning when I get to derail yet another day and spend time doing every type of ops work that is not my job and prevents us from delivering product goals. This is every company I’ve ever worked for or ever heard any colleagues, friends or associates work for, that tries to build their own SRE practice on top of data centers. Large cloud platforms alleviate so so much of this pain. The “back to the 70s” workflow sarcastically described in the quoted passage you shared sounds like an absolute dream compared to the nightmare of shipping code on internally created developer tool & deployment platforms interfacing to bare metal or space leased from data centers. I can’t give the cloud providers my money fast enough and I have only ever felt happy about that for years and years now.
deleted 6y ago
[deleted]
ta988 6y ago
The fun thing is that many C-people are fed the VC kool-aid about scalability. They want everything to br scalable because they want to cash in. Except that not many companies reach the level where you need to scale your tech stack "exponentially". And you usually don't want to scale the dumb prototype that ended up in prod anyway. The trap here is that those systems will allow you just that in a totally unauditable way... Good luck replacing parts of those systems as well when everything is interconnected...
mikepurvis 6y ago
"You deploy, get an error message, and login to CloudWatch to see what actually happened - it’s all batch-driven, just like the bad old days, so progress is slow." Even without cloud APIs and containers and whatever, I find there's a creeping thing that happens here which has been helpful to become consciously aware of and intentional about fighting back on. Basically, the thing is that wrappers become interfaces. You write tool A, and then tool B, and then some kind of script C which calls the two tools and produces a result. Then someone says "dang, so convenient", and sets up a Jenkins/Rundeck/Airflow/whatever job that runs script C against some source of inputs automatically— maybe there's even a parameterized build capability where arbitrary inputs may be submitted to it on demand. Before you know it, the fact that the original tools and script exist has been forgotten, because everyone's interface to this functionality is a handy webpage. And when someone wants to add more features, they don't even go back and change the script, they end up just adding it directly into the job definition. Before you know it, the full scope of what is being done on "the system" isn't even really available any more to be installed, invoked, and tinkered with locally, not in a meaningful way, anyway. It's not even necessarily all bad, but when I see this happening in a domain where I have the awareness of the underlying tools (or I may even have written or contributed to them), I do try to help demystify them by writing up docs on how to do the local setup, helping others get going with it, etc.
- pjc50 6y ago
  You can solve anything by adding a layer of abstraction - except having too many layers of abstraction. "Serverless" always reminds me of CGI/fastcgi, which was still a thing the last time I wrote web apps for money. It had the advantage of "scaling to zero" and could run a huge number of low-usage apps on a single 2000-era server.
flarg 6y ago
The cloud makes money for people so it's important. Would you rather that money went to shareholders or to wage earners?
heisenbit 6y ago
> I've mentioned as much to several C-suite executives and professional venture investors I know who tend to wax philosophical about "exponential growth in cloud services and big data" and whatnot. Invariably, their response has been to tell me that I just don't get it, or that I make no sense. They give me puzzled looks and maybe even get a bit exasperated with me. One word: Worldcom I still remember coming up to 2000 sitting in a meeting room of an equipment provider I recently joined looking at projections of market growth, competitors and our own sales and profitability. Shortly afterwards that part of the business was sold, I remained with the other non internet part and we sailed through the .com crash unscathed. A lot of what AWS offers makes no sense. A lot what some AWS customer's teams are building is overly complex and a horror show to maintain. But there is also value: Maintained images, backups, on demand resources and specialized resources. We hit one of the inflection points in Moore's Law and that drives us into horizontal scaling and specialized hardware. Like with Worldcom and the fantastic internet growth story back then there is a kernel of truth and value. Find people that understand it and be glad others are playing around and subsidizing the investments in the core.
bob1029 6y ago
I love the scalability "argument". The capabilities of a single thread of processing in a modern x86 CPU are underestimated by several orders of magnitude in most shops. The unfortunate part is that the more you chase the scalability with the cloud bullshit, the more elusive it becomes. If you want to push tens of millions of transactions per second, you need a fast core being fed data in the most optimal way possible. Going inwards and reducing your latency per transaction is the best option for complex, synchronous problem domains. Going outwards is only going to speed up if you can truly decouple everything in the time domain.
- heipei 6y ago
  Judging by the occasional post on HN, even a lot of experienced software engineers (the "experienced" being my take) seem to have no good handle on what a single thread and a single machine should be capable of handling, what the latency should look like and what it should cost as a result. The posts that I mean are about "look how we handled the load for a moderately active web-app which is 90% cacheable with only five nodes in our k8s cluster". I really don't know why that is, my guess would be that few people have really built something beginning to end with the most boring tech possible (start out with PHP+MySQL like everyone did 15 years ago). Or they always operate at levels of abstraction where everything is slow, so they have simply gotten used to it, like their text-editor not running smooth unless it's on an i9, because the text-editor now is a pile of abstractions running on top of Electron when vim was able to run smooth decades ago. It's sad and both an opportunity at the same time, because you can be the one with the correct gut feeling of "This should really be doable with a single machine, if not we're doing something fundamentally wrong".
  - alpaca128 6y ago
    I think you're right on the money with people having gotten used to it. Once I truly started harnessing the power of Vim combined with shell scripts and terminal multiplexers, my patience for many other programs and tasks decreased even further. We have the computing power to run complex physical simulations or AI training sequences on a normal home computer, but for some reason use programs that take 100 times longer to start than old software despite not having more features, and websites that sometimes even come with loading screens. Electron isn't even as bad or slow as many people think, but somehow developers still manage to throttle it so hard that I might as well just use a website instead. As someone who is just starting with nodejs and web development I find a lot of the tech feels nice but sometimes also unnecessarily abstract. Sure, it tends to make a lot of the code very elegant and simple, but every additional framework adds another guide and documentation to look at, another config file that has to be correctly referenced by an already existing one so npm knows what to do, another couple of seconds of build time and another source of bugs and build problems. Then of course you need that trendy package for easier imports maintenance - which IDEs automatically handled in the past, but now we gotta use an editor running in a hidden web browser which started from scratch in terms of features but is just as slow in return.

by cs702 6y ago

mattmanser 6y ago

Everyone's chasing "scalable", mostly for no reason at all. There's also this enduring myth you see repeated even here of how much saving there is by not having "devops", when only very big companies needed them in the first place. A few minutes per month is the most devops any startup would need to do in reality running their own VMs or servers, after an hour or two of setup. And that hour or two is often less than faffing around getting all the labyrinth moving parts of AWS/Azure/etc. working. And the whole thing becomes a transparent farce when you look at providers like Azure, where their own admin interface is so slow and prone to random breaks you have to ask: How is this scalable? It's clearly not scalable for themselves, so how is it "scalable" for everyone else? There are benefits, mainly for me I'll use them for hassle free deploys[1], but the whole thing sometimes feels like a massive con to sell the performance of a potato by advertising it as a jet engine. [1] Like a month back I deployed a demo prototype web app I wrote in 3 hours onto Azure straight from github with like two clicks, and got the contract, can't really beat that.

waheoo 6y ago
Makes me think of those old "Webscale" skits on YouTube.
- amiga-workbench 6y ago
  https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=bzkRVzciAZg https://www.youtube.com/watch?v=bzkRVzciAZg
empthought 6y ago
> A few minutes per month is the most devops any startup would need to do in reality running their own VMs or servers, after an hour or two of setup. Right up until something crashes or some hardware fails.
- jiggawatts 6y ago
  > Right up until something crashes or some hardware fails. Unlike the public cloud where Nothing Ever Goes Wrong (tm). There are never brownouts. There are never latency spikes. No undocumented quota limits. Documented but confusing quota limits. Unexpected performance issues caused by other customers. Forced upgrades. Forced patching. Shock bills. No siree Bob, that kind of thing just doesn't go on in the public cloud! It's all rainbows and unicorns....
  - rpedela 6y ago
    You have all that DIY too, except maybe shock bills, but it's your problem to fix. Like with everything, extremes are bad and all managed or all unmanaged are both bad. My personal guideline is use off-the-shelf services if they are cheaper than DIY, including labor, or if DIY provides no competitive advantage and off-the-shelf is cheap enough. But if DIY is much cheaper or gives a competitive advantage, do it yourself.
  - empthought 6y ago
    I fail to see how this supports your claim that you can get by with "a few minutes per month" of devops. In fact, it's a classic red herring. Your original claim is quite unrealistic and you'd do well to retract it. It's bad advice both for public cloud and for self-managed efforts alike.

tyre 6y ago

I think what’s important to remember is that these kinds of “truth” are not the full story. Yes, a technology probably won’t live up to its marketing. Yes, what works is probably good enough. Yes, changing has switching costs and sacrifices institutional knowledge. Yes, we’ve seen these technologies before and switched away from them from various reasons. That’s all fine and sometimes frustrating as an engineer. But it’s not the only reason decisions are made. Sometimes pitching your exec team on The New Hotness gives cover for changes that actually do need to happen. Sometimes adopting The New Hotness is marketing to recruit engineers who want to try something new. Sometimes adopting The New Hotness is because some people are bored and uninspired in their jobs and just want to believe in something good again. The success of things like MongoDB and blockchain are not because they are better or even very good at most things people use them for. They’ll never live up to the hype. But people need things to get excited about. It’s psychological. Attacking them purely on technical points can lead one to miss the actual point.

still_grokking 6y ago
I don't think burning in sum hundreds of billions of Dollars (or even more) because "it's OK being irrational, because emotions" makes sense.
- travbrack 6y ago
  The business case is that you won't be able to retain good engineers if you refuse to rewrite your AS400 COBOL app.
  - yters 6y ago
    I’ve wondered whether these good engineers are really a net benefit. They crank out something spiffy, move your big systems to it, and then disappear to work on the next hotness, leaving an already burdened tech team to support yet one more spiffy stack that is not quite as reliable or easy to use as we would like.
    - 6y ago

la6471 6y ago

TLDR

FpUser 6y ago

>"Invariably, their response has been to tell me that I just don't get it" I cured this one when my native server running on not so expensive multicore CPU with enough RAM left their cloudy setup in a dust performing more than a 100 times faster and still having huge potential reserve in vertical scalability. They were just astonished at what a single piece of modern hardware in combination with the native code could do.

rangoon626 6y ago
We need to meme c++ back into popularity
- FpUser 6y ago
  I do not think there is any real need. To me it is doing just fine. It does not have to beat every other language into submission.
vntok 6y ago
Congratulations, from now on you are on call 24/7 in case anything happens to that machine. It is now your own pet. When RAM fails in unpredictable random ways, you'll spend hours reading the syslogs. When your SSDs fail, you'll spend hours ordering new ones, setting raid up and migrating the backups you've had to provision, automate, supervise and secure yourself. When the server gets even remotely suspected of having been hacked into, you must trash it overnight and never use any of it again.
- chousuke 6y ago
  You talk like using a physical server somehow prevents building a highly available system or from using automation. Why would the hardware be any more of a pet than any random VM in the cloud? You could also just run a hypervisor on the host and get the benefits of virtualization along with full control of hardware. Sure, if a drive breaks you'd probably want to replace it instead of just replacing the entire machine; it's a thing you don't need to do with VMs, but that doesn't mean it's difficult. It definitely doesn't take "hours". Managing physical setups is different from managing fleets of cloud instances, but it's not necessarily automatically inferior.

mlthoughts2018 6y ago

I find your quote here and the follow up seriously unconvincing. You know what a miserable workflow is? Preparing all my build scripts according to the half baked build tool that our in-house Deployment team cooked up to use fabric to spray my deployment onto a bunch of nodes in our data center, all the tests and script linting pass, so then I go push it through our CI/CD pipeline (which can’t do the above tests for me because again this is all threadbare maintained by in-house tooling teams who could not be less aligned to end product deliverables), and lo and behold it breaks. Good thing I’m not paying so much for CloudWatch so I get to literally go ssh to the worker node of the CI/CD system where my task failed and cross my fingers that it wasn’t reclaimed for the next task and that the log files weren’t rotated off, grep my way to tracebacks that probably involve things no one on my team has ever heard of, spend 1/2 a day chasing them all down (if I’m lucky), and finally kicking the box to restart a build a few times and it goes. Then it goes to the in house homebrewed kubernetes deployment scripts where it promptly chokes on a secret that wasn’t configured right (thanks home made script linters), so then I have to go down a rabbit hole of eye melting Helm config of secrets for an hour but thank god I’m not paying for a simple cloud UI for this. So we get the secret cleared up and everything is great, until the kubernetes scheduler can’t schedule my pods because our data center is at capacity and now I literally can’t get work done until I chase down some probably incorrect and certainly unmaintained Grafana chart of who is using how much excess memory on which nodes, and wrangle someone with authority to force another team to stop eating lunch and downsize pods right now. After all this, 6-7 hours later and 20-30 labor hours later, my workload is finally deployed, but it will die overnight for an uncaught reason that doesn’t alert anyone, and even if it did, the team managing compute resources is in a permanent state of PTSD from never ending pages so they wouldn’t bother caring about why my workload failed and will just assume it’s application logic bugs or Docker bugs on my side for as long as they can just to avoid yet another un-debuggable failure landing on their plate, so that’ll be great for morale in the morning when I get to derail yet another day and spend time doing every type of ops work that is not my job and prevents us from delivering product goals. This is every company I’ve ever worked for or ever heard any colleagues, friends or associates work for, that tries to build their own SRE practice on top of data centers. Large cloud platforms alleviate so so much of this pain. The “back to the 70s” workflow sarcastically described in the quoted passage you shared sounds like an absolute dream compared to the nightmare of shipping code on internally created developer tool & deployment platforms interfacing to bare metal or space leased from data centers. I can’t give the cloud providers my money fast enough and I have only ever felt happy about that for years and years now.

deleted 6y ago

[deleted]

ta988 6y ago

The fun thing is that many C-people are fed the VC kool-aid about scalability. They want everything to br scalable because they want to cash in. Except that not many companies reach the level where you need to scale your tech stack "exponentially". And you usually don't want to scale the dumb prototype that ended up in prod anyway. The trap here is that those systems will allow you just that in a totally unauditable way... Good luck replacing parts of those systems as well when everything is interconnected...

mikepurvis 6y ago

"You deploy, get an error message, and login to CloudWatch to see what actually happened - it’s all batch-driven, just like the bad old days, so progress is slow." Even without cloud APIs and containers and whatever, I find there's a creeping thing that happens here which has been helpful to become consciously aware of and intentional about fighting back on. Basically, the thing is that wrappers become interfaces. You write tool A, and then tool B, and then some kind of script C which calls the two tools and produces a result. Then someone says "dang, so convenient", and sets up a Jenkins/Rundeck/Airflow/whatever job that runs script C against some source of inputs automatically— maybe there's even a parameterized build capability where arbitrary inputs may be submitted to it on demand. Before you know it, the fact that the original tools and script exist has been forgotten, because everyone's interface to this functionality is a handy webpage. And when someone wants to add more features, they don't even go back and change the script, they end up just adding it directly into the job definition. Before you know it, the full scope of what is being done on "the system" isn't even really available any more to be installed, invoked, and tinkered with locally, not in a meaningful way, anyway. It's not even necessarily all bad, but when I see this happening in a domain where I have the awareness of the underlying tools (or I may even have written or contributed to them), I do try to help demystify them by writing up docs on how to do the local setup, helping others get going with it, etc.

pjc50 6y ago
You can solve anything by adding a layer of abstraction - except having too many layers of abstraction. "Serverless" always reminds me of CGI/fastcgi, which was still a thing the last time I wrote web apps for money. It had the advantage of "scaling to zero" and could run a huge number of low-usage apps on a single 2000-era server.

flarg 6y ago

The cloud makes money for people so it's important. Would you rather that money went to shareholders or to wage earners?

heisenbit 6y ago

> I've mentioned as much to several C-suite executives and professional venture investors I know who tend to wax philosophical about "exponential growth in cloud services and big data" and whatnot. Invariably, their response has been to tell me that I just don't get it, or that I make no sense. They give me puzzled looks and maybe even get a bit exasperated with me. One word: Worldcom I still remember coming up to 2000 sitting in a meeting room of an equipment provider I recently joined looking at projections of market growth, competitors and our own sales and profitability. Shortly afterwards that part of the business was sold, I remained with the other non internet part and we sailed through the .com crash unscathed. A lot of what AWS offers makes no sense. A lot what some AWS customer's teams are building is overly complex and a horror show to maintain. But there is also value: Maintained images, backups, on demand resources and specialized resources. We hit one of the inflection points in Moore's Law and that drives us into horizontal scaling and specialized hardware. Like with Worldcom and the fantastic internet growth story back then there is a kernel of truth and value. Find people that understand it and be glad others are playing around and subsidizing the investments in the core.

bob1029 6y ago

I love the scalability "argument". The capabilities of a single thread of processing in a modern x86 CPU are underestimated by several orders of magnitude in most shops. The unfortunate part is that the more you chase the scalability with the cloud bullshit, the more elusive it becomes. If you want to push tens of millions of transactions per second, you need a fast core being fed data in the most optimal way possible. Going inwards and reducing your latency per transaction is the best option for complex, synchronous problem domains. Going outwards is only going to speed up if you can truly decouple everything in the time domain.

heipei 6y ago
Judging by the occasional post on HN, even a lot of experienced software engineers (the "experienced" being my take) seem to have no good handle on what a single thread and a single machine should be capable of handling, what the latency should look like and what it should cost as a result. The posts that I mean are about "look how we handled the load for a moderately active web-app which is 90% cacheable with only five nodes in our k8s cluster". I really don't know why that is, my guess would be that few people have really built something beginning to end with the most boring tech possible (start out with PHP+MySQL like everyone did 15 years ago). Or they always operate at levels of abstraction where everything is slow, so they have simply gotten used to it, like their text-editor not running smooth unless it's on an i9, because the text-editor now is a pile of abstractions running on top of Electron when vim was able to run smooth decades ago. It's sad and both an opportunity at the same time, because you can be the one with the correct gut feeling of "This should really be doable with a single machine, if not we're doing something fundamentally wrong".
- alpaca128 6y ago
  I think you're right on the money with people having gotten used to it. Once I truly started harnessing the power of Vim combined with shell scripts and terminal multiplexers, my patience for many other programs and tasks decreased even further. We have the computing power to run complex physical simulations or AI training sequences on a normal home computer, but for some reason use programs that take 100 times longer to start than old software despite not having more features, and websites that sometimes even come with loading screens. Electron isn't even as bad or slow as many people think, but somehow developers still manage to throttle it so hard that I might as well just use a website instead. As someone who is just starting with nodejs and web development I find a lot of the tech feels nice but sometimes also unnecessarily abstract. Sure, it tends to make a lot of the code very elegant and simple, but every additional framework adds another guide and documentation to look at, another config file that has to be correctly referenced by an already existing one so npm knows what to do, another couple of seconds of build time and another source of bugs and build problems. Then of course you need that trendy package for easier imports maintenance - which IDEs automatically handled in the past, but now we gotta use an editor running in a hidden web browser which started from scratch in terms of features but is just as slow in return.