5 ms·

12 years ago, even without AWS, you could get a Linode, host an app on its naked IP, and not made to feel too much like you weren't "serious" about hosting your

12 years ago, even without AWS, you could get a Linode, host an app on its naked IP, and not made to feel too much like you weren't "serious" about hosting your app. These days you almost always need to know a fair bit of Docker and Docker Compose, a lot of people want to Kubernetes, ELBs got replaced by ALBs + NLBs which you gotta manage in VPCs which you gotta manage through Security Groups and your traffic gets routed through CDNs. Logs are structured and passed to a SaaS which will have a custom search syntax that takes tens of seconds to search them poorly, and they'll get lost in the noise of all those components. You'll spend a non-trivial amount of time devising systems for tracing requests through all these components. That sounds so spot on. We do have a choice though. Static HTML, server-side templating, vertical scaling and /var/log still exist and work just fine. We don’t need to adopt the newfangled techniques unless the use case calls for them.

maccard 3y ago
I disagree. There are so many simpler options available. DO app platform is point and click deploy from GitHub for $5/month. Knowing docker these days is a huge boon - it immediately removes the "works on my machine" problem, and honestly the amount of docker knowledge required to deploy a basic even crud app is about 10 lines that can be copy and pasted from stack overflow by searching for "Dockerfile for X"
- collyw 3y ago
  It solves that problem but adds it's own. I guess its overall a positive.
- kqr 3y ago
  Just one anecdote, but I've had a script run flawlessly on its own in multiple environments, but it's docker image exhibited the "works on my machine" problem for some reason! (Never bothered figuring out why since the script ran fine on its own and was replaced with a proper application shortly thereafter.)
- arp242 3y ago
  The problem isn't so much that it's hard to get started with Docker, it's debugging stuff when it goes wrong is so hard. There's this 80M dockerd binary that needs to run as root and last time I checked it's over a million lines of code – lots of potential for failure there, ranging from "Docker doesn't do the right thing and I can't figure out so let's just nuke the entire server and start up a new one" to "Oh dockerd magical firewall rules made my firewall ineffective". Is that a good tradeoff? I don't know; it seems to me you can get 95% of what Docker gives you with 5% of the complexity. Luckily the ecosystem has started to realize that and better tools have started to materialize.
  - spiffytech 3y ago
    Which alternatives are you thinking of?
    - arp242 3y ago
      I've used runc, which worked out well for me, but it's not a "drop-in replacement" for everything Docker does; it's essentially just an OCI runtime (originally extracted from Docker I believe), but 9 times out of 10 that's what I want. There are other tools too, such as containerd (and more I don't recall the name of first-hand), but I don't have first-hand experience with them.
collyw 3y ago
Trying to convince people of that seems to be a loosing battle.
Klonoar 3y ago
I feel fortunate to run one of my current projects in that aforementioned "Linode-esque" fashion. It is just so, so much simpler to debug and trace what's going on. It can handle a surprising amount of traffic and work before any actual issues come up. Nothing changes from an upstream vendor, at all. It really "just works". That said, I'm very aware there's a sweet spot and that this doesn't work past a certain point. I just think that point is so far beyond where most people actually get.
- namaria 3y ago
  Niche is ok, niche is good. The entire western civilization was built on shopkeepers and tradesmen. The concepts of private domicile, personal freedom, even democracy itself one could argue, are firmly founded on skilled people doing niche work for their communities. Corporations are inherently imperialistic. Bring back niche entrepreneurship, I say!
ilrwbwrkhv 3y ago
But you literally don't have to do any such thing though. A simple make file and maybe a load balancer can easily let you launch large web applications used by millions of users. I do not understand where people are getting stuck here that they need all of this complexity. How many users do these apps have?
- notmypenguin 3y ago
  Not only that, half this supposedly necessary tech actually slows you down: containers and kubernetes, websockets galore, clients like giant barnacles on your server, then we get these best practices that weren’t thought out well: sessions, etc… mostly it’s the cargo cult implementation knee jerk response “gotta have it” like some frontend maniac from 2010 adding jquery to everything, when what is needed is to keep the hood open and keep people talking about the engine parts instead of calling it done and schlepping some guys tractor drag race motor around town delivering pizzas
  - dotxlem 3y ago
    > instead of calling it done and schlepping some guys tractor drag race motor around town delivering pizzas This sounds way too specific to be a hypothetical
- physicsguy 3y ago
  Way less than people think they’re going to. When I started my current job, the project was veering into microservices hell by a team that did not understand how to build a web application. A rough back of envelope calculation of potential customer base and what the product was aimed at (B2B in renewables) told me that even with world domination, the backend of the application would at most have 50k active users ever. So we didn’t really need 30 different microservices all which could scale independently, maybe just 2 or 3 larger services.
  - namaria 3y ago
    Do you work where I left earlier this year? Exact same description: extremely small total user base, 30~ people company, deploying 30~ services on Kubernetes... I was being asked to take over repositories full of configuration files. No, thanks.
deleted 3y ago
[deleted]
aa-jv 3y ago
>We do have a choice though. I took the choice: to run from the web with my hair on fire, screaming. Towards embedded, where none of this matters, and where the crux of the whole thing is really, really important: its the atoms, then the electrons... We don't need the web if the user holds our interface in their hands. Surprisingly enough, the web holds an iron grip on that face/phone interface, but .. one easy solution to all the cruft, is simply return to the roots. Yes, this requires hardware production capabilities. Its not for everyone, then. But I tell you, 40 years into this industry, it was so refreshing to abandon all of the fancy front-end stuff and just get straight down to the embedded realm. Nothing nicer than having the machine - and the user - all to oneself. Of course, this edge case gets discussed further in the article, but that is just one of a few things this old-timer could say about what went wrong with programming culture... (*cough* *cough*..should never have removed the compiler..*cough* *cough*.. this opened the door for all the weirdos to take over what should've stayed in the OS .. *cough*)
- namaria 3y ago
  I keep finding myself reading Forth literature and playing with gforth in the evenings, thinking fondly of hardware to escape the atrocities of editing yaml and waiting 10-30 minutes to see results. Maybe I should run in the same direction...
  - aa-jv 3y ago
    You really should. I encourage you to do so. There is still so much potential in the embedded world to "get it right" - i.e. undo all the cruft and bloat of the desktop/web/etc. But, be careful, its requires just as much rigour as any other market. Perhaps more so, in some ways. Tooling and methodology are a lot more important ..
marginalia_nu 3y ago
Works better than ever. If anyone ever asks how my search engine, running on consumer hardware and hosted off domestic broadband, how it survives the hacker news front page time and time again without the load times even deteriorating a bit, while many other websites that are ostensibly static in nature tend to keel over? Then the answer is something like this. * nginx * Java * Macroservice architecture * Vertical scaling * Server-side rendering * Minimal javascript, no session cookies. These things bump the cost of each page-load. * /var/log with relatively sparse logging in the sunny day path Like this is a pretty strict design philosophy. You can probably do more JS and front-end stuff than I do, but it should be understood it's not free. My page loads incur for the most part a single request each. There are websites where each page load incurs hundreds of requests. It's worth repeating: Back-end requests are not free. They are cheap, but they are not free. When you have a dozen users and they incur a few thousand requests, that may be within what your server can deal with. When you have a thousand users and they incur hundreds of thousands of requests, it's probably not as fine.
- Tade0 3y ago
  Let's also not forget that browsers generally limit the number of requests they send at a time - IIRC the current limit is 6, so a site may be slow just because it sends many requests. Even (or maybe especially?) popular products like Gmail are guilty of this, sending easily over 200 requests. That's 700ms(!) worth of stalled requests in the best case scenario.
  - marginalia_nu 3y ago
    HTTP/2 does mitigate this a bit, I think it's mostly just a limitation in the number of threads. Even with pools or green threads, a CPU can only do so many things in parallel. Context switching is expensive.
- mdaverde 3y ago
  Question: how do you reliably deploy? And if it's just scp-ing tarballs, how do you handle dependencies? Do you run & upgrade any third-party processes that rely on having their filesystem or deps? Struggling with this because even in a small app in a Linode box, I don't mind just cp-ing my bins but there's always other things/versions they need. Thinking of bringing in Ansible to help me out here. Deploying feels easier in the container, Kubernetes world

maccard 3y ago

I disagree. There are so many simpler options available. DO app platform is point and click deploy from GitHub for $5/month. Knowing docker these days is a huge boon - it immediately removes the "works on my machine" problem, and honestly the amount of docker knowledge required to deploy a basic even crud app is about 10 lines that can be copy and pasted from stack overflow by searching for "Dockerfile for X"

collyw 3y ago
It solves that problem but adds it's own. I guess its overall a positive.
kqr 3y ago
Just one anecdote, but I've had a script run flawlessly on its own in multiple environments, but it's docker image exhibited the "works on my machine" problem for some reason! (Never bothered figuring out why since the script ran fine on its own and was replaced with a proper application shortly thereafter.)
arp242 3y ago
The problem isn't so much that it's hard to get started with Docker, it's debugging stuff when it goes wrong is so hard. There's this 80M dockerd binary that needs to run as root and last time I checked it's over a million lines of code – lots of potential for failure there, ranging from "Docker doesn't do the right thing and I can't figure out so let's just nuke the entire server and start up a new one" to "Oh dockerd magical firewall rules made my firewall ineffective". Is that a good tradeoff? I don't know; it seems to me you can get 95% of what Docker gives you with 5% of the complexity. Luckily the ecosystem has started to realize that and better tools have started to materialize.
- spiffytech 3y ago
  Which alternatives are you thinking of?
  - arp242 3y ago
    I've used runc, which worked out well for me, but it's not a "drop-in replacement" for everything Docker does; it's essentially just an OCI runtime (originally extracted from Docker I believe), but 9 times out of 10 that's what I want. There are other tools too, such as containerd (and more I don't recall the name of first-hand), but I don't have first-hand experience with them.

collyw 3y ago

Trying to convince people of that seems to be a loosing battle.

Klonoar 3y ago

I feel fortunate to run one of my current projects in that aforementioned "Linode-esque" fashion. It is just so, so much simpler to debug and trace what's going on. It can handle a surprising amount of traffic and work before any actual issues come up. Nothing changes from an upstream vendor, at all. It really "just works". That said, I'm very aware there's a sweet spot and that this doesn't work past a certain point. I just think that point is so far beyond where most people actually get.

namaria 3y ago
Niche is ok, niche is good. The entire western civilization was built on shopkeepers and tradesmen. The concepts of private domicile, personal freedom, even democracy itself one could argue, are firmly founded on skilled people doing niche work for their communities. Corporations are inherently imperialistic. Bring back niche entrepreneurship, I say!

ilrwbwrkhv 3y ago

But you literally don't have to do any such thing though. A simple make file and maybe a load balancer can easily let you launch large web applications used by millions of users. I do not understand where people are getting stuck here that they need all of this complexity. How many users do these apps have?

notmypenguin 3y ago
Not only that, half this supposedly necessary tech actually slows you down: containers and kubernetes, websockets galore, clients like giant barnacles on your server, then we get these best practices that weren’t thought out well: sessions, etc… mostly it’s the cargo cult implementation knee jerk response “gotta have it” like some frontend maniac from 2010 adding jquery to everything, when what is needed is to keep the hood open and keep people talking about the engine parts instead of calling it done and schlepping some guys tractor drag race motor around town delivering pizzas
- dotxlem 3y ago
  > instead of calling it done and schlepping some guys tractor drag race motor around town delivering pizzas This sounds way too specific to be a hypothetical
physicsguy 3y ago
Way less than people think they’re going to. When I started my current job, the project was veering into microservices hell by a team that did not understand how to build a web application. A rough back of envelope calculation of potential customer base and what the product was aimed at (B2B in renewables) told me that even with world domination, the backend of the application would at most have 50k active users ever. So we didn’t really need 30 different microservices all which could scale independently, maybe just 2 or 3 larger services.
- namaria 3y ago
  Do you work where I left earlier this year? Exact same description: extremely small total user base, 30~ people company, deploying 30~ services on Kubernetes... I was being asked to take over repositories full of configuration files. No, thanks.

deleted 3y ago

[deleted]

aa-jv 3y ago

>We do have a choice though. I took the choice: to run from the web with my hair on fire, screaming. Towards embedded, where none of this matters, and where the crux of the whole thing is really, really important: its the atoms, then the electrons... We don't need the web if the user holds our interface in their hands. Surprisingly enough, the web holds an iron grip on that face/phone interface, but .. one easy solution to all the cruft, is simply return to the roots. Yes, this requires hardware production capabilities. Its not for everyone, then. But I tell you, 40 years into this industry, it was so refreshing to abandon all of the fancy front-end stuff and just get straight down to the embedded realm. Nothing nicer than having the machine - and the user - all to oneself. Of course, this edge case gets discussed further in the article, but that is just one of a few things this old-timer could say about what went wrong with programming culture... (*cough* *cough*..should never have removed the compiler..*cough* *cough*.. this opened the door for all the weirdos to take over what should've stayed in the OS .. *cough*)

namaria 3y ago
I keep finding myself reading Forth literature and playing with gforth in the evenings, thinking fondly of hardware to escape the atrocities of editing yaml and waiting 10-30 minutes to see results. Maybe I should run in the same direction...
- aa-jv 3y ago
  You really should. I encourage you to do so. There is still so much potential in the embedded world to "get it right" - i.e. undo all the cruft and bloat of the desktop/web/etc. But, be careful, its requires just as much rigour as any other market. Perhaps more so, in some ways. Tooling and methodology are a lot more important ..

marginalia_nu 3y ago

Works better than ever. If anyone ever asks how my search engine, running on consumer hardware and hosted off domestic broadband, how it survives the hacker news front page time and time again without the load times even deteriorating a bit, while many other websites that are ostensibly static in nature tend to keel over? Then the answer is something like this. * nginx * Java * Macroservice architecture * Vertical scaling * Server-side rendering * Minimal javascript, no session cookies. These things bump the cost of each page-load. * /var/log with relatively sparse logging in the sunny day path Like this is a pretty strict design philosophy. You can probably do more JS and front-end stuff than I do, but it should be understood it's not free. My page loads incur for the most part a single request each. There are websites where each page load incurs hundreds of requests. It's worth repeating: Back-end requests are not free. They are cheap, but they are not free. When you have a dozen users and they incur a few thousand requests, that may be within what your server can deal with. When you have a thousand users and they incur hundreds of thousands of requests, it's probably not as fine.

Tade0 3y ago
Let's also not forget that browsers generally limit the number of requests they send at a time - IIRC the current limit is 6, so a site may be slow just because it sends many requests. Even (or maybe especially?) popular products like Gmail are guilty of this, sending easily over 200 requests. That's 700ms(!) worth of stalled requests in the best case scenario.
- marginalia_nu 3y ago
  HTTP/2 does mitigate this a bit, I think it's mostly just a limitation in the number of threads. Even with pools or green threads, a CPU can only do so many things in parallel. Context switching is expensive.
mdaverde 3y ago
Question: how do you reliably deploy? And if it's just scp-ing tarballs, how do you handle dependencies? Do you run & upgrade any third-party processes that rely on having their filesystem or deps? Struggling with this because even in a small app in a Linode box, I don't mind just cp-ing my bins but there's always other things/versions they need. Thinking of bringing in Ansible to help me out here. Deploying feels easier in the container, Kubernetes world