5 ms·
The appeal of serving your web pages with a single process
- tryauuum 2y agoand no race conditions whatsoever
- Deebster 2y agoWell, unless you're using threads. Or did I miss the sarcasm?
- jeffrallen 2y agoIf you're using Go right, you won't have race conditions.
- tryauuum 2y agono sarcasm, the code won't have race conditions and sometimes it's great I actually recently planned to write a single threaded API to free myself from thinking about race conditions
- kevmo314 2y agoWholly agree. Too often we think about scale far too early. I've seen very simple services get bogged down in needing to be "scalable" so they're built so they can be spun up or torn down easily. Then a load balancer is needed. Then an orchestration layer is needed so let's add Kubernetes. Then a shared state cache is needed so let's deploy Redis. Then we need some sort of networking layer so let's add a VPC. That's hard to configure though so let's infra-as-code it with terraform. Then wow that's a lot of infrastructure so let's hire an SRE team. Now nobody is incentivized to remove said infrastructure because now jobs rely on it existing so it's ossified in the organization. And that's how you end up with a simple web server that suddenly exploded into costing millions a year.
- roncesvalles 2y agoAlso, I think people vastly overestimate how much uptime their application really needs and vastly underestimate how reliable a single VPS can be. I currently have VPSes running on both lowend and big cloud providers that have been running for years with no downtime except when it restarts for updates.
- kqr 2y ago> no downtime except when it restarts for updates. This sounds a little like saying "all of North America except the U.S." I don't think people are worried about random breakdowns on a single VPS, but scheduled updates are still downtime, and downtime causes revenue loss regardless of why it happened. Any time a service is important enough I ask for two servers and a load balancer specifically to handle deployments and upgrade windows transparently. But! I agree services are usually less important than people think.
- RadiozRadioz 2y ago> upgrade windows Ok, that explains this and the above comment. The last time I had to restart anything to apply an OS update was when I moved to a new RHEL LTS version, the lifespan of which is about 10 years. And there are many ways to do similar GNU/Linux upgrades without a restart at all. Does Windows Server really need to restart for updates like normal Windows? If so, that's hilariously crap and I'm glad I've never had to touch it. Edit: not saying a single VPS is fine if it's GNU/Linux, just remaking on the "restart to update" thing they mentioned
- HappMacDonald 2y ago> to handle deployments and upgrade windows transparently GP might have meant "upgrade: Windows(tm)", or he might have meant "windows of time which we have allocated to upgrading the server", and on my first reading I interpreted the second without a single shred of thought towards the possibility of the first.
- 2y ago
- Lord_Zero 2y agoI guess it depends on what world you live in. For example, using ASPNET Core, I just drop in this https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-9.0 https://learn.microsoft.com/en-us/aspnet/core/performance/ra... and boom I have rate limiting and I do not have to stress about threads or state or whatever.
- colonCapitalDee 2y agoASP.NET Core truly is a joy to work with
- geewee 2y agoThat's almost certainly per server instance though, there's no mention of any type of synchronization across multiple instances, so if you e.g run many small ones or run the service as a lambda I'd be surprised if it worked like you expected.
- Lord_Zero 2y agoAt that point you should be rate limiting at the reverse proxy or load balancer. - Nginx https://blog.nginx.org/blog/rate-limiting-nginx https://blog.nginx.org/blog/rate-limiting-nginx - Caddy https://github.com/mholt/caddy-ratelimit https://github.com/mholt/caddy-ratelimit - Treafik https://doc.traefik.io/traefik/middlewares/http/ratelimit/ https://doc.traefik.io/traefik/middlewares/http/ratelimit/ IMO Lambda is kind of an unfair example because the author doesn't mention having multiple instances. Plus a hot take I have is you should not be building an entire web-app as a Lambda or series of Lambda functions... AWS does not have solutions for load balancing in things like APIG so you would have to architect that via DynamoDB or ElastiCache which is the "extra layer or two of overhead" the author mentioned.
- porridgeraisin 2y agoThat's rate limiting locally i.e single instance. It is a fairly trivial thing and isn't the topic here.
- 2y ago
- DaSexiestAlive 2y agoSingle-thread could be a thing if it's like a full-stack all sitting in a web browser--like Dioxus is going toward.. If a web browser is in a glorified chromebook like a 2025 Macbook Air, indeed there's a lot of breathing room. A lot of ram. Processing power. Cores. It's nice. I get that. And then you can do off-line first: meaning use the cached local storage available to WASM apps. Then whatever needs to go to the mother ship, then call web apis in the cloud. That would, in theory, basically giving power back from "net pc theory of things" back to "fat client"--if you ask the grey-haired nerds among you. And you would gain something. But outside of a glorified chromebook like a 2025 Macbook Air--we have to remember that we are working with all kinds of web devices--everything from crap phones to satellite servers with terabytes of ram--so the scalability story as we have it isn't entirely wrong. I have been to U of Toronto, very smart people. But honestly this is a troll piece. Doesn't go into any depth and one-sided. Unhelpful. I think U of Toronto's reputation would be better served by something more sophisticated than this asinine blog entry.
- remram 2y agoIt's easy to have two implementations of your rate-limiting thing, in-process and Redis. Or change your implementation when you need it. Just put a nice interface in front of it. There is a cost to the network synchronisation, so you definitely want to scale vertically until you really must scale horizontally.
- AStonesThrow 2y agoThe filmmakers of “A Quiet Place” or the “Terminator” franchises should employ cks as a technical consultant
- theideaofcoffee 2y agoThen why not go to the extreme and just build a static version and serve that? Why do you need any dynamic content at all? Then you won't need shared state, then you won't need a database, then you won't need 90% of the stuff that dynamically driven pages need. Be the l33t h4x0r you are and just go static, save the complexity for your build process, because that shows the true hacker spirit. Hell, you may even be able to wring out another few blog posts from that. Also, why are people submitting every single post from this blog recently? Does this person actually do any work at UToronto, or is he just paid to write? There are -8000- links to various pages under this domain. I hope it's just a collective pseudonym like Nicolas Bourbaki and one person didn't write 8000 pages. I'm desperate to use some of the insights from a navel-gazing university computing center in my infrastructure: IPv6 NAT (huh? what? What?!), custom config management driven by pathological NIH (I know precisely zilch about anything at utcc but I can already say with 100% confidence that your environment isn't special enough to do that), 'run more fibers', 'keep a list of important infrastructure contacts in case of outages', 'i just can't switch away from rc shell', and that's just in the last six months. On second thought, I'll just avoid all links to here in the future to save my sanity.
- no_protocol 2y agoThe traditional pattern seen here of serving pages under a hierarchy called `~cks` indicate this is the personal site of someone who is affiliated with the university. Unless otherwise noted you should probably assume all the content is from "cks", not an army of dozens of coders.
- AStonesThrow 2y agoHe's been working at the same job, in the same place, since he graduated from college over 30 years ago. So that's an average rate of 5 blog posts per week. Not too shabby. And your surface-level scans indicate a lot of specialized deep-thinking about some specific tools. Sure, but you'll also find some good generalizations that arose from the depth and breadth of experience. He knows Linux like the back of his hand, and he's been using Debian and Ubuntu, and Fedora, so perhaps we can derive some takeaways from those? And thoughts on ZFS and anti-spam email hosting, those are good too. cks is the guy who influenced me to run Byron's rc shell, and also to install and run MH as my mail reader in 1993, and he also singlehandedly convinced me to install Ubuntu in 2006, which I maintained through multiple computers and upgrades through 2020. I cannot say that many, if any, of his blog posts were directly helpful to me, except for his opinions on PC hardware such as PCIe, parity RAM, and the like. But his wisdom is truly inspiring, as is his ability to stay with one employer for 100% of his career, doing more or less the same sysadmin things as he did in 1995.
- lleymrl651 2y agonice
- throwaway984393 2y ago[dead]
- stana 2y agoWould love to explore this in Python. But would it be correct to assume single process service would not be as performant due to GIL?
- simonw 2y agoThe GIL means you only get to use a single core for your code that's written in Python, but that limit doesn't hold for parts that use C extensions which release the GIL - and there are a lot of those. The GIL is also on the way out: Python 3.13 already shipped the first builds of "free threading" Python and 3.14 and onwards will continue to make progress on that front: https://docs.python.org/3/howto/free-threading-python.html https://docs.python.org/3/howto/free-threading-python.html And honestly, a Python web app running on a single core is still likely good for hundreds or even thousands of requests a second. The vast majority of web apps get a fraction of that.
- nogul 2y agoI’ve been running essential production systems with pythons uvicorn and 1 worker reliably, mainly because I get to use global state and the performance is more than fine. It’s much better than having to store global state somewhere else. Just make sure you measure the usage and chose a different design when you notice when this becomes a problem.
- dmurray 2y agoI have inherited a suite of apps that do this. I have mixed feelings about this: on the one hand, many of them work just fine and will never need a second worker, and they were relatively easy for inexperienced devs to put together. On the other hand, some of them do have performance problems and are a nightmare to migrate to a different solution, it's hard to reason about what parts of the application turn out to be stateful.
- j45 2y agoIronically, in a good way - simple scales, complex fails.
- fsckboy 2y agowhat was that superfast web server, opensource of some sort, from about 25 years ago, single process, single thread? it just raced around a loop taking care of many queued i/o streams
- koakuma-chan 2y ago> I discovered some that were making repeated requests at a quite aggressive rate, such as every five minutes 5 minutes sounds reasonable
- emacsen 2y agoAll that's old is new again. More than a decade ago, I inherited a Node application and was able to use many of these exact techniques inside the application. Even moreso, I could introspect the entire application state, including providing myself a shell and modify application state within the application. Keeping a blocklist inside a simple array- no problem! And being able to run a shell inside the same process meant I could inspect and even modify the array while the application ran. That made it incredibly pleasant to use and run. On the flip side, upgrades can be very challenging. In a modern web application it's standard practice to run (at least) two instances of the application at once and use the load balancer to test both, or to drain jobs from one to the other. This is relatively easy if the applications are stateless. Once the application holds all the state in memory, there's a real challenge. That array that seemed so clever- you'll need to serialize it so it can be reloaded at initialization time. Keeping all the session identifiers in memory- be ready to dump that. Worse, if the application is not designed to share this state with another application, you're now in some trouble. This is fine if you're running a small site with a few users who can accept some downtime, but if you're running a serious service, you'd like to have some kind of upgrade path other than shutting the application down and starting it back up again. "I'll just share state with other application servers", you might think, and then use something like ZeroMQ to transmit state", but once you think about sharing state between application servers, you realize you'd probably be better off using a tool like Redis, and you're right back where we started.
- loxs 2y agoYou can migrate to the heavier infra later. I am serving hundreds of concurrent users from a single Rust binary backed by Sqlite. So far it hasn't shown the slightest problems and migration can happen if the service grows an order of magnitude (or two) from here.
- emacsen 2y agoMigration/upgrades need to happen no matter the size of the service, so I'm not sure what you mean.
- loxs 2y agoI do this for my personal projects. Rust. Axum. Single compiled binary. Even html/js/css is baked into it via RustEmbed. Sqlite + litestream to S2. Cloudflare in front. Works extremely well.
- torginus 2y agoI was shat on multiple times by throbbing-brain architecture guys for this, but what I've done multiple times that worked in prod, was to start with a single monolith with dependency injection, and run all services in the same process. Once the compute and robustness demands mandated we should scale the thing across multiple machines, both horizontally and vertically, I would replace most DI services with proxies that wrapped the original functionality and called out to a remote host. So for this scaled up version, I would only need to change how the DI container got started, which could either be in 'Manager' mode (meaning a high level functionality, with the submodules injected as proxies calling out to services on client machines), 'Worker' mode (meaning it served the proxy requests on said worker machines) or 'Standalone' mode (meaning the DI container actually injected the actual fat versions of the services, that allowed the whole thing to run in a single process, very useful for local testing and debugging). The was only a single executable which could be run in multiple 'modes' selected via a command line switch, which made versioning and deployment trivial.
- pdimitar 2y agoIn what programming language did you do this? Sounds like a nice mostly future-proof architecture.
- torginus 2y agoIt was C# with ASP.NET containers. Not the sexiest tech stack in existence, but it's really versatile. I kind of simplified the architecture in my post, there were like almost a dozen different 'roles', and not all of them ran off the same executable (since some roles had specialist requirements, with some running off of Windows, the others off Linux, so there was some real world messiness to account for), but the Standalone 'god' mode was there and pretty much worked well on dev machines.
- gregors 2y ago>>>> The simple great thing about doing everything through a single process (with threads, goroutines, or whatever inside it for concurrency) is that you have all the shared state you could ever want, and that shared state makes it so easy to do so many things. If you like the sound of this, check out Elixir/Phoenix