11 ms·
This is the second or third "it's CGI again" thing I've seen in the past year. While these things are cool and definitely have their place, it's still worth not
by jchw 8y ago
This is the second or third "it's CGI again" thing I've seen in the past year. While these things are cool and definitely have their place, it's still worth noting that process per connection scales fairly poorly, simply because processes and forking are relatively expensive, and therefore it's probably unwise to deploy something like this in production anymore. It is what it is, I suppose.
- jchook 8y agoIs that a feature or a bug of *nix?
- yjftsjthsd-h 8y agoNot really; IIRC forking is more expensive on NT, and *nix processes/threads tend to be fairly cheap. It might be a generally expensive model, and threads/pools are probably the better-scaling option regardless of OS.
- jchw 8y agoOn NT though, there's a deeper divide between processes and threads. Threads on NT should be a lot cheaper. Especially since you can't fork on NT without diving into undocumented APIs* :) *On the Windows subsystem, of course. There's the lower level ZwCreateProcess function which can/could be used to fork, but it's undocumented and I believe it only existed for the old SfU. Now that that's gone and Linux Subsystem uses something called Pico Processes, I'm guessing this old fork flag is pretty much unsafe to use for anything at all.
- ape4 8y agoBut websocket connections are usually long lasting. So the cost of the fork is less important.
- ignoramous 8y agoI think it all boils down to how much can it scale. At what point do # of processes tip over a server versus how many # of threads can it handle versus how many # green threads a program runtime can manage.
- jchw 8y agoIt probably scales better than a forking HTTP server, but probably not much; modern HTTP connections tend to be at least a little bit long lasting and few people would dare serve a large site on a forking webserver (in part thanks to the fact that most webservers have moved on to workers or event loops.) It certainly would hold up inordinately poor to a DDoS attack.
- paulddraper 8y ago> modern HTTP connections tend to be at least a little bit long lasting But CGI doesn't fork per TCP connection; it forks per HTTP request/response.
- jchw 8y agoTrue, but as far as I know none of the major HTTP servers use forking anymore either. I believe there was a time when forking per connection was fairly standard for servers.
- toast0 8y agoA process per client with keepalive is definitely a losing proposition. But if you're willing to run old-school one request per connection, it's not unreasonable (although adding TLS to that is). Many years ago, while I was at Yahoo, someone made a clever hack: have a daemon that holds keepalive sockets and passes them to the (y)Apache daemon when they have something to read -- when Apache is done with the request, give it back to the daemon. (Sockets passed back and forth as file descriptors on a Unix socket). A further many years ago, David Filo came up with the idea of accept filters -- allowing a program to request the kernel to accept connections and have accept only return sockets that have a fully formed http request already, so an Apache (or whatever crazy webserver before Yahoo switched to Apache) wouldn't have to wait for the client there either.
- limsup 8y agoIt's not so much the fork but the memory cost. Each of those subprocesses has at least one call stack = 2 megabytes of memory. 2 megabytes per connection is many many orders of magnitude more that you would use in an asynchronous server.
- sebcat 8y ago1) that's virtual size, and most likely (depending on OS/cfg) COW (assuming no call to execve). 2) that's a default - most systems allow tuning You can have pretty decent performance with forking models if you 1) have an upper bound for # of concurrent processes 2) have an input queue 3) cache results and serve from cache even for very small time windows. Not execve'ing is also a major benefit, if your system can do that (e.g. no mixing of threads with forks). In forking models, execve+runtime init is the largest overhead. It will not beat other models, but forking processes offer other benefits such as memory protection, rlimits, namespace separation, capsicum/seccomp-bpf based sandboxing, ... YMMV
- MrTonyD 8y agoI think you guys are both right. Back in the days when I measured UNIX performance, it was fork that was expensive due to memory allocation - but not the memory itself. It takes time to allocate all the page tables associated with the memory when you are setting up for the context switch. But I should admit that it was a long time ago that I traced that code path.
- sebcat 8y agoprior thread with some ad-hoc measurements: https://news.ycombinator.com/item?id=16714403 https://news.ycombinator.com/item?id=16714403
- hendry 8y agoSocket connection alone is too expensive. IRC shows us that maintaining a reliable socket for most folks is next to impossible.
- vidarh 8y agoOn the contrary. IRC shows that maintaining a consistent network where every part can always reach every other part is tricky, but the most common problem with irc networks is not clients getting booted off, but net-splits that usually automatically resolves pretty quickly. That they're visible to clients is an issue with how channels spans servers and how operator status and channel membership is tied to who happens to be on a channel on a specific partition at a certain time, and how messages are propagated when splits resolve, and how inter-server communications happens. So it has plenty of lessons if you want to build a chat network, but nothing with it suggests maintaining a connection is otherwise a big problem. In general what it boils down to is the word "reliable": You would want to write your app so that a client that disconnects and reconnects gets a sensible behavior on reconnecting, e.g. by queuing messages when it makes sense, and discard them if it does not, to paper over temporary connection failures. But you would need to do that if you were to use stateless request/response pairs anyway.
- lexicality 8y agoI disagree. I've not seen a netsplit in months, but clients constantly timing out and rejoining is a fact of life.
- symisc_devel 8y agoNot that much on modern Linux/BSD systems at least. The kernel is smart enough to just copy fewer pages when forking not the entire parent process image.
- chrisseaton 8y agoThen you run a GC and dirty all your pages.
- paulddraper 8y agoYeah; forking is cheaper for low memory programs with no moving GC.
- sebazzz 8y agoGC usually runs on a separate thread right? Isn't it true you shouldn't run fork() when you have a multithreaded program?
- chrisseaton 8y agoNot usually. Only an advanced GC will be concurrent. Something like Perl or Ruby for example runs in the foreground. If it is concurrent then you shut down the GC thread, fork, and then start a new thread in each new process.
- romeisendcoming 8y agoIt's probably not safe to fork() from most frameworks threads (OpenMP) and is hard to make safe for complicated parallel processing. But if you control the flow of execution and minimize mutex and critical section(s) it can be done withal. Several examples of thread served queues performing user defined actions upon message receipt to include fork() + exec() come to mind.
- jchw 8y agoOf course any UNIX-like worth its salt is going to support Copy on Write and whatnot. But even then, forking is still quite slow relative to not doing anything at all. Here's my point of view: 1. Forking is first and foremost a system call. (To be fair, I realize even memory allocation is much of the time, but still.) The kernel is going to do a bunch of work (as fast as it can, of course) and you're going to end up with two different OS level tasks by the end. 2. Those two tasks are now scheduled in tandem by the OS scheduler. For two tasks, this is fine. For hundreds of tasks, it becomes less effective. Threads will rapidly go in and out of I/O wait and the scheduler has to balance all of this. 3. CGI dies right here, though: CGI is not just a fork. It is a fork and exec! The exec, of course, also run in the kernel. It's going to effectively load the binary from scratch. Then you probably hit the linker in usermode, which has to go through the shared library resolution, resolving and mapping shared objects into memory and filling out import tables. This stuff has some caching as far as I know, but still... it's not free. 4. Now you are in the entrypoint of your program... or are you? If you are using CGI with Perl or Python or even Bash, we're not done yet because the script interpreter has to load all of its state from scratch, parse your script, bla bla, and THEN finally we can run your program. 5. Your application now has to do all of its common setup. Every. Connection. If you need to connect to a database, you can't connection pool: you have to open a new socket every time. Redis? same thing. Need to read a config file? Yep, every dang time. You can hack around some of this but in general it's probably going to be like this for most CGI applications. There's a reason why FastCGI exists after all. The forking model of servers is elegant... but it doesn't work so well in my opinion. The cost of OS-level context switching is non-trivial, forking is relatively expensive, and things get worse when you are talking about something like CGI where your app effectively gets loaded from scratch each time. My favorite model is definitely the Go model, where the language schedules lightweight threads or fibers across n OS threads (where n = number of logical threads.) It is cheap for the OS scheduler, and the language scheduler can very efficiently deal with things like I/O blocking and GC without huge latency hits. But you can go about it many ways. Node.JS's event loop model proves pretty effective. Node is far from perfect but I think I would bet on a Node.JS server with a proper event loop over a CGI server forking a C program, myself. Of course I'm really no expert. But, I think the jump from CGI to FastCGI told me all I need to know: CGI just didn't scale well at all. FastCGI was a much better experience for me, though I no longer use it.
- blitmap 8y agoI really enjoyed SCGI in Lua: https://github.com/blitmap/lua-snippets/blob/master/scgi-server.lua https://github.com/blitmap/lua-snippets/blob/master/scgi-ser...
- jashmatthews 8y ago> processes and forking are relatively expensive This and "async is always faster" are two things which are no longer true on modern hardware. Forking a process on Linux uses the same clone() syscall as creating a thread so forking a small binary takes only tens of microseconds, leaving plenty of time for <1ms total response times. https://eli.thegreenplace.net/2018/launching-linux-threads-and-processes-with-clone/ https://eli.thegreenplace.net/2018/launching-linux-threads-a...
- CraftThatBlock 8y agoFor sure, but the biggest overhead is usually application startup (in say Java, Dart, Node, Python, etc) where just the VM can take a few 100ms to start. Using a compiled language would most likely be the way to go for maximum performance (although it's less important if the WebSocket is long lasting)
- jchw 8y agoSure. Under Linux, processes and threads are both just tasks anyways. Processes may be nearly as efficient as threads, but you can do one better by not even having to spawn new threads per worker, which is what a concurrency model like Go's or Erlang's enable with ease. CGI and Websocketd also have significantly more costs than just a fork since they need to execute a target program. And even forgetting the costs of forking, threads are not necessarily much better anyways. I can easily have a single Go program scheduling literally hundreds of thousands of Goroutines across just a few OS level threads and have no problems whatsoever, and in fact I've done exactly that in production, whereas I would never even dream of doing that with threads. Node.JS's event loop model also doesn't need to spawn threads per connections. When I say processes and forking are relatively expensive, I don't mean compared to doing the exact same thing with threads; I mean compared to more modern alternatives, like using an event loop or Goroutines.
- jashmatthews 8y agoI never wrote that going back to CGI and creating a process or thread per request was a good idea, only that forking is much cheaper than often thought. Event loops and userland cooperative multitasking like Goroutines predate Linux 2.6 kernel threads. They're not "more modern".
- ori_b 8y agoDepends on what your needs are. In many cases, you need to scale to a few dozen connections per day, and simplicity is far more valuable than performance. If you're pushing the limits of performance, any off the shelf solution is probably not going to scale as well as it could for your specific workload.
- mbrock 8y agoYes! Every mention of CGI also gets a sneer and a reminder of how poorly it scales, but nothing I do requires scale in that sense. "In production" doesn't always mean "hundreds of requests per second." Every road doesn't need 8 lanes.
- laumars 8y agoI’ve had developers say this before when building public websites and it has cost us in lost revenue when the site has gone offline or cost us lots in virtual hardware having to scale the number of servers available to meet higher than expected volumes of traffic. Don’t get me wrong, I'm not suggesting that everyone should be building their site like it’s Facebook or Google, but when every public website is already sat on a proverbial motorway, it can be dangerously shortsighted to build your site to only meet demand with no room to scale. CGI falls under that category. There are plenty of platforms that are still very easy to build and deploy, developer friendly, and still runs circles around the performance of CGI. (Please note that I am talking about public sites specifically and not intranets or other IP white listed resources)
- askmike 8y agoUsed in production doesn't always mean used by all users of a public app that might go viral. If you have an app with an admin interface/CMS/monitoring interface and you know there won't be 50 administrators for your app before the end of the year this works great (assuming you can reverse proxy this to add auth or something).
- laumars 8y ago
- deleted 8y ago[deleted]
- Jedi72 8y agoUnless you develop in Elixir/Erlang where you can spawn millions of concurrent processes.
- arendtio 8y agoAFAIK, those are not 'processes' in terms of operating system processes, but instead some kind of parallel running tasks within the Erlang VM (much closer to threads). Edit: If you think I am wrong, could you please explain what is wrong?
- masklinn 8y agoAFAIK there is very little difference between processes and threads in linux (both are instances of task_struct), the variability in implementation details specific makes that use for classification useless, and thus the only possible separation of any use is semantics: the difference between processes and threads is whether they share internal state (generally / by default). Erlang's don't, thus processes. Go's do, thus threads.
- arendtio 8y agoInteresting. My interpretation was more along the lines of having its own process identifier (PID) and to my knowledge, OS processes have that, OS threads don't and I thought Erlangs processes don't get PIDs by the operating system either (although that might be wrong too).
- masklinn 8y ago> OS processes have that, OS threads don't At the kernel level, both "processes" and "threads" have PIDs since they're the same thing. In fact, before NTPL was implemented Linux broke posix because it returned the kernel PID directly when queried[0]. That's no more than an interface choice of POSIX though. And POSIX threads still have a thread id, a (pid, tid) is a unique visible identifier. So the distinction seems completely arbitrary. And Erlang processes do have (erlang-level) PIDs. In fact, they get (erlang-level) PIDs across multiple machines when in a cluster. And again I think ancillary properties are not what matter, a PID is a consequence of being a process, not a cause. The cause of being a process is not sharing internal state, that's the useful bit. Erlang's tasks are not os processes (that would rather defeat the points), but it doesn't seem useful to call them "not processes". [0] it now returns the tgid, a "thread group" at the kernel level is what you see as a process from userland: the tgid is the pid of the original task, creating a "process" will create a new task with a new tgid while creating a "thread" will create a new task with the existing tgid https://stackoverflow.com/a/9306150/8182118 https://stackoverflow.com/a/9306150/8182118 provides an excellent primer on how this works
- geocar 8y agoThe myth that fork is expensive is pervasive, and speaking to the ways that it is true†, well: performance is relative. fork() only takes around 8ms on my Linux machine and I can get 100,000 posix_spawn() per second there with 100MB RSS. That's "fast enough" for a large number of applications. †: fork() is a lot slower (over 20x) on Windows
- iforgotpassword 8y agoRelated: when I worked on a server (on linux) that would spawn threads for new connections that were mostly short lived, I tried to use a thread pool instead. Hand crafted, with push and pop both O(1). It was still mostly slower than just spawning a new thread every time.
- kevingadd 8y agoIt's generally considered the right move to just spawn your own thread for things like connections instead of pooling threads - people tend to think they should throw everything into a threadpool but it's not actually encouraged to do that for stuff like web server connections, compiles, etc unless the task is short-lived. I wish more people knew that going in :) Some threadpool APIs actually ask you whether the job is going to take a while and if it is, they function more as a job limiter - not 500 active threads all competing for CPU - than a thread reuser.
- inglor 8y ago8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).
- jondubois 8y agoI was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection. In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.
- sureaboutthis 8y agoThis is something that confuses me. Servers offer CGI and fastCGI and if you aren't using those to interface with your own code, how do you do so otherwise? Do some languages have their own deep connections into the server flow that makes them faster? I just don't understand this.
- aasasd 8y agoYou can also just serve HTTP.
- sureaboutthis 8y agoThat has nothing to do with my question. Of course you "serve HTTP". I want to know how one interfaces with a server without CGI or fastCGI.
- aasasd 8y agoYou serve HTTP to the server. To the front-end server or the load balancer, aka ‘reverse proxy’. That's how. Both Apache and Nginx can proxy HTTP, out of the box.
- delinka 8y agoIOW, opening a socket from the routing HTTP server to an application HTTP server. That’s pretty much how FCGI works.
- aasasd 8y agoYeah, people just seemingly realized that there's nothing special about FastCGI that couldn't be done with plain http. And as a bonus, the app itself can work as a server in the dev env, and testing is simpler. (Only, CGI managed to get headers sorta right by embedding them in protocol vars and not the other way around.)
- 8y ago
- gumby 8y agoWhile it's cute, I've never liked fork(). It's almost always followed by exec(), which means the default case is to do bookkeeping that is immediately discarded, something CoW helps with but doesn't eliminate (consider all those fds!)
- hawski 8y agoI would really like to see a benchmark for: stand alone fork in small statically compiled binary and also every combination of: [[vfork, fork] + exec, posix_spawn] + [small static binary, small dynamic binary]. I understand that for big interpreted languages like Python it's certainly slow, but with advent of Rust I would believe that CGI could make a come back. It's fast enough for many tasks and it gives great isolation. Using separation of concerns by using separate processes gives also ability to easily sandbox whole application even by as crude means as original seccomp.
- andyidsinga 8y agoA long time ago I used to write CGI endpoints in C (strangely, not because I had to, I just wasn't as good at perl ). I used to say "its not a CGI 'script' maaan! I wrote it in C" with the accompanying l33t glance of respect in response ;) Anyhoo - same arguments about forking, process & memory limits arose ; so things like FastCGI and ISAPI were the solution to reduce/avoid some of those overheads. I suspect Rust based CGI endpoints would have similar <quote>issues<unquote>. But I've been thinking about this CGI comeback from time to time - the left side of backend infrastructure (load balancerss, reverse proxies etc) and middle tier machine resources are much better now. Docker containers and lambda spin up is similar to some degree in terms of topology but maybe with even more overhead? ..so does does make one wonder if cgi like simplicity make a comeback with today's order of magnitude better infrastructure? I keep waiting for someone to come up with "lambdaGI" ..with will lead to "fastLambdaGI" and so on (and that's not a snarky comment).