12 ms·
This seems unnecessary to me. In the example, the core of git should be a library yo can link so you don't need to run the binary. That would be better in eve
by lokar 5mo ago
This seems unnecessary to me. In the example, the core of git should be a library yo can link so you don't need to run the binary. That would be better in every way.
- sanderjd 5mo agoThere are lots of reasons to want to spawn fresh processes, which aren't solved by linking a library.
- aerzen 5mo agoSpawning processes should not be on the hot path of any program.
- 1718627440 5mo agoWhy? That's a very useful processing primitive.
- lokar 5mo agoIt’s a hack with many disadvantages. Sometimes a hack is the right answer, but the kernel should it add a primitive for it.
- MBCook 5mo agoShould bash link in every program the user might want? Load them up as dynamic libraries?
- m132 5mo agoNode, Python, PowerShell, and the rest do (almost) just that. launchd and systemd famously strived to remove as much shell from the start up process as possible because it was harming boot times and introducing unpredictability.
- sanderjd 5mo agoI don't know Node or PowerShell very well, but I'm not sure what you mean by this with respect to Python.
- m132 5mo agoCPython doesn't usually create subprocesses unless specifically asked to, it loads Python modules and native extensions into its process. The former is similar (you're still extending an existing process with new code, just interpreted), the latter is literally dlopen(), so loading dynamic libraries. A lot of other Python implementations don't have the ability to spin up new processes at all too.
- sanderjd 5mo agoI still don't really get this point. It's just two different things, spawning processes and running libraries. Seems like you're comparing apples and oranges to me.
- lokar 5mo agoBash as an interactive tool is very different. It is used to run an almost arbitrary number of things, and a pretty low rate. Bash as a programming language is just a bad idea.
- sanderjd 5mo agoAren't we discussing just such a primitive?
- lokar 5mo agooh, sorry, typo s/it/not/
- pizlonator 5mo agoIt ends up on the hot path of programs that use process isolation aggressively
- lokar 5mo agoSure, and there a primary thing you want is a whole new environment/context for the child (new environment, fds, memory, cgroups, namespaces, etc).
- lokar 5mo agoSure, but not many times a second
- kllrnohj 5mo agoEvery build system ever says hello.
- sanderjd 5mo agoWhy not?
- m132 5mo agoBecause it comes with a lot of overhead and, unless for some reason you really need every of those processes to have their own address space, set of privileges, file descriptors, etc., there's no point in wasting resources repeatedly setting those up only to tear them down milliseconds later. Running the same workloads in an nginx-style process pool usually works better.
- sanderjd 5mo agoI see what you mean now. I agree, a sustained workload of creating many processes very quickly is probably not a great idea. But it's also useful to be able to spawn that process pool (and any number of other use cases like that) efficiently.
- 1718627440 5mo agoBut when you use a process, you get tons of things for free, the subtask is invoked in parallel, you get isolation and you can control execution for free. Unless you are already writing a multithreaded program or already accept passing objects in memory, using a process is actually easier to write than using a library. If I use a library, I also need to start using threads and need to invent some core synchronization mechanism. I essentially are reinventing a small scheduler, when I already get this from the OS for free. Also know any crash in the third-party code will crash the whole program, the third-party code has access to the whole address space. With invoking a process you also have a standardized API implemented by the OS.
- lokar 5mo agoI'm not sure what you mean by inventing a sync mechanism, all languages come with one. Same with a scheduler, either your language runtime or the OS (or both) will deal with scheduling.
- omoikane 5mo agoLaunching git repeatedly was probably not the best example. But it's hard to think of good examples where launching processes repeatedly is the most performant thing to do, probably because launching processes had been expensive and everyone has learned to do something else (libraries, zygotes, etc). Maybe a different question is: if launching processes were cheap, is there something we would implement as processes instead of libraries? I can recall just one program that's intentionally not implemented as a library, but I think people have since built a library on top of it: https://dechifro.org/dcraw/#:~:text=Why%20don%27t%20you%20implement%20dcraw%20as%20a%20library https://dechifro.org/dcraw/#:~:text=Why%20don%27t%20you%20im...