5 ms·
The Linux kernel can spawn processes on its own
- phendrenad2 4y agoInteresting find. The kernel has all kinds of weird and wacky things in drivers.
- chasil 4y agoThis seems similar to nfsd, which is only present in many UNIX implementations to prompt the kernel to service NFS requests. A lot of (local) filesystems seem to do this as well (XFS in particular).
- deleted 4y ago[deleted]
- temac 4y agoA lot of kernel code spawn threads. I was not aware that it can also spawn usermode processes running code embedded into the kernel. That should be advertised and developed more IMO. Tons of drivers should have nothing to do in kernelmode.
- convolvatron 4y agohow else would init get started?
- anyfoo 4y agoAfter bootstrapping, the kernel sits in its idle task and does absolutely nothing except serving interrupts, the scheduler only switching between kernel threads at most. Then, after a set amount of time (5 minutes), another device with full DRAM access (or sufficiently configured IOMMU) and in the same coherency domain (or some contraption to make things coherent) starts to DMA the init task and the appropriate changes to the kernel's data structures into memory. When made coherent, the kernel's scheduler will see the new init task's thread in its set of runnable threads and eventually switch that. What? You did not ask for a method that is not completely batshit insane.
- dmitrygr 4y agoExcept given the size of caches nowadays, without some cause, that coherency might never happen :-)
- anyfoo 4y agoYep. That’s why I kind of copped out with just mentioning that the device has to either be in the same coherency domain, or there must be some super vague hand-wavy “contraption” to achieve coherency. Be creative I guess! :P
- josephcsible 4y agoI wonder what the parent PID of these processes is.
- cmeacham98 4y agoinit has a ppid of 0, which effectively is a special value that means the kernel. These processes probably also do the same thing (and thus, ppid=0 is a good way to detect kernel-started processes).
- sounds 4y agoThat's what the "kthreadd" kernel process is for. Try this command: ps axo pid,ppid,comm | less The kernel-created processes will all have "kthreadd" as their Parent Process ID (PPID). Typically init and kthreadd are the only processes with a PPID of 0.
- CameronNemo 4y agoYep, and aiui kthreadd on Linux always has a PID of 2. If you are ever trying to filter out kernel processes from ps output (and usermode helpers, apparently), you can check if the pid is 2 or ppid is 2.
- pengaru 4y agoWhen the kernel is the gatekeeper for the system, obviously it can do anything userspace can, and then some.
- temac 4y agoIt can do anything theoretically. It doesn't do anything architecturally. For example, contrary to NT, syscalls can't callback to userspace.
- the8472 4y ago> For example, contrary to NT, syscalls can't callback to userspace. There are some mechanisms that can call back into userspace during syscalls such as seccomp filters, FUSE, ptrace, userfaultfd, fanotify, the syscall_user_dispatch feature used by wine... There's the core_pattern handler too. Someone summarized it that a mov instruction could be serviced by starting a python process.
- saagarjha 4y agoThis is convenient but having the kernel block on userspace doing something is typically bad design :(
- throwawaylinux 4y agoIt really isn't. "The" kernel doesn't block, a context blocks and typically that context is associated with a user process (directly or indirectly). sleep(3) the kernel blocks for a time specified by userspace. futex(2) can cause the kernel to block until usespace wakes it. Similar wait(2). Also read(2) from a pipe or localhost socket. So whether or not it is a bad design depends highly on what gets blocked, in what situations, and what can be done to recover the situation if things misbehave.
- AlexSW 4y agoGenuine question: how do you know such things? I'd love to learn them, so I'm wondering how others learn them (other than remembering random comments from here).
- tremon 4y agoThis isn't new, right? Firmware loaders and hardware hotplug events (when not using sysfs' netlink firehose) are also spawned directly from the kernel, IIRC?
- dezgeg 4y agoIn those cases you mentioned the kernel has spawned a process loaded from normal ELF executable from filesystem, just like you could do with exec(). But in this bpfilter_umh case the executable is not coming from the filesystem but instead is embedded in the kernel itself. That seems to be quite new invention.
- sounds 4y agoThe interest is warranted, but it was been true for a long time that the kernel spawns processes where the executable is in the kernel itself. https://www.quora.com/What-is-kswapd0-in-Linux-Kernel https://www.quora.com/What-is-kswapd0-in-Linux-Kernel https://stackoverflow.com/questions/9154042/where-can-i-find-documentation-on-the-kflushd https://stackoverflow.com/questions/9154042/where-can-i-find... Here's some official documentation on it: https://tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO-8.html https://tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO-8.h...
- throwawaylinux 4y agoThose are kernel threads and not considered to be like spawning a process.
- Ericson2314 4y agoI want to stop doing -j with make, and instead allow the OS scheduler --- which alone is in the position to make the right call ---- to "pull" more batch jobs to run as system resources allow. Perhaps one can do this or almost do this with io_uring?
- alchemist1e9 4y agoThat would be a really neat feature. I do run slurm on single nodes and sbatch with dependencies to emulate some of the effect you are asking for.
- Ericson2314 4y agoGlad to hear you think so too!
- gtirloni 4y ago> which alone is in the position to make the right call Why is the OS scheduler alone in a better position than make itself to make the right call?
- Ericson2314 4y agoEveryone else is guessing how may "fork" commands to throw out the OS. The OS itself has much more information at its disposal about what resources are being used, where the bottle necks are etc. An elegant design on doing this in userspace probably boils down to going well on the awy to making a micro kernel.
- nyanpasu64 4y agoSpecifically one failure mode I've seen with Ninja is that it starts 12 processes for my 12-threaded CPU, each process eats 2 gigabytes of RAM, and I only have 16 gigabytes of RAM total, so it sends my system into swap or oomkiller. Ideally when the RAM fills up (whether because of compilers spawned by Ninja, or because I opened Discord), Ninja would kill one of the build processes and send it back onto the build queue, and reduce the number of compilers running at a time by 1 until I either tell it, or (some heuristic like) over 1.5 times the maximum RAM taken by a single compiler so far gets freed up. I don't know if this is best handled by the OS, userspace Ninja, or some combination (like cgroups).
- 762236 4y agoIf you use SELinux, you'd see that it has the 'kernel' domain.
- marcodiego 4y agoSomething I'd like to have: a magic sysrq combo to pause all processes and run a predefined binary. This would allow me to, when the userspace if broken, do something useful and save the jobs running on the machine. Of course, some way to prevent it from turning it into a DDoS is important, of course.
- _blaise_ 4y agoSomething like stop_machine() along with freeze_processes() might get you fairly close to that.
- zaphirplane 4y agoThis seems very close to hibernate to disk, so seems the plumbing is already there
- 5e92cb50239222b 4y agoLook at criu maybe? https://criu.org/ https://criu.org/
- anonymousiam 4y ago"which means it’s probably safe to trust the process’s title and, well, that says bpfilter_umh, so there you have it. (Just saying: Don’t trust process titles in general. They’re easy to manipulate.)" And then there's this: https://www.bleepingcomputer.com/news/security/bpfdoor-stealthy-linux-malware-bypasses-firewalls-for-remote-access/ https://www.bleepingcomputer.com/news/security/bpfdoor-steal...
- Matthias247 4y agoI learned about the existence of usermode helpers last year by working with coredump helpers (/proc/sys/kernel/core_pattern) and found it quite interesting too. The modes where the kernel spawns a new process on behalf of the users (or as init process) are quite obvious. But this case - where the kernel just executes a helper program in userspace which then immediately terminates - are slightly different. It's a neat design - because it allows for customizing kernel behavior without having to compile a custom kernel.