4 ms·

I'm so old I remember when ps would directly get the processes via system calls to the kernel and none of this reading from /proc.

deepsun 2y ago
To me it actually sounds like a better approach than blindly following "everything is a file" mantra.
- kjeetgill 2y ago
  I'm totally with you for processes should probably use glibc/platform API calls, but procfs is nothing short of brilliant and a great validation of the power of the "everything is a file".
  - Sophira 2y ago
    Except that any process that has obtained root (through a security flaw or otherwise) can extremely easily hide itself, as shown here. I will admit that it's likely that the same thing could be done with loadable kernel modules, that said.
- yjftsjthsd-h 2y ago
  What's blind about it? Reading files is nicer than parsing ps output or needing to write native code over a syscall.
  - jcranmer 2y ago
    Trying to parse the output of several procfs files isn't as easy as it looks. The per-pid maps file for one (smaps is even worse). Much better if I could just get a struct with all this information.
    - 0xedd 2y ago
      Asking DevOps to program for basic tasks. Cool story, bro.
    - djbusby 2y ago
      I've not looked at kernel source but, when I had a task to read info from /proc it looked like a lot of info was just pack() in there. So my parsing was just done with unpack(). Mapping those values from things that could be found in existing userland tools.
  - zokier 2y ago
    One problem with reading /proc is that its impossible to get consistent view of the whole system because you are iterating the processes one by one, instead of getting snapshot of all the processes.
    - yjftsjthsd-h 2y ago
      Is there an OS that provides the whole result of ps(1) or equivalent in one syscall? I was under the impression that it was ~always iteration based.
      - zokier 2y ago
        On NT you can get this sort of stuff through WMI, although I'll admit it's not entirely clear to me how that resolves internally
        yjftsjthsd-h 2y ago
        I guess my point/question was more - is whatever system/library call you'd use atomic? There's technically no reason the system couldn't have a syscall that returns a full data structure of process info for as many processes as are in scope and guarantee that it's a perfect atomic snapshot of exactly how the process table looked when that call returned, but I'd want to be very confident and have explicit documentation promising as much before I depended on it.
  - deepsun 2y ago
    The problems are: "what file?", "what format?", "merge data from multiple files". Not sure if relevant to this discussion, but here's the problems I've experienced working with files for system devices: [1] here's just a discovery of a logitech mouse receiver, notice that it touches: * /sys/class/* * /sys/module/* * /sys/class/hidraw/*/device/driver I must take care that each dir might not be present, symlinks may be dangling, and access to multiple files is not atomic. It would be much easier and stable using just one SQLite query, for example. [1] https://github.com/Lekensteyn/ltunify/blob/master/ltunify.c#L1164 https://github.com/Lekensteyn/ltunify/blob/master/ltunify.c#...
metadat 2y ago
Is this still possible in Linux? Or were the aforementioned syscalls removed?
- kmeisthax 2y ago
  Linux never removes syscalls.
  - LegionMammal978 2y ago
    Though it does stop implementing obsolete syscalls on newer architectures. So the set of syscalls you can effectively use often rotates as the hardware treadmill turns.
    - dwattttt 2y ago
      And since they haven't been removed, just not implemented, you get the joy of finding out it doesn't work anymore when you try to use it, instead of when you build it!
upon_drumhead 2y ago
Are you sure? I know BSDs use kvm_getprocs, but I don't know what the comparable sys call would have been on Linux. FWIW, /proc was added in Linux v0.97.3, September 1992, which is early enough I couldn't find ps source for linux earlier then that date.
- yjftsjthsd-h 2y ago
  The post you're replying to didn't say "on Linux" - IIRC, unix ps worked by reading /dev/kmem or so Edit: My mistake, it was /dev/mem - https://github.com/lsahn-gh/unix-v6/blob/master/source/s2/ps.c#L100 https://github.com/lsahn-gh/unix-v6/blob/master/source/s2/ps...
  - upon_drumhead 2y ago
    Agh, completely fair. The topic was about Linux and I incorrectly presumed that the original comment was made in the same context. My mistake.
- donio 2y ago
  I remember this too. Very early on there were both proc and a /dev/kmem versions of ps and maybe top too. Yep, found it: https://www.ibiblio.org/pub/historic-linux/ftp-archives/sunsite.unc.edu/Sep-29-1996/system/Status/ps/INDEX.html https://www.ibiblio.org/pub/historic-linux/ftp-archives/suns... And the Linux kernel CREDITS file has an entry mentioning it too: N: Rick Sladkey D: utility hacker: Emacs, NFS server, mount, kmem-ps, UPS debugger, strace, GDB [...]
- dfox 2y ago
  kvm_getprocs is in userspace library libkvm which abstracts away how that is done. And that library is primary meant for accessing /dev/kmem and crashdumps, with this functionality being bolted onto it (FreeBSD manapage even mentions that as a bug) and it works by examining the symbol table of running kernel and just walking the kernel datastructures. On FreeBSD there seems to be some trick where opening "/dev/null" instead of "/dev/kmem" causes the process "to not access kernel memory directly". Looking at the code it seeems to me that it means that libkvm will really open /dev/null and treat it as if it was /dev/kmem, which raises a question of exactly how that works. [Edit: apparently this works because in case of querying processes of running kernel, the code in kvm_proc.c does not read from the file and instead calls sysctl().]
rwmj 2y ago
The original 'ps' got the data by reading kernel memory directly (/dev/kmem or /dev/mem), and I'm old enough to remember systems doing that. I don't think there were any Unix-derivatives that used system calls? Minix is closest, in that you could query the 'mm' task using RPCs which are sort of system calls.

deepsun 2y ago

To me it actually sounds like a better approach than blindly following "everything is a file" mantra.

kjeetgill 2y ago
I'm totally with you for processes should probably use glibc/platform API calls, but procfs is nothing short of brilliant and a great validation of the power of the "everything is a file".
- Sophira 2y ago
  Except that any process that has obtained root (through a security flaw or otherwise) can extremely easily hide itself, as shown here. I will admit that it's likely that the same thing could be done with loadable kernel modules, that said.
yjftsjthsd-h 2y ago
What's blind about it? Reading files is nicer than parsing ps output or needing to write native code over a syscall.
- jcranmer 2y ago
  Trying to parse the output of several procfs files isn't as easy as it looks. The per-pid maps file for one (smaps is even worse). Much better if I could just get a struct with all this information.
  - 0xedd 2y ago
    Asking DevOps to program for basic tasks. Cool story, bro.
  - djbusby 2y ago
    I've not looked at kernel source but, when I had a task to read info from /proc it looked like a lot of info was just pack() in there. So my parsing was just done with unpack(). Mapping those values from things that could be found in existing userland tools.
- zokier 2y ago
  One problem with reading /proc is that its impossible to get consistent view of the whole system because you are iterating the processes one by one, instead of getting snapshot of all the processes.
  - yjftsjthsd-h 2y ago
    Is there an OS that provides the whole result of ps(1) or equivalent in one syscall? I was under the impression that it was ~always iteration based.
    - zokier 2y ago
      On NT you can get this sort of stuff through WMI, although I'll admit it's not entirely clear to me how that resolves internally
      - yjftsjthsd-h 2y ago
        I guess my point/question was more - is whatever system/library call you'd use atomic? There's technically no reason the system couldn't have a syscall that returns a full data structure of process info for as many processes as are in scope and guarantee that it's a perfect atomic snapshot of exactly how the process table looked when that call returned, but I'd want to be very confident and have explicit documentation promising as much before I depended on it.
- deepsun 2y ago
  The problems are: "what file?", "what format?", "merge data from multiple files". Not sure if relevant to this discussion, but here's the problems I've experienced working with files for system devices: [1] here's just a discovery of a logitech mouse receiver, notice that it touches: * /sys/class/* * /sys/module/* * /sys/class/hidraw/*/device/driver I must take care that each dir might not be present, symlinks may be dangling, and access to multiple files is not atomic. It would be much easier and stable using just one SQLite query, for example. [1] https://github.com/Lekensteyn/ltunify/blob/master/ltunify.c#L1164 https://github.com/Lekensteyn/ltunify/blob/master/ltunify.c#...

metadat 2y ago

Is this still possible in Linux? Or were the aforementioned syscalls removed?

kmeisthax 2y ago
Linux never removes syscalls.
- LegionMammal978 2y ago
  Though it does stop implementing obsolete syscalls on newer architectures. So the set of syscalls you can effectively use often rotates as the hardware treadmill turns.
  - dwattttt 2y ago
    And since they haven't been removed, just not implemented, you get the joy of finding out it doesn't work anymore when you try to use it, instead of when you build it!

upon_drumhead 2y ago

Are you sure? I know BSDs use kvm_getprocs, but I don't know what the comparable sys call would have been on Linux. FWIW, /proc was added in Linux v0.97.3, September 1992, which is early enough I couldn't find ps source for linux earlier then that date.

yjftsjthsd-h 2y ago
The post you're replying to didn't say "on Linux" - IIRC, unix ps worked by reading /dev/kmem or so Edit: My mistake, it was /dev/mem - https://github.com/lsahn-gh/unix-v6/blob/master/source/s2/ps.c#L100 https://github.com/lsahn-gh/unix-v6/blob/master/source/s2/ps...
- upon_drumhead 2y ago
  Agh, completely fair. The topic was about Linux and I incorrectly presumed that the original comment was made in the same context. My mistake.
donio 2y ago
I remember this too. Very early on there were both proc and a /dev/kmem versions of ps and maybe top too. Yep, found it: https://www.ibiblio.org/pub/historic-linux/ftp-archives/sunsite.unc.edu/Sep-29-1996/system/Status/ps/INDEX.html https://www.ibiblio.org/pub/historic-linux/ftp-archives/suns... And the Linux kernel CREDITS file has an entry mentioning it too: N: Rick Sladkey D: utility hacker: Emacs, NFS server, mount, kmem-ps, UPS debugger, strace, GDB [...]
dfox 2y ago
kvm_getprocs is in userspace library libkvm which abstracts away how that is done. And that library is primary meant for accessing /dev/kmem and crashdumps, with this functionality being bolted onto it (FreeBSD manapage even mentions that as a bug) and it works by examining the symbol table of running kernel and just walking the kernel datastructures. On FreeBSD there seems to be some trick where opening "/dev/null" instead of "/dev/kmem" causes the process "to not access kernel memory directly". Looking at the code it seeems to me that it means that libkvm will really open /dev/null and treat it as if it was /dev/kmem, which raises a question of exactly how that works. [Edit: apparently this works because in case of querying processes of running kernel, the code in kvm_proc.c does not read from the file and instead calls sysctl().]

rwmj 2y ago

The original 'ps' got the data by reading kernel memory directly (/dev/kmem or /dev/mem), and I'm old enough to remember systems doing that. I don't think there were any Unix-derivatives that used system calls? Minix is closest, in that you could query the 'mm' task using RPCs which are sort of system calls.