3 ms·

Is it a scarce resource? For what it’s worth when the caching was removed from glibc it caused a performance problem in libsystemd which had a tight loop to det

Is it a scarce resource? For what it’s worth when the caching was removed from glibc it caused a performance problem in libsystemd which had a tight loop to detect when a process was forked. So while this may be true in many cases, there’s clearly cases where it’s not and your answer doesn’t explain why getpid isn’t a vdso.

dmitrygr 3y ago
Memory is always a scarce resource. However, a big you make the vdso, that amount of memory, multiplied by the number of processes is the amount wasted. In reality, it will be more, because of the required additional page tables to map it at an address range that is not commonly used. Also, what sort of raving lunatic had a busy loop checking the current process PID? Who allowed such a person access to a compiler? That seems like the bigger problem.
- charcircuit 3y ago
  >multiplied by the number of processes Why aren't the pages shared like other dynamic libraries?
  - fweimer 3y ago
    If we put the PID into the vDSO, this data page can no longer be shared among different processes (at least not in the same container). We could use the rseq area for PID and TID.
  - dmitrygr 3y ago
    Because you just asked to put data there that would differ per process?
    - planede 3y ago
      That doesn't mean that code pages couldn't be shared. The data that differs per process is a single number in this case. Having said that I agree that it's probably not worth it to put getpid() into vDSO.
      - dmitrygr 3y ago
        A number here, a number there, before you know it, it is a lot. If we're going to add all sorts of dross to the vdso, like getpid(), who knows what else will end up there?
- vlovich123 3y ago
  I would assume good faith on the part of systemd developers. Afaik they’re as talented as any other top tier group of developers. Somehow I doubt the code for getpid as a vdso would meaningfully impact the amount of memory used on a running system nor would I expect any meaningful impact to the number of page table entries. Do you have any supporting evidence for your claim otherwise?
  - actionfromafar 3y ago
    Good faith, yes. But even talented people get in strange waters sometimes.
    - vlovich123 3y ago
      No, libsystemd did this because it’s a library. They went with pthread_at_fork as an alternative to detect when the application code forked, but it’s not without its own set of risks. Of course they could have required the user of the API supply the PID everywhere to avoid this but it’s possible even that wouldn’t solve the problem and it complicates the UI and the UI of the library API is equally important as other technical details - it’s a design tradeoff and judging it as some kind of correctness thing without having a clear picture and insulting the authors out of hand is weird behavior.
      - actionfromafar 3y ago
        I agree with everything you said except that "systemd" and "insulting" don't go together like peanut butter and jelly. Insults seem to be the fuel the project is run on.
pjc50 3y ago
> which had a tight loop to detect when a process was forked. This sounds so mad it needs more context. You don't need to detect when a process is forked, it's an action issued from inside the process?
- planede 3y ago
  Maybe the problem is that it is "lib"systemd, meaning it doesn't really have control over when the process forks, it's the application code's responsibility. Although I would probably also avoid this design, but that would probably make the library interface a bit unfriendly, with the application needing to pass its own pid to library functions where appropriate.
  - actionfromafar 3y ago
    The audacity of the design to detect a pid change by looking at it in a loop, is interesting. I don't think I could have come up with something like that. Feels very "sync the web DOM by traversing it and comparing with a shadow DOM".
  - dmitrygr 3y ago
    If only we had an API to tell us when a fork happened. We could place it into the usual library for threading stuff: pthread, and could call it, oh I don't know, maybe pthread_atfork() https://man7.org/linux/man-pages/man3/pthread_atfork.3.html https://man7.org/linux/man-pages/man3/pthread_atfork.3.html
    - gpderetta 3y ago
      It is mentioned in the glibc bug tracker entry as a replacement to the polling. systemd didn't use it because they thought it would be impossible for a library using to be dlclose'd as there is no way to unregister the callback. But apparently glibc has (undocumented) magic under the hood to take care of it.

dmitrygr 3y ago

Memory is always a scarce resource. However, a big you make the vdso, that amount of memory, multiplied by the number of processes is the amount wasted. In reality, it will be more, because of the required additional page tables to map it at an address range that is not commonly used. Also, what sort of raving lunatic had a busy loop checking the current process PID? Who allowed such a person access to a compiler? That seems like the bigger problem.

charcircuit 3y ago
>multiplied by the number of processes Why aren't the pages shared like other dynamic libraries?
- fweimer 3y ago
  If we put the PID into the vDSO, this data page can no longer be shared among different processes (at least not in the same container). We could use the rseq area for PID and TID.
- dmitrygr 3y ago
  Because you just asked to put data there that would differ per process?
  - planede 3y ago
    That doesn't mean that code pages couldn't be shared. The data that differs per process is a single number in this case. Having said that I agree that it's probably not worth it to put getpid() into vDSO.
    - dmitrygr 3y ago
      A number here, a number there, before you know it, it is a lot. If we're going to add all sorts of dross to the vdso, like getpid(), who knows what else will end up there?
vlovich123 3y ago
I would assume good faith on the part of systemd developers. Afaik they’re as talented as any other top tier group of developers. Somehow I doubt the code for getpid as a vdso would meaningfully impact the amount of memory used on a running system nor would I expect any meaningful impact to the number of page table entries. Do you have any supporting evidence for your claim otherwise?
- actionfromafar 3y ago
  Good faith, yes. But even talented people get in strange waters sometimes.
  - vlovich123 3y ago
    No, libsystemd did this because it’s a library. They went with pthread_at_fork as an alternative to detect when the application code forked, but it’s not without its own set of risks. Of course they could have required the user of the API supply the PID everywhere to avoid this but it’s possible even that wouldn’t solve the problem and it complicates the UI and the UI of the library API is equally important as other technical details - it’s a design tradeoff and judging it as some kind of correctness thing without having a clear picture and insulting the authors out of hand is weird behavior.
    - actionfromafar 3y ago
      I agree with everything you said except that "systemd" and "insulting" don't go together like peanut butter and jelly. Insults seem to be the fuel the project is run on.

pjc50 3y ago

> which had a tight loop to detect when a process was forked. This sounds so mad it needs more context. You don't need to detect when a process is forked, it's an action issued from inside the process?

planede 3y ago
Maybe the problem is that it is "lib"systemd, meaning it doesn't really have control over when the process forks, it's the application code's responsibility. Although I would probably also avoid this design, but that would probably make the library interface a bit unfriendly, with the application needing to pass its own pid to library functions where appropriate.
- actionfromafar 3y ago
  The audacity of the design to detect a pid change by looking at it in a loop, is interesting. I don't think I could have come up with something like that. Feels very "sync the web DOM by traversing it and comparing with a shadow DOM".
- dmitrygr 3y ago
  If only we had an API to tell us when a fork happened. We could place it into the usual library for threading stuff: pthread, and could call it, oh I don't know, maybe pthread_atfork() https://man7.org/linux/man-pages/man3/pthread_atfork.3.html https://man7.org/linux/man-pages/man3/pthread_atfork.3.html
  - gpderetta 3y ago
    It is mentioned in the glibc bug tracker entry as a replacement to the polling. systemd didn't use it because they thought it would be impossible for a library using to be dlclose'd as there is no way to unregister the callback. But apparently glibc has (undocumented) magic under the hood to take care of it.