5 ms·

A few thoughts: * The space savings are moot as other processes such as the daemons are going to load libc into virtual memory anyway, and the kernel shares li

by leeter 10y ago

A few thoughts: * The space savings are moot as other processes such as the daemons are going to load libc into virtual memory anyway, and the kernel shares libc's page among all processes. * This adds a lot of LOC you have to maintain, instead of shoving it off on the compiler/libc vendor, this increases the chance of bugs. * This will prevent the use of VDSOs to optimize high volume system calls like gettimeofday. * It's still probably good to know how these happen, even if you're not doing them yourself. * The only place this would really see benefit is in a single process environment, however in those cases I would suggest a unikernel anyway for simplicity sake.

taeric 10y ago
I'd be interested to see benchmarks to consider point one. At face value, I fully agree with your point and doubt it has any benefit. However, it is less that has to be swapped in for your program to run. Would be curious if this has an odd cache friendliness for an application. The tooling answer to this, it seems, would be to support statically linked libraries. But again, I would want to see numbers before personally worrying about this.
- leeter 10y ago
  So a few things to consider in regards to point 1: * Your executable is going to be loaded as a whole page regardless of how large it is, on most platforms this means you'll need at least 4k of User VM. * You'll need a page table, which has its own overhead. If someone was going to push back on the libc assertion I'd expect it here, a the PTEs for libc and and the VDSOs cannot be shared between processes (as far as I'm aware). * I would expect it to in theory RUN faster assuming it was a small toy program like the example, this is because there is less work to be done even with shared pages.
  - taeric 10y ago
    Right, my question is along the lines of avoiding the pages of libc. An easy question here would be how many pages libc takes up. I'm assuming not many, but more than one. I think there is a strong argument that this page is often already paged into memory from everyone using it. However, if the function you used from it would have fit in the pages you were already using for your application, I could imagine some benefit. I continue to stress, though, that this is just imagined. Numbers would be first thing I would have to collect before acting on this. (And I hope it doesn't sound like I am tasking you or anyone else with this. That is not my intent.)
    - leeter 10y ago
      At least two is the best answer I can give without specifying a specific libc and architecture. Something to think about is that the binary size of libc is only half the story. Even if the executable portion of libc fits in a single page. A libc implementation has a lot of per thread and per process statics it holds onto. A good libc developer could actually put these into separate pages based on how often they change. In other words, if a static only is ever set once then coalesce it into a page with other statics that are only set once and are not process dependent. Why that's important: because of fork, when fork creates a new process it sets the parent processes pages read only and then preforms copy on write when they are modified. In theory you can share both the binary and some of the statics between all the processes.
jstimpfle 10y ago
He spoke about the space savings of the binary itself (which must be debug things and statically linked code). There is no sharing of that , unless multiple processes run that same binary.
- leeter 10y ago
  Unless the binary size exceeds VM page size this has no real value even in the unshared case. Because the OS is still going to have to map a full page for the executable to be loaded into. That said techniques to allow a binary to fit under page size do have value; as the unused extra page can be used for other things.
  - jstimpfle 10y ago
    That's totally true, and there is more overhead associated to processes. 8K executables are negligible. Some other languages produce executables in the dozens of megabytes.
  - Symmetry 10y ago
    The OS may map a full page for the binary but I'd expect that the time to load the binary from disk to memory would be saved. And once in memory the program would certainly take up fewer cache lines when executed.
    - slededit 10y ago
      At 4K the majority of the load from disk will be seek time (even on an SSD).
bogomipz 10y ago
Can you elaborate on why this "will prevent the use of VDSOs"? Why can't I just call gettimeofday() which will access the vdso page mapped into my address space?
- leeter 10y ago
  Because the code that knows how to call the VDSOs is in libc, and how VDSOs work is very architecture dependent. Not saying you couldn't in theory do it. But at that point you might as well just load libc.

by leeter 10y ago

taeric 10y ago

I'd be interested to see benchmarks to consider point one. At face value, I fully agree with your point and doubt it has any benefit. However, it is less that has to be swapped in for your program to run. Would be curious if this has an odd cache friendliness for an application. The tooling answer to this, it seems, would be to support statically linked libraries. But again, I would want to see numbers before personally worrying about this.

leeter 10y ago
So a few things to consider in regards to point 1: * Your executable is going to be loaded as a whole page regardless of how large it is, on most platforms this means you'll need at least 4k of User VM. * You'll need a page table, which has its own overhead. If someone was going to push back on the libc assertion I'd expect it here, a the PTEs for libc and and the VDSOs cannot be shared between processes (as far as I'm aware). * I would expect it to in theory RUN faster assuming it was a small toy program like the example, this is because there is less work to be done even with shared pages.
- taeric 10y ago
  Right, my question is along the lines of avoiding the pages of libc. An easy question here would be how many pages libc takes up. I'm assuming not many, but more than one. I think there is a strong argument that this page is often already paged into memory from everyone using it. However, if the function you used from it would have fit in the pages you were already using for your application, I could imagine some benefit. I continue to stress, though, that this is just imagined. Numbers would be first thing I would have to collect before acting on this. (And I hope it doesn't sound like I am tasking you or anyone else with this. That is not my intent.)
  - leeter 10y ago
    At least two is the best answer I can give without specifying a specific libc and architecture. Something to think about is that the binary size of libc is only half the story. Even if the executable portion of libc fits in a single page. A libc implementation has a lot of per thread and per process statics it holds onto. A good libc developer could actually put these into separate pages based on how often they change. In other words, if a static only is ever set once then coalesce it into a page with other statics that are only set once and are not process dependent. Why that's important: because of fork, when fork creates a new process it sets the parent processes pages read only and then preforms copy on write when they are modified. In theory you can share both the binary and some of the statics between all the processes.

jstimpfle 10y ago

He spoke about the space savings of the binary itself (which must be debug things and statically linked code). There is no sharing of that , unless multiple processes run that same binary.

leeter 10y ago
Unless the binary size exceeds VM page size this has no real value even in the unshared case. Because the OS is still going to have to map a full page for the executable to be loaded into. That said techniques to allow a binary to fit under page size do have value; as the unused extra page can be used for other things.
- jstimpfle 10y ago
  That's totally true, and there is more overhead associated to processes. 8K executables are negligible. Some other languages produce executables in the dozens of megabytes.
- Symmetry 10y ago
  The OS may map a full page for the binary but I'd expect that the time to load the binary from disk to memory would be saved. And once in memory the program would certainly take up fewer cache lines when executed.
  - slededit 10y ago
    At 4K the majority of the load from disk will be seek time (even on an SSD).

bogomipz 10y ago

Can you elaborate on why this "will prevent the use of VDSOs"? Why can't I just call gettimeofday() which will access the vdso page mapped into my address space?

leeter 10y ago
Because the code that knows how to call the VDSOs is in libc, and how VDSOs work is very architecture dependent. Not saying you couldn't in theory do it. But at that point you might as well just load libc.