3 ms·

> It doesn't matter if it is the kernel or your application that calls dlopen(). My application calls mmap and mprotect(PROT_EXEC), not dlopen.

by a1369209993 6y ago

sgtnoodle 6y ago
Neat, memory mapping is a nifty concept. Are you loading a raw binary blob and directly jumping into it then? That's hardcore. I'm not familiar with the inner workings of dlopen, but I wouldn't be surprised if dlopen itself is mostly userspace code that knows how to interpret the ELF format, and internally it uses mmap as the actual system call. Maybe it has its own system call, though, to let the kernel do most of the work? I've seen some really cool uses of mmap. It's nice for reducing the number of memory copies when you're churning through a bunch of data on disk, and often necessary when using shared memory for low overhead inter-process communication. I worked on a project that needed publish-subscribe message passing semantics with fine-grained synchronization across dozens of processes, for example, and a lock-free shared memory "disrupter" queue structure was able to handle millions of messages per second, while sockets could only handle a few hundred thousand or so. I've seen some unfortunate uses as well. Sometimes folk like the idea of manipulating files on disk as arrays in memory, and use mmap even though there's no practical performance need for it. The code often times ends up harder to read than it would have had it used file descriptors. Also, the performance ends up worse because the implemented algorithm's access patterns unknowingly cause lots of page faults and disk seeks because the data is an "array". Often times there's an alternative way to do the same work sequentially with an additional in-memory data structure, and using file descriptors naturally biases one towards doing that.
pjmlp 6y ago
Try that on an OS that isn't a Linux distribution.
- a1369209993 6y ago
  Works fine on BSD (though it's been long enough that I forget which BSD) as long as you ignore the parts of the documentation written by facist morons. Haven't had occasion to beat proper linking behaviour into a Windows machine, but what I've heard suggests it'd be a pain the ass, yes. (Then again, Windows isn't a OS, it's a shitty GUI for DOS that bloated until it could read email. Kinda like OSX, except there isn't a vaguely decent OS buried under the manure.)
  - pjmlp 6y ago
    A systems programming language cannot afford to silo itself on a single OS solutions. There are plenty of OSes to choose from by the way, even your rant barely scratches the surface of the options available for any serious systems programming language.
    - a1369209993 6y ago
      > A systems programming language cannot afford to silo itself on a single OS solutions. If a OS (or alleged OS) demands that you dynamically link against libc (or anything else written in C) to interact with it, that problem is not the (non-C) programming language's fault. > your rant barely scratches the surface of the [OS] options available That's fair, but most of the not-linux I have experience with is microcontrollers or other things that are far enough from unix/posix that they don't have dlopen (mmap itself is hit-or-miss, but tends to be hit in the sorts of situations where I'd have occasion to dynamically load stuff in the first place), so there's not much I could do about that.

> It doesn't matter if it is the kernel or your application that calls dlopen(). My application calls mmap and mprotect(PROT_EXEC), not dlopen.

sgtnoodle 6y ago

Neat, memory mapping is a nifty concept. Are you loading a raw binary blob and directly jumping into it then? That's hardcore. I'm not familiar with the inner workings of dlopen, but I wouldn't be surprised if dlopen itself is mostly userspace code that knows how to interpret the ELF format, and internally it uses mmap as the actual system call. Maybe it has its own system call, though, to let the kernel do most of the work? I've seen some really cool uses of mmap. It's nice for reducing the number of memory copies when you're churning through a bunch of data on disk, and often necessary when using shared memory for low overhead inter-process communication. I worked on a project that needed publish-subscribe message passing semantics with fine-grained synchronization across dozens of processes, for example, and a lock-free shared memory "disrupter" queue structure was able to handle millions of messages per second, while sockets could only handle a few hundred thousand or so. I've seen some unfortunate uses as well. Sometimes folk like the idea of manipulating files on disk as arrays in memory, and use mmap even though there's no practical performance need for it. The code often times ends up harder to read than it would have had it used file descriptors. Also, the performance ends up worse because the implemented algorithm's access patterns unknowingly cause lots of page faults and disk seeks because the data is an "array". Often times there's an alternative way to do the same work sequentially with an additional in-memory data structure, and using file descriptors naturally biases one towards doing that.

pjmlp 6y ago

Try that on an OS that isn't a Linux distribution.

a1369209993 6y ago
Works fine on BSD (though it's been long enough that I forget which BSD) as long as you ignore the parts of the documentation written by facist morons. Haven't had occasion to beat proper linking behaviour into a Windows machine, but what I've heard suggests it'd be a pain the ass, yes. (Then again, Windows isn't a OS, it's a shitty GUI for DOS that bloated until it could read email. Kinda like OSX, except there isn't a vaguely decent OS buried under the manure.)
- pjmlp 6y ago
  A systems programming language cannot afford to silo itself on a single OS solutions. There are plenty of OSes to choose from by the way, even your rant barely scratches the surface of the options available for any serious systems programming language.
  - a1369209993 6y ago
    > A systems programming language cannot afford to silo itself on a single OS solutions. If a OS (or alleged OS) demands that you dynamically link against libc (or anything else written in C) to interact with it, that problem is not the (non-C) programming language's fault. > your rant barely scratches the surface of the [OS] options available That's fair, but most of the not-linux I have experience with is microcontrollers or other things that are far enough from unix/posix that they don't have dlopen (mmap itself is hit-or-miss, but tends to be hit in the sorts of situations where I'd have occasion to dynamically load stuff in the first place), so there's not much I could do about that.