4 ms·

Where do you think those zeros come from? The kernel demand faults that mmap, and on each fault, has to find a page containing zero. Sometimes it has to make on

Where do you think those zeros come from? The kernel demand faults that mmap, and on each fault, has to find a page containing zero. Sometimes it has to make one.

gumby 7y ago
Yes but the large array needs to allocate that memory already.
- quotemstr 7y ago
  Sure. I'm just saying that the kernel doesn't have some kind of magic capability that lets it zero memory faster than userspace can. It does have a small free-list of already-zero pages, granted, but a really big allocation will deplete that list, and that list needs to be replenished somehow. For Linux, I want to grant MAP_UNINITIALIZED access to unprivileged users (keep reading) but actually zero pages it gives to these unprivileged MAP_UNINITIALIZED --- users unless it knows that a page doesn't contain privileged information. For example, if a process munmap()s a particular region and then immediately mmap()s a different one, and no other process has ever mapped that page, there's no harm in allowing the process to see its own previous page content.
  - BeeOnRope 7y ago
    Heh, I've had similar thoughts about "safe MAP_UNINITIALIZED", although the demand for this feature is probably pretty low since you can do much the same thing in userspace (just don't return that memory in the first place). Of course, the kernel way has some advantages: - You are making the memory available to the rest of the system if needed. - It could give you other types of pages w/o privileged info, e.g., page cache pages from files you have permission to read, or something like that.
- BeeOnRope 7y ago
  Yes, if you know you want zeroed memory, calloc() is best: because a smart userspace allocator will have two paths: if it is able to re-use memory already allocated and freed from userspace, it will explicitly zero it (probably with a memset) since it it is not (guaranteed to be) zero. OTOH, if it is asking the OS for fresh memory to satisfy the request, or is satisfying the request from an earlier allocation returned by the OS (which hasn't yet been fully used), it knows the memory is zero and can just return it directly. In that scenario calloc() is as fast as malloc(), yet it provides the additional guarantee of zeroed memory. Unfortunately, calloc() and C++ don't really place nice together, and C++ has no equivalent concept to calloc() in its allocator model (the same complaint applies to realloc() which would otherwise be very useful).

Where do you think those zeros come from? The kernel demand faults that mmap, and on each fault, has to find a page containing zero. Sometimes it has to make one.

gumby 7y ago

Yes but the large array needs to allocate that memory already.

quotemstr 7y ago
Sure. I'm just saying that the kernel doesn't have some kind of magic capability that lets it zero memory faster than userspace can. It does have a small free-list of already-zero pages, granted, but a really big allocation will deplete that list, and that list needs to be replenished somehow. For Linux, I want to grant MAP_UNINITIALIZED access to unprivileged users (keep reading) but actually zero pages it gives to these unprivileged MAP_UNINITIALIZED --- users unless it knows that a page doesn't contain privileged information. For example, if a process munmap()s a particular region and then immediately mmap()s a different one, and no other process has ever mapped that page, there's no harm in allowing the process to see its own previous page content.
- BeeOnRope 7y ago
  Heh, I've had similar thoughts about "safe MAP_UNINITIALIZED", although the demand for this feature is probably pretty low since you can do much the same thing in userspace (just don't return that memory in the first place). Of course, the kernel way has some advantages: - You are making the memory available to the rest of the system if needed. - It could give you other types of pages w/o privileged info, e.g., page cache pages from files you have permission to read, or something like that.
BeeOnRope 7y ago
Yes, if you know you want zeroed memory, calloc() is best: because a smart userspace allocator will have two paths: if it is able to re-use memory already allocated and freed from userspace, it will explicitly zero it (probably with a memset) since it it is not (guaranteed to be) zero. OTOH, if it is asking the OS for fresh memory to satisfy the request, or is satisfying the request from an earlier allocation returned by the OS (which hasn't yet been fully used), it knows the memory is zero and can just return it directly. In that scenario calloc() is as fast as malloc(), yet it provides the additional guarantee of zeroed memory. Unfortunately, calloc() and C++ don't really place nice together, and C++ has no equivalent concept to calloc() in its allocator model (the same complaint applies to realloc() which would otherwise be very useful).