3 ms·

First of all, you only need to touch a single byte of each page to populate it. Besides, this approach doesn't ensure that the backing pages is allocated, for i

by htfy96 7y ago

First of all, you only need to touch a single byte of each page to populate it. Besides, this approach doesn't ensure that the backing pages is allocated, for it's (technically) legal to only allocate one physical zero page after the calloc. Therefore, you must use OS-dependent primitives to force eager allocation. On linux this can be achieved by two flags for mmap: MAP_POPULATE initializes the page table entries eagerly (but doesn't guarantee a backing store allocation), and MAP_LOCKED for touching all pages (just like the calloc, but at kernel level and more deterministic). Note that MAP_LOCKED doesn't raise error for touching errors due to low memory, so you still need to touch it again at user-level.

OskarS 7y ago
Why wouldn’t the “volatile” approach be valid? I mean, the only way to guarantee that you can write to memory without error (or to make sure it’s backed by actual memory pages) is to actually write to it, and the way to guarantee you’re doing that in a platform independent way in C is to use volatile, right? Are you really saying that this is not possible with malloc, you have to drop down to platform-dependent system calls like mmap?
- codys 7y ago
  Because it's legal for a platform to perform page deduplication as long as from the perspective of the program the correct operating model is followed, many operating systems (including Linux) have a "zero page" that is used for _all_ zeroed memory blocks in the system. Writing a non zero value then would replace it with a different page. This means that writing zero is unlikely to cause a real allocation to occur. Writing a non zero value may be more likely to cause a page allocation. That said: it can be entirely legal for a platform to do more page deduplication than just special casing the zero page. Linux has some kernel tooling called KSM (kernel same page merging).

by htfy96 7y ago

OskarS 7y ago

Why wouldn’t the “volatile” approach be valid? I mean, the only way to guarantee that you can write to memory without error (or to make sure it’s backed by actual memory pages) is to actually write to it, and the way to guarantee you’re doing that in a platform independent way in C is to use volatile, right? Are you really saying that this is not possible with malloc, you have to drop down to platform-dependent system calls like mmap?

codys 7y ago
Because it's legal for a platform to perform page deduplication as long as from the perspective of the program the correct operating model is followed, many operating systems (including Linux) have a "zero page" that is used for _all_ zeroed memory blocks in the system. Writing a non zero value then would replace it with a different page. This means that writing zero is unlikely to cause a real allocation to occur. Writing a non zero value may be more likely to cause a page allocation. That said: it can be entirely legal for a platform to do more page deduplication than just special casing the zero page. Linux has some kernel tooling called KSM (kernel same page merging).