3 ms·
> If you malloc a big buffer, almost certainly page aligned anyway. Not necessarily. Remember that malloc() must produce a pointer which can later be passed to
by kentonv 6y ago
> If you malloc a big buffer, almost certainly page aligned anyway.
Not necessarily. Remember that malloc() must produce a pointer which can later be passed to free() without any other details. free() needs some way to figure out, at the very least, how big the allocation is, to properly free it. Many allocators, such as glibc's, accomplish this by placing metadata in the bytes immediately before the allocated block -- which implies that a large allocation won't be page-aligned.
Some other allocators, such as tcmalloc, manage to place the metadata somewhere else, and in that case maybe they return page-aligned buffers, but it's hardly safe to assume.
(I just ran a quick test and verified -- a 4MB malloc() under glibc is typically 16 bytes off of the page boundary, but under tcmalloc it is aligned.)
> 2. You can try the same with mmap, but large files won't fit in (32 bit) memory, and read() is cheaper than mapping and unmapping as you go.
This thread is discussing the claim that the Linux kernel transparently "optimizes" large read() calls into mmap(), with me arguing that doesn't make a lot of sense as an optimization.
If you're saying that this "optimization" would actually be slower, then that supports my point.
You may be right. munmap() (or remapping with MAP_FIXED) triggers TLB shootdowns, which are rather expensive in multithreaded programs. This is another reason why "optimizing" read() to mmap() sounds awfully dubious to me.
That said, shootdowns would have a constant cost per read(), independent of the size. For a sufficiently gigantic read, the cost of the shootdown would be less than the cost of copying data into the target memory, so the "optimization" would possibly start to make sense at that point?