5 ms·
The meat of it is: the functions (for copying data) used for syscall and mmap are very different, and not only in the name. __memmove_avx_unaligned_erms, call
by noncoml 6y ago
The meat of it is:
the functions (for copying data) used for syscall and mmap are very different, and not only in the name.
__memmove_avx_unaligned_erms, called in the mmap experiment, is implemented using Advanced Vector Extensions (AVX) (here is the source code of the functions that it relies on).
The implementation of copy_user_enhanced_fast_string, on the other hand, is much more modest. That, in my opinion, is the huge reason why mmap is faster. Using wide vector instructions for data copying effectively utilizes the memory bandwidth, and combined with CPU pre-fetching makes mmap really really fast.
Why can’t the kernel implementation use AVX? Well, if it did, then it would have to save and restore those registers on each system call, and that would make domain crossing even more expensive. So this was a conscious decision in the Linux kernel
- cwzwarich 6y agoWouldn't REP MOVSB be as fast as an AVX memcpy for 4 KB sizes on recent Intel CPUs?
- adzm 6y agoIt should be, I think, though it's a complicated question whose answer varies on so much, cpu architecture and how it is used. There is a great discussion on it here, too. https://stackoverflow.com/questions/43343231/enhanced-rep-movsb-for-memcpy https://stackoverflow.com/questions/43343231/enhanced-rep-mo...
- justin_ 6y agoThe glibc implementation[0] uses Enhanced REP MOVSB when the array is long enough. It takes a few cycles to start up the ERMS feature, so it's only used on longer arrays. Edit: Wait a minute... if this is true, then how can AVX be responsible for the speed up? Is it related to the size of the buffers being copied internally? [0] Line 48 here: http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S;h=cf645dd7ffe8785b731cd1332e7f504ab8f5ba8b;hb=88b57b8ed41d5ecf2e1bdfc19556f9246a665ebb http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86_...
- JoshTriplett 6y ago> The glibc implementation[0] uses Enhanced REP MOVSB when the array is long enough. It takes a few cycles to start up the ERMS feature, so it's only used on longer arrays. That isn't true anymore either, on sufficiently recent processors with "Fast Short REP MOVSB (FSRM)". If the FSRM bit is set (which it is on Ice Lake and newer), you can just always use REP MOVSB.
- jabl 6y agoStill waiting for the "Yes, This Time We Really Mean It Fast REP MOVSB" (YTTWRMIFRM) bit. More seriously, if REP MOVSB can be counted on always being the fastest method that's fantastic. One thing that easily gets forgotten in microbenchmarking is I$ pollution by those fancy unrolled SIMD loops with 147 special cases.
- magicalhippo 6y agoEver since reading the advice to avoid these CISC-y instructions (IIRC right back to the first Pentium), I've been wondering why. Like what made them not implement the best possible microcode for that? I mean sure for very short loops I guess I can see unrolling being faster than accessing microcode, but yeah.
- fangyrn 6y agoI'm a bit of an idiot, when I think of AVX I think of something that speeds up computation (specifically matrice stuff), not memory access. How wrong am I?
- jstimpfle 6y agoIt's a set of SIMD (single instruction, multiple data) extensions to the amd64 instruction set. They allow you to operate on larger chunks of data with a single instruction - for example, do 16 integer multiplications in parallel, etc.
- jeffbee 6y agoIts registers are just larger. The way x86 moves memory is through registers, register-to-register or register-to/from-memory. The AVX registers move up to 64 bytes in one move. A general purpose register moves at most 8 bytes.
- aliceryhl 6y agoAVX is useful for both.
- magicalhippo 6y agoBack in the days, this was on my 486 IIRC, using floating-point to copy was faster than using the regular integer instructions for pretty much the same reason, you could copy 64bit at a time rather than "just" 32bit.
- pengaru 6y agoYou're completely ignoring MAP_POPULATE being used to aggressively asynchronously read-ahead the data on the kernel side, which the read-based implementation isn't attempting any equivalent of (posix_fadvise(2) and readahead(2) come to mind). Frankly TFA is relatively useless as an mmap vs. read comparison.
- noncoml 6y agoI am not the author :)
- jiggawatts 6y ago> if it did, then it would have to save and restore those registers on each system call Stupid question: Does the entire AVX state need to be saved and restored for a memcpy routine? It seems like it would be sufficient to save at most a couple of registers for a tight copy loop. Even for 512-byte copies that ought to be a net win...