4 ms·

I'm not sure the conclusion that vector instructions are responsible for the speed-up is correct. Both implementations seem to use ERMS (using REP MOVSB instruc

I'm not sure the conclusion that vector instructions are responsible for the speed-up is correct. Both implementations seem to use ERMS (using REP MOVSB instructions)[0]. Looking at the profiles, the syscall implementation spends time in the [xfs] driver (even in the long test), while the mmap implementation does not. It appears the real speed-up is related to how memory-mapped pages interact with the buffer cache. I might be misunderstanding things. What is really going on here? [0] Lines 56 and 180 here: http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S;h=cf645dd7ffe8785b731cd1332e7f504ab8f5ba8b;hb=88b57b8ed41d5ecf2e1bdfc19556f9246a665ebb http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86_...

petters 6y ago
I thought this strange as well. The author even directly links to the source code where REP MOVSB is used.
sltkr 6y ago
The copy hypothesis also doesn't explain the difference between warm and cold start performance, at least for the "random read" case. For warm start, the speed for read vs mmap is 3 GB/s vs 6 GB/s (i.e. mmap is 2x faster). For cold start, that changes to 0.05 GB/s vs 0.25 GB/s (i.e. mmap is 5x faster). If the only significant difference between read() and mmap() is how data is copied, then shouldn't the performance gap get much smaller in the cold start scenario? After all, you're adding the identical (and pretty large) overhead of reading from disk. Instead, the gap gets much larger. That's strange. (The sequential read case looks more reasonable.) So there must be something else going on besides more efficient copying with mmap.
scaramanga 6y ago
Long story short. Spectre mitigations make ret_from_syscall much slower. Much slower than taking a page-fault.

4 ms·

I'm not sure the conclusion that vector instructions are responsible for the speed-up is correct. Both implementations seem to use ERMS (using REP MOVSB instruc

by justin_ 6y ago

petters 6y ago
I thought this strange as well. The author even directly links to the source code where REP MOVSB is used.
sltkr 6y ago
The copy hypothesis also doesn't explain the difference between warm and cold start performance, at least for the "random read" case. For warm start, the speed for read vs mmap is 3 GB/s vs 6 GB/s (i.e. mmap is 2x faster). For cold start, that changes to 0.05 GB/s vs 0.25 GB/s (i.e. mmap is 5x faster). If the only significant difference between read() and mmap() is how data is copied, then shouldn't the performance gap get much smaller in the cold start scenario? After all, you're adding the identical (and pretty large) overhead of reading from disk. Instead, the gap gets much larger. That's strange. (The sequential read case looks more reasonable.) So there must be something else going on besides more efficient copying with mmap.
scaramanga 6y ago
Long story short. Spectre mitigations make ret_from_syscall much slower. Much slower than taking a page-fault.