6 ms·
1/4 second to plow through 1 GB of memory is certainly fast compared to some things (like a human reader), but it seems oddly slow relative to what a modern com
by nkurz 13y ago
1/4 second to plow through 1 GB of memory is certainly fast compared to some things (like a human reader), but it seems oddly slow relative to what a modern computer should be capable off. Sure, it's a lot faster than a human, but that's only 4 GB/s! A number of comments here have mentioned adding some prefetch statements, but for linear access like this that's usually not going to help much. The real issue (if I may be so bold) is all the TLB misses. Let's measure.
Here's the starting point on my test system, an Intel Sandy Bridge E5-1620 with 1600 MHz quad-channel RAM:
$ perf stat bytesum 1gb_file
Size: 1073741824
The answer is: 4
Performance counter stats for 'bytesum 1gb_file':
262,315 page-faults # 1.127 M/sec
835,999,671 cycles # 3.593 GHz
475,721,488 stalled-cycles-frontend # 56.90% frontend cycles idle
328,373,783 stalled-cycles-backend # 39.28% backend cycles idle
1,035,850,414 instructions # 1.24 insns per cycle
0.232998484 seconds time elapsed
Hmm, those 260,000 page-faults don't look good. And we've got 40% idle cycles on the backend. Let's try switching to 1 GB hugepages to see how much of a difference it makes:
$ perf stat hugepage 1gb_file
Size: 1073741824
The answer is: 4
Performance counter stats for 'hugepage 1gb_file':
132 page-faults # 0.001 M/sec
387,061,957 cycles # 3.593 GHz
185,238,423 stalled-cycles-frontend # 47.86% frontend cycles idle
87,548,536 stalled-cycles-backend # 22.62% backend cycles idle
805,869,978 instructions # 2.08 insns per cycle
0.108025218 seconds time elapsed
It's entirely possible that I've done something stupid, but the checksum comes out right, but the 10 GB/s read speed is getting closer to what I'd expect for this machine. Using these 1 GB pages for the contents of a file is a bit tricky, since they need to be allocated off the hugetlbfs filesystem that does not allow writes and requires that the pages be allocated at boot time. My solution was a run one program that creates a shared map, copy the file in, pause that program, and then have the bytesum program read the copy that uses the 1 GB pages.
Now that we've got the page faults out of the way, the prefetch suggestion becomes more useful:
$ perf stat hugepage_prefetch 1gb_file
Size: 1073741824
The answer is: 4
Performance counter stats for 'hugepage_prefetch 1gb_file':
132 page-faults # 0.002 M/sec
265,037,039 cycles # 3.592 GHz
116,666,382 stalled-cycles-frontend # 44.02% frontend cycles idle
34,206,914 stalled-cycles-backend # 12.91% backend cycles idle
579,326,557 instructions # 2.19 insns per cycle
0.074032221 seconds time elapsed
That gets us up to 14.5 GB/s, which is more reasonable for a a single stream read on a single core. Based on prior knowledge of this machine, I'm issuing one prefetch 512B ahead per 128B double-cacheline. Why one per 128B? Because the hardware "buddy prefetcher" is grabbing two lines at a time. Why do prefetches help? Because the hardware "stream prefetcher" doesn't know that it's dealing with 1 GB pages, and otherwise won't prefetch across 4K boundaries.
What would it take to speed it up further? I'm not sure. Suggestions (and independent confirmations or refutations) welcome. The most I've been able to reach in other circumstances is about 18 GB/s by doing multiple streams with interleaved reads, which allows the processor to take better advantage of open RAM banks. The next limiting factor (I think) is the number of line fill buffers (10 per core) combined with the cache latency in accordance with Little's Law.
- exDM69 13y agoGood stuff! The low numbers (4 GB/s, one tenth the available memory bandwidth) that we're seeing sounds like a lot of time is wasted in page faults (and indeed, perf stat confirms it). However, the solution you propose sounds difficult, you need a special file system and need to know what kind of page sizes the CPU supports. Can you think of ways to get reduce the number of page faults from inside the application itself? Or methods that would be portable to architectures with different page sizes? I tried a simple call to madvise and posix_fadvise to inform the operating system ahead of time that I am going to need the memory but that did not have any effect on the number of page faults. Any other tips for squeezing some more perf out? Did you happen to do any cache miss stats your benchmarks?
- pbsd 13y agoYou get a similar effect to huge pages by passing MAP_POPULATE to mmap, which pre-faults all pages itself before returning.
- exDM69 13y agoAh, thanks! Using MAP_POPULATE, I'm seeing this go from 104,276 page faults down to 130 page faults. However, I'm only seeing a modest increase in overall performance, goes from 160 ms to 130 ms for my ~500 MB test file. MAP_POPULATE + MAP_NONBLOCK is as bad as not using MAP_POPULATE (same number of page faults). Could the number of TLB misses from using huge pages be a major factor too? Page faults alone do not explain the huge perf difference. (edit: perf stat tells me I'm only missing 0.19% of TLB lookups and only 3% dcache misses) I also tried with a 4 GB file and I see two interesting effects. First of all, using MAP_POPULATE is slower, which I presume is because physical memory is exhausted (6 G memory total). Secondly, I see less than 100k page faults which suggests that bigger pages (2M or 1G?) are being used (10x bigger file, a little less page faults).
- dchichkov 13y agoHow about the following? HUGETLB_MORECORE=yes LD_PRELOAD=libhugetlbfs.so