3 ms·
Very good article, facts looked correct and it had useful advice. I'd add, keep things local. Don't access memory (or cache) outside core (L1 & L2), NUMA regio
by vardump 7y ago
Very good article, facts looked correct and it had useful advice.
I'd add, keep things local. Don't access memory (or cache) outside core (L1 & L2), NUMA region or processor socket boundary unnecessarily.
Keep networking, GPU, etc. code in same NUMA region where the physical adapters are.
Use memory like tape, stream through. CPU branch predictors love that kind of access pattern.
Oh, and perhaps most importantly: use a profiler that can access CPU internal performance counters. Do this on different system types, from low power laptops to servers with 2 or more CPU sockets.
One annoying thing, though. Remember that the fastest thing in a microbenchmark might not be the fastest thing on a real system when different code modules fight for shared limited resources, like memory bandwidth, caches and inter-core communication links.
- vardump 7y ago> CPU branch predictors love that kind of access pattern. My brain's "branch predictor" had some sort of issue. CPU prefetchers of course. :-)
- ibrault 7y agoCould you elaborate on what it means to "use memory like tape"?
- tyingq 7y agoSequential access instead of random. Like arrays instead of linked lists, for example.
- vardump 7y agoSequential access patterns, forward or backward. Repeating predictable gaps are ok, but do remember minimum unit that can be read from memory is a cache line. So if you read one byte, you'll read 64 bytes on modern x86.
- ajross 7y agoDRAM latency for a random access read can get into the low hundreds of cycles on modern multi-socket devices. But the streaming bandwidth remains very high. Cache systems will routinely prefetch the next block ahead of an access if they detect that memory is being used sequentially, eliminating a huge chunk of that pipeline stall.