2 ms·
One critical thing I wish I could go back and remind my younger self: The cache is shared across processes. The net result is that cache latency/hit rate is h
by chris_va 11y ago
One critical thing I wish I could go back and remind my younger self:
The cache is shared across processes.
The net result is that cache latency/hit rate is highly dependent on what else is using the machine. CPU bound tasks (like, say search engine backends) tend to leave extra storage space (say for things like image serving). Streaming images out of a machine can completely thrash the cache without using many cores. When your net latency is based on the slowest percentile to come back, one badly co-scheduled process can ruin your entire cluster capacity.
- corysama 11y agoThis comes up within a single process as well. Algo A might be faster than Algo B. But, if it gets that speed by completely filling your L3 cache with short-term data, it might make your process slower as a whole.
- packetslave 11y agoIntel finally added cache partitioning in Broadwell, which lets you restrict parts of the cache to certain cores. Combining that with pinning tasks to cores can help with cache thrashing.
- bogomipz 11y agoIs that enabled by default? Turned on in the BIOS or early boot?
- bogomipz 11y agoCan you explain what you mean by "tend to leave extra space". Do you mean they don't use all available cache lines in L1? Instruction cache, data cache?