4 ms·
Ran into similar pathological behavior on servers with huge amounts of ram (>1 TiB). Negative dentries builds up until the kernel decides it is time to reclaim
by alyandon 3y ago
Ran into similar pathological behavior on servers with huge amounts of ram (>1 TiB). Negative dentries builds up until the kernel decides it is time to reclaim and locks up the server for >60 seconds scanning pages. :-/
- tremon 3y agoYes, indeed. The more ram available, the more pages are available as buffer cache, the more pronounced the system stalls are. Though in my example, the determining factor is available_memory/disk_write_troughput, while in your example it is available_memory/single_core_throughput (assuming the page scan is a single-thread process).
- mkhnews 3y agoDoes disabling transparent huge pages help any here ?
- tremon 3y agoHave not tried it, but I would not expect any difference. FAFAIK, transparent huge pages are only about organizing free/unused/allocated memory; in both mine and the parent's example, the issue isn't about memory management per se but about synchronous operations on data taking too long (I/O write in my case, cache cleanup for the GP).
- anonacct37 3y agoI also ran into spooky negative dentry issues. On a fleet of seemingly identical physical machines one machine would constantly check for files that didn't exist (iirc it was somehow part of nss and a quick Google search seems to confirm https://lwn.net/Articles/894098/ https://lwn.net/Articles/894098/ ). I'm okayish at systems level debugging but I never figured that one out. It caused the kernel to use alot of memory. Arguable whether or not it impacted performance since it was a cache.