16 ms·
I once wrote a command called trickle-rm, which was designed to be an i/o constrained rm, this the exact opposite of this article. I needed trickle-rm because a
by jeffrallen 4y ago
I once wrote a command called trickle-rm, which was designed to be an i/o constrained rm, this the exact opposite of this article. I needed trickle-rm because after extensive analysis, I'd found that the i/o load of a diagnostic data cleanup job was interfering with the very tight latency requirements of the main app on the server. My first effort was "nice rm -rf $oldLogs". When I'm feeling a bit evil I will ask an interview candidate why that didn't work.
Always something interesting to learn at the margins
- mherdeg 4y agoWhy didn't it work? Was try 2 ionice?
- b112 4y agoNot the parent, but ionice only works with certain elevators. And elevator choice mattered a lot on spinning disks.
- jeffrallen 4y agoNice reduces your scheduling priority for access to the CPU, which means it might take your niced rm longer to do the processing to submit a bunch of i/o to the kernel. But once those i/o ops are in the kernel, they are competing directly on an equal playing field with the latency critical i/o ops of the real app, causing the same degradation of latency to the end users. ionice was not available on my platform, thus the invention of trickle-rm. Now the second interview question: how did trickle-rm work? How can you simply "reflect" the back pressure of the "x rm's per second" constraint back onto the "walk the directory tree to find the rm's to do" so that the tree walk generates a trickle of i/o operations?
- aeonik 4y agoRun rm in a bash loop for each file and sync after each iteration? I'm surprised that rm's io would cause that much of an issue. I thought it only removed entries from the partition table.