4 ms·
What I need is a du that caches the results somewhere and then does not rescan the 90% of dirs that have not changed when I run it again a month later...
by shellfishgene 2y ago
What I need is a du that caches the results somewhere and then does not rescan the 90% of dirs that have not changed when I run it again a month later...
- kstrauser 2y agoAnd it would know they did not change without scanning them because how?
- svpg 2y agoIt could hash the contents of a dir. Along the lines of git
- Galanwe 2y agoExcept hashing requires... reading. There is not much to be done here. Directories entries are just names, no guarantees that the files were not modified or replaced. The best you could do is something similar to the strategies of rsync, rely on metadata (modified date, etc) and cross fingers nobody did `cp -a`.
- shellfishgene 2y agoI would be fine with the latter, the program could display a warning like "Results may be inaccurate, full scan required" or something. I guess I'm just annoyed that for Windows/NTFS really fast programs are available but not for Linux filesystems.
- legends2k 2y agoAnd to hash something needs reading all of its data. I think deducing the file size would actually be faster in some file systems and never slower with any.
- mort96 2y agoFaster in all file systems I'd guess, stat is fast, opening the file and reading its contents and updating a checksum is slow, and gets slower the larger the file is.
- deleted 2y ago[deleted]
- shellfishgene 2y agoMaybe it could run in the background and use inotify to just update the database all the time, or at least keep track of what needs rescanning?
- shellfishgene 2y agoThinking about this some more, does this system not already exist for the disk quota calculation in the kernel? How does that work? Would it be possible for a tool to scan the disk once, and then get information about file modifications from the system that's used to update quota info?