3 ms·
I've ran into this issue myself and it's not filesystem watchers missing the point, it's actually something not supported in a number of operating systems. Thus
by hnlmorg 6y ago
I've ran into this issue myself and it's not filesystem watchers missing the point, it's actually something not supported in a number of operating systems. Thus you then either have to resort to nasty user-space hacks (which will be comparatively slow and resource hungry) or accept that you can't watch nested directories.
A more technical description is mentioned here: https://github.com/fsnotify/fsnotify/issues/18 https://github.com/fsnotify/fsnotify/issues/18
- inshadows 6y agoinotifywait can do this just fine. Here's an emulation of `entr` with an `include pattern`: inotifywait -rq -m -e close_write --format %f . | grep '\.rules$' Sure, there may races due to limitations of underlying OS interfaces, but no such races really matter when you edit with mammal speed, so it's perfectly sufficient, and so it is perfectly possible to make good file system watcher, without any user-space power-hungry hacks.
- hnlmorg 6y agoinotifywait is a user-space program that borrows it's name from inotify (because it uses inotify) and when using the -r (recursive) flag it sets up multiple inotify watches to work around the very problem I just described. In fact it's man page comes with a big fat warning about using -r: > Warning: If you use this option while watching the root directory of a large tree, it may take quite a while until all inotify watches are established, and events will not be received in this time. Also, since one inotify watch will be established per subdirectory, it is possible that the maximum amount of inotify watches per user will be reached. The default maximum is 8192; it can be increased by writing to /proc/sys/fs/inotify/max_user_watches. (ref: https://linux.die.net/man/1/inotifywait https://linux.die.net/man/1/inotifywait) Really what would be nice would be for Linux to support this natively so we don't have to resort to dirty hacks.
- inshadows 6y ago> inotifywait is a user-space program that borrows it's name from inotify (because it uses inotify) and when using the -r (recursive) flag it sets up multiple inotify watches to work around the very problem I just described. What problem? What you quoted is totally irrelevant for my workflow. On my crappy laptop and all projects in my work directory, the watches are established almost instantly, without any spike in CPU. There are 4677 directories in Linux kernel and you can just put a knob change in /etc/sysctl.d/ if your code base is bigger than kernel. You're just inventing OS limitations to justify limitations of lazy user-space programming. EDIT: Setting up watches on kernel tree including directories in .git: ~/src/linux % inotifywait -r -m -e close_write --format %f . |& ts Jul 01 16:03:10 Setting up watches. Beware: since -r was given, this may take a while! Jul 01 16:03:10 Watches established. Less than 1 second. Big deal!
- hnlmorg 6y agoYour workflow might be ok, others might not. But blaming developers for being "lazy user-space [programmers]" when Linux lacks a feature that Windows and macOS support, and which could cause complications when worked around, is a really unproductive way to hold a discussion on HN.
- inshadows 6y agoYou still didn't specify which feature Linux lacks that Windows and macOS supports. I have shown you that there's no need for any workaround, that the OS feature works as intended, but it seems you're keen on believing FUD scraped from web pages of lazy user-space programmers instead of considering my arguments and evidence. FYI: What OS interface do you think that `entr` uses on Linux? That's right, inotify.
- heleninboodler 6y agoYou're being quite aggressive about your disagreement here. I don't think it's "FUD" to rightfully point out that using inotify on a large directory tree takes extra userspace code that is explicitly called out in the inotifywatch docs as performing poorly enough to worry about race conditions. Not to mention that it requires a limits tweak. This isn't "lazy," it's wishing for the mechanism to be more capable. Would you have this same attitude if the default limit were 1024? 512?
- tdumitrescu 6y agoI agree with your frustrations around this limitation and think file watchers should do their best to support arbitrary file addition/removal in directories. That being said, the races do matter when you edit with "mammal speed" but use source control to merge/rebase commits that affect your watched directories. Especially in a team setting when multiple coworkers are working on the same directories, git can be blasting rapidly through a lot of file moves, deletions etc so it's not really an edge case.
- inshadows 6y agoGit is a good point. However, the only time I desire commands triggered by changes in source tree is during actual code writing to instantly test my changes. On the other hand stopping it during Git operations could harm my workflow. This could be easily solved by simple 1s debounce, with this hypothetical debounce filter inotifywait -rq -m -e close_write --format %f --exclude .git | grep '\.rules$' | debounce 1 | xargs ...