3 ms·

From my experience, they break every time if you are making changes with tools that replace the file rather than change it in-place, like most text editors.

by remram 4y ago

amelius 4y ago
In that case, you might want to monitor the directory too, not just the file. I believe this is even how it is supposed to work.
- remram 4y ago
  Yeah, and then filter out the events in the directory to find out when the correct file changed, etc. That is exactly what I would want the tool to abstract for me.
Brian_K_White 4y ago
At least with inotify, ie, incron, this is just a matter of setting a less stupid watch. You can ask the kernel to watch for several different specific kinds of fs events, either at the file or directory level. Detecting when a file is either created, or closed from writing, is no problem at all. Ie, handles the case of writing a file normally, editing a file, and renaming a file into exitence without having opened it for writing (that happened while it had some other name you weren't watching). This would handle unlinking and recreating the same as editing or uploading. You can watch specifically for the close event only, and even more specifically for close-from-write-mode to detect updates but ignore until the update is done. And at the same time you can also watch for creation, which handles creating a file under some other name and then renaming it into existence under the name you care about without the name you care about ever having a close-from-write event. That would fire off your script when a filename you were watching was unlinked and re-created instead of edited in-place, and then it's up to your script to handle the case where it's a "new" file that is really just replacing an old file. I don't even see why that should be any kind of special problem. Are some tools doing something like maintaining ooen file handles or something for every watched file??? That would break by unlink-create, but that would be insane. Perhaps these wrappers that try to simplify things are as usual just a way to break the thing they are trying to simplify. Just misguided crap. If the feature is baked into an ide or media server and failing to handle that case, well they don't have to fail to handle that case. The subsystem totally supplies the functionality. For something random adhoc where you're writing your own incrontab and handler script, it's no problem at all.
diffxx 4y ago
Yeah, this is difficult, though not impossible, to get right with the inotify/kqueue api. Windows and mac do provide apis for recursive directory monitoring but a cross platform tool will have to solve this problem. At a high level, the way to do it is to create an in memory representation of the file system that caches a watch handle for every file. When a deletion of a file is detected, you must create a watch on the parent directory, if there wasn't one already. Then you should be able to detect the ensuing create. To make this more concrete, the problem that a file watcher needs to solve is the problem of keeping its in memory representation of the file system consistent with the actual file system. Watch events are just a useful side effect of this process. The other fun part is that there is often a lag between the deletion and the create in the text editor case so it is necessary to defer triggering events when a deletion is detected and wait a little while to see if there is a corresponding creation. Otherwise you may rerun your command that depends on the file before it exists. It is possible to get like a 99+% solution to this problem without polling but it is a lot harder than what these simple tools, including entr do. The upshot is that file monitoring should be looked at as a lowest common denominator solution. A better solution is to build automatic command running into the text editor itself.

From my experience, they break every time if you are making changes with tools that replace the file rather than change it in-place, like most text editors.

by remram 4y ago

From my experience, they break every time if you are making changes with tools that replace the file rather than change it in-place, like most text editors.

amelius 4y ago

In that case, you might want to monitor the directory too, not just the file. I believe this is even how it is supposed to work.

remram 4y ago
Yeah, and then filter out the events in the directory to find out when the correct file changed, etc. That is exactly what I would want the tool to abstract for me.

Brian_K_White 4y ago

At least with inotify, ie, incron, this is just a matter of setting a less stupid watch. You can ask the kernel to watch for several different specific kinds of fs events, either at the file or directory level. Detecting when a file is either created, or closed from writing, is no problem at all. Ie, handles the case of writing a file normally, editing a file, and renaming a file into exitence without having opened it for writing (that happened while it had some other name you weren't watching). This would handle unlinking and recreating the same as editing or uploading. You can watch specifically for the close event only, and even more specifically for close-from-write-mode to detect updates but ignore until the update is done. And at the same time you can also watch for creation, which handles creating a file under some other name and then renaming it into existence under the name you care about without the name you care about ever having a close-from-write event. That would fire off your script when a filename you were watching was unlinked and re-created instead of edited in-place, and then it's up to your script to handle the case where it's a "new" file that is really just replacing an old file. I don't even see why that should be any kind of special problem. Are some tools doing something like maintaining ooen file handles or something for every watched file??? That would break by unlink-create, but that would be insane. Perhaps these wrappers that try to simplify things are as usual just a way to break the thing they are trying to simplify. Just misguided crap. If the feature is baked into an ide or media server and failing to handle that case, well they don't have to fail to handle that case. The subsystem totally supplies the functionality. For something random adhoc where you're writing your own incrontab and handler script, it's no problem at all.

diffxx 4y ago

Yeah, this is difficult, though not impossible, to get right with the inotify/kqueue api. Windows and mac do provide apis for recursive directory monitoring but a cross platform tool will have to solve this problem. At a high level, the way to do it is to create an in memory representation of the file system that caches a watch handle for every file. When a deletion of a file is detected, you must create a watch on the parent directory, if there wasn't one already. Then you should be able to detect the ensuing create. To make this more concrete, the problem that a file watcher needs to solve is the problem of keeping its in memory representation of the file system consistent with the actual file system. Watch events are just a useful side effect of this process. The other fun part is that there is often a lag between the deletion and the create in the text editor case so it is necessary to defer triggering events when a deletion is detected and wait a little while to see if there is a corresponding creation. Otherwise you may rerun your command that depends on the file before it exists. It is possible to get like a 99+% solution to this problem without polling but it is a lot harder than what these simple tools, including entr do. The upshot is that file monitoring should be looked at as a lowest common denominator solution. A better solution is to build automatic command running into the text editor itself.