7 ms·

Entr(1) – Run tests whenever files change

108 points by jsnathan 10y ago

evinism 10y ago
this is one of those problems I've needed to solve for forever, but which i've never explicitly looked up a clean solution for. I can immediately think of 3 instances in the past week where this would have been perfect.
- jmtulloss 10y ago
  Watchman from facebook is another great, single-purpose solution. https://facebook.github.io/watchman/ https://facebook.github.io/watchman/
  - seanparsons 10y ago
    We switched from watchman to entr on the project I'm currently working on because watchman was troublesome to configure and IIRC it completely detaches from the shell as well.
strictfp 10y ago
What a coincidence! I tested it yesterday. Worked like a charm. Thanks! One note: the browser reloading script was not installed on debian, so I downloaded it. It was not clear from the documentation if that was the intention.
contingencies 10y ago
Another approach would be to use the Linux kernel's fantastic inotify() feature, which does not require running tools designed for other purposes... https://linux.die.net/man/7/inotify https://linux.die.net/man/7/inotify https://en.wikipedia.org/wiki/Inotify https://en.wikipedia.org/wiki/Inotify There is also a less well utilized fsnotify feature for filesystem-wide notifications. Here is a cross-platform go implementation which also supports OSX: https://github.com/fsnotify/fsnotify https://github.com/fsnotify/fsnotify
- detaro 10y ago
  The program discussed in the submission uses inotify
  - contingencies 10y ago
    Err, not as far as I can see.
    - detaro 10y ago
      It has a header file that on Linux emulates kqueue using inotify: https://bitbucket.org/eradman/entr/src/1dc74ab543e2bced2994db4e2d1a250620e992a2/missing/kqueue_inotify.c?at=default https://bitbucket.org/eradman/entr/src/1dc74ab543e2bced2994d...
      - contingencies 10y ago
        Aha. Be that as it may, using inotify directly from the shell is still a fair approach which is worth putting out there.
- jsnathan 10y ago
  Hey! OP here. I submitted this exactly because I kept running into problems with inotify-tools, and other naive inotify wrappers, where tests would execute multiple times in quick succession, or stop executing on file changes, or other odd behavior. Entr on the other hand works like a charm. You can see on the website that it details a number of edge cases with inotify that it works around. I'm not even sure it's an exhaustive list.
- lloeki 10y ago
  I myself quite enjoy using the very nice fswatch[0]. https://github.com/emcrisostomo/fswatch https://github.com/emcrisostomo/fswatch
  - mmjaa 10y ago
    Indeed, I live by this one-liner which makes life so smooth: $ fswatch -0 -event ./main.lua | xargs -0 -n 1 -I {} lua ./main.lua
pkghost 10y ago
Cross-platform alternative: http://emcrisostomo.github.io/fswatch/ http://emcrisostomo.github.io/fswatch/
whizzkid 10y ago
Ruby developers have been using Guard for a long time now. https://github.com/guard/guard https://github.com/guard/guard
- theodorton 10y ago
  Am I the only one having issues with guard? It's performance has been abysmal for the most recent projects I've used it with.
  - jlebrech 10y ago
    I turn off autorun and only have it test what's changed. And it also doesn't run coverage checks, it's only for running the test i'm currently working on. But yes, the ruby infrastructure is pretty bloated i recon.
bulutsuzku 10y ago
Files? Tests were invented for OOP, no need to have files
- Sidnicious 10y ago
  This is a troll/unhelpful comment… but I genuinely like the reminder.
andy_ppp 10y ago
Looks interesting; anyone know how well this works inside docker? For me watching files can often cause huge CPU usage.
cgag 10y ago
I use entr all the time. I usually do something like piping go files to entry to run a script to build and run my app or tests. Much simpler than most tools I've seen like it.
dorianm 10y ago
There is also Guard https://github.com/guard/guard https://github.com/guard/guard
scoopr 10y ago
Many years ago I did a little utility I called autobuild[0] to do similar things. It has osx and windows support, but never got around doing linux support. I don't think inotify has an easy way of doing recursive filesystem notifications, having to manage the hierarchy and their notifiers, so it lapsed when I didn't need it at the time. It felt like this kind of tool would/should be one of those standard command line tools always available, or the at least of those standard things everyone knows to install, like silversearcher/ripgrep etc. [0] https://github.com/scoopr/autobuild https://github.com/scoopr/autobuild
floatboth 10y ago
For node: https://github.com/kimmobrunfeldt/chokidar-cli https://github.com/kimmobrunfeldt/chokidar-cli
leni536 10y ago
ls -d * | entr make Just a nit: this seems to be imperfect for filenames containing new lines (I know, I know). An xargs like -0, --null option could make it more robust. Then the equivalent of the above command would be something like: find . -maxdepth 1 -name '[!.]*' -print0 | entr -0 make Edit: Seems like -print0 is not POSIX compliant. Is there any safe way to pass a list of paths using the shell in a fully POSIX compliant way?
- Arkanosis 10y ago
  Imperfect as well because any filename starting with a dash will be interpreted as an option to ls. Nitpicking… or maybe not: if you take the habit of running `cmd * ` whatever the command, I guess you might end up writing dangerous things like `rm * ` in a script which will fatally end up running in a directory containing a file named “-R”.
- zimbatm 10y ago
  On the other hand, since the target users are developers, they are in a good position to fix these newline-containing filenames. Even if they are POSIX compliant they shouldn't be encouraged :)
- Hello71 10y ago
  printf '%s\0' * | xxd
oever 10y ago
A similar tool written in Rust is watchexec. https://github.com/mattgreen/watchexec https://github.com/mattgreen/watchexec
- jsnathan 10y ago
  This looks nice too! Very intuitive interface. Thanks!
michaelmior 10y ago
> It is not uncommon for modern web frameworks to continuously hunt for file system changes and auto-reloads when run in single threaded or standalone mode. This functionality is superfluous if the application can respond to signals. This is true if all that happens is a page reload, butamy frameworks are able to patch code or styles into the running page without a reload. This requires more than a signal handler.
- michaelmior 10y ago
  s/butamy/but many/
- simonlc 10y ago
  Yeah this is a huge difference if what you're working on has state, being able to patch styles or modules makes development much smoother.
amelius 10y ago
Next step: run tests whenever you enter a keystroke in your editor?
- jsnathan 10y ago
  You're probably joking, but if you use a real-time linter, maybe you could run tests whenever the file you're editing has no errors. You might even try to limit which tests to run based on the test coverage (from previous runs) of the changed functions or statements. And then there's also ideas like LightTable's inline evaluation [1]. [1]: https://www.youtube.com/watch?v=7YIaHyTdjTY https://www.youtube.com/watch?v=7YIaHyTdjTY
tony 10y ago
entr is is what I eventually settled on after years of searching the perfect utility to build/run tests after file changes. After using gulp, grunt, watching_testrunner (which I eventually inherited maintainership of), watchman, watchdog (https://github.com/gorakhargosh/watchdog https://github.com/gorakhargosh/watchdog), sniffer (https://github.com/jeffh/sniffer https://github.com/jeffh/sniffer) entr is cross-platform on OS X, linux and BSD. watching_testrunner has no BSD support watchman is way too big and not domain specific enough to my needs sniffer worked quite well but it required having a scent.py file everywhere entr keeps it all in a neat, unix like package you can pipe files to. I keep some example usage in my book, The Tao of tmux at https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-file-watching https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-file-w.... In this section I demonstrate my workflow with entr(1) in a Makefile. The code I use in the example should work across OS X / BSD / Linux (note the utilities like find(1) may behave a bit differently across unix-like systems). I use the Makefile w/ entr(1) in development on my projects like tmuxp at https://github.com/tony/tmuxp/blob/master/Makefile https://github.com/tony/tmuxp/blob/master/Makefile. tmuxp is BSD-licensed so you're free to work off that if you'd like to try it on your own project.
junkblocker 10y ago
A long time entr user, I recently switch to modd for flexible almost makefile like configurability, and watching/ignoring of multiple patterns to name a few convenient feature. https://github.com/cortesi/modd https://github.com/cortesi/modd
awinter-py 10y ago
awesome. What would be really great is a fuse DAG. Some paths should run programs on write, others on read. The filesystem is already the lock provider for a lot of programs; we should be using it as the event processor. This is inherently polyglot -- every programming language in common use has FS utils.
nameless912 10y ago
I wrote a dead-simple version of this (that uses polling and doesn't give you filters for files) in python and called it `jonsnow` and threw it on pypi. I understand this is an insanely useful utility, but it's interesting that everyone seems to spin their own :)

108 points by jsnathan 10y ago

this is one of those problems I've needed to solve for forever, but which i've never explicitly looked up a clean solution for. I can immediately think of 3 instances in the past week where this would have been perfect.

jmtulloss 10y ago
Watchman from facebook is another great, single-purpose solution. https://facebook.github.io/watchman/ https://facebook.github.io/watchman/
- seanparsons 10y ago
  We switched from watchman to entr on the project I'm currently working on because watchman was troublesome to configure and IIRC it completely detaches from the shell as well.

strictfp 10y ago

What a coincidence! I tested it yesterday. Worked like a charm. Thanks! One note: the browser reloading script was not installed on debian, so I downloaded it. It was not clear from the documentation if that was the intention.

contingencies 10y ago

Another approach would be to use the Linux kernel's fantastic inotify() feature, which does not require running tools designed for other purposes... https://linux.die.net/man/7/inotify https://linux.die.net/man/7/inotify https://en.wikipedia.org/wiki/Inotify https://en.wikipedia.org/wiki/Inotify There is also a less well utilized fsnotify feature for filesystem-wide notifications. Here is a cross-platform go implementation which also supports OSX: https://github.com/fsnotify/fsnotify https://github.com/fsnotify/fsnotify

detaro 10y ago
The program discussed in the submission uses inotify
- contingencies 10y ago
  Err, not as far as I can see.
  - detaro 10y ago
    It has a header file that on Linux emulates kqueue using inotify: https://bitbucket.org/eradman/entr/src/1dc74ab543e2bced2994db4e2d1a250620e992a2/missing/kqueue_inotify.c?at=default https://bitbucket.org/eradman/entr/src/1dc74ab543e2bced2994d...
    - contingencies 10y ago
      Aha. Be that as it may, using inotify directly from the shell is still a fair approach which is worth putting out there.
jsnathan 10y ago
Hey! OP here. I submitted this exactly because I kept running into problems with inotify-tools, and other naive inotify wrappers, where tests would execute multiple times in quick succession, or stop executing on file changes, or other odd behavior. Entr on the other hand works like a charm. You can see on the website that it details a number of edge cases with inotify that it works around. I'm not even sure it's an exhaustive list.
lloeki 10y ago
I myself quite enjoy using the very nice fswatch[0]. https://github.com/emcrisostomo/fswatch https://github.com/emcrisostomo/fswatch
- mmjaa 10y ago
  Indeed, I live by this one-liner which makes life so smooth: $ fswatch -0 -event ./main.lua | xargs -0 -n 1 -I {} lua ./main.lua

pkghost 10y ago

Cross-platform alternative: http://emcrisostomo.github.io/fswatch/ http://emcrisostomo.github.io/fswatch/

whizzkid 10y ago

Ruby developers have been using Guard for a long time now. https://github.com/guard/guard https://github.com/guard/guard

theodorton 10y ago
Am I the only one having issues with guard? It's performance has been abysmal for the most recent projects I've used it with.
- jlebrech 10y ago
  I turn off autorun and only have it test what's changed. And it also doesn't run coverage checks, it's only for running the test i'm currently working on. But yes, the ruby infrastructure is pretty bloated i recon.

bulutsuzku 10y ago

Files? Tests were invented for OOP, no need to have files

Sidnicious 10y ago
This is a troll/unhelpful comment… but I genuinely like the reminder.

andy_ppp 10y ago

Looks interesting; anyone know how well this works inside docker? For me watching files can often cause huge CPU usage.

cgag 10y ago

I use entr all the time. I usually do something like piping go files to entry to run a script to build and run my app or tests. Much simpler than most tools I've seen like it.

dorianm 10y ago

There is also Guard https://github.com/guard/guard https://github.com/guard/guard

scoopr 10y ago

Many years ago I did a little utility I called autobuild[0] to do similar things. It has osx and windows support, but never got around doing linux support. I don't think inotify has an easy way of doing recursive filesystem notifications, having to manage the hierarchy and their notifiers, so it lapsed when I didn't need it at the time. It felt like this kind of tool would/should be one of those standard command line tools always available, or the at least of those standard things everyone knows to install, like silversearcher/ripgrep etc. [0] https://github.com/scoopr/autobuild https://github.com/scoopr/autobuild

floatboth 10y ago

For node: https://github.com/kimmobrunfeldt/chokidar-cli https://github.com/kimmobrunfeldt/chokidar-cli

leni536 10y ago

ls -d * | entr make Just a nit: this seems to be imperfect for filenames containing new lines (I know, I know). An xargs like -0, --null option could make it more robust. Then the equivalent of the above command would be something like: find . -maxdepth 1 -name '[!.]*' -print0 | entr -0 make Edit: Seems like -print0 is not POSIX compliant. Is there any safe way to pass a list of paths using the shell in a fully POSIX compliant way?

Arkanosis 10y ago
Imperfect as well because any filename starting with a dash will be interpreted as an option to ls. Nitpicking… or maybe not: if you take the habit of running `cmd * ` whatever the command, I guess you might end up writing dangerous things like `rm * ` in a script which will fatally end up running in a directory containing a file named “-R”.
zimbatm 10y ago
On the other hand, since the target users are developers, they are in a good position to fix these newline-containing filenames. Even if they are POSIX compliant they shouldn't be encouraged :)
Hello71 10y ago
printf '%s\0' * | xxd

oever 10y ago

A similar tool written in Rust is watchexec. https://github.com/mattgreen/watchexec https://github.com/mattgreen/watchexec

jsnathan 10y ago
This looks nice too! Very intuitive interface. Thanks!

michaelmior 10y ago

> It is not uncommon for modern web frameworks to continuously hunt for file system changes and auto-reloads when run in single threaded or standalone mode. This functionality is superfluous if the application can respond to signals. This is true if all that happens is a page reload, butamy frameworks are able to patch code or styles into the running page without a reload. This requires more than a signal handler.

michaelmior 10y ago
s/butamy/but many/
simonlc 10y ago
Yeah this is a huge difference if what you're working on has state, being able to patch styles or modules makes development much smoother.

amelius 10y ago

Next step: run tests whenever you enter a keystroke in your editor?

jsnathan 10y ago
You're probably joking, but if you use a real-time linter, maybe you could run tests whenever the file you're editing has no errors. You might even try to limit which tests to run based on the test coverage (from previous runs) of the changed functions or statements. And then there's also ideas like LightTable's inline evaluation [1]. [1]: https://www.youtube.com/watch?v=7YIaHyTdjTY https://www.youtube.com/watch?v=7YIaHyTdjTY

tony 10y ago

entr is is what I eventually settled on after years of searching the perfect utility to build/run tests after file changes. After using gulp, grunt, watching_testrunner (which I eventually inherited maintainership of), watchman, watchdog (https://github.com/gorakhargosh/watchdog https://github.com/gorakhargosh/watchdog), sniffer (https://github.com/jeffh/sniffer https://github.com/jeffh/sniffer) entr is cross-platform on OS X, linux and BSD. watching_testrunner has no BSD support watchman is way too big and not domain specific enough to my needs sniffer worked quite well but it required having a scent.py file everywhere entr keeps it all in a neat, unix like package you can pipe files to. I keep some example usage in my book, The Tao of tmux at https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-file-watching https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-file-w.... In this section I demonstrate my workflow with entr(1) in a Makefile. The code I use in the example should work across OS X / BSD / Linux (note the utilities like find(1) may behave a bit differently across unix-like systems). I use the Makefile w/ entr(1) in development on my projects like tmuxp at https://github.com/tony/tmuxp/blob/master/Makefile https://github.com/tony/tmuxp/blob/master/Makefile. tmuxp is BSD-licensed so you're free to work off that if you'd like to try it on your own project.

junkblocker 10y ago

A long time entr user, I recently switch to modd for flexible almost makefile like configurability, and watching/ignoring of multiple patterns to name a few convenient feature. https://github.com/cortesi/modd https://github.com/cortesi/modd

awinter-py 10y ago

awesome. What would be really great is a fuse DAG. Some paths should run programs on write, others on read. The filesystem is already the lock provider for a lot of programs; we should be using it as the event processor. This is inherently polyglot -- every programming language in common use has FS utils.

nameless912 10y ago

I wrote a dead-simple version of this (that uses polling and doesn't give you filters for files) in python and called it `jonsnow` and threw it on pypi. I understand this is an insanely useful utility, but it's interesting that everyone seems to spin their own :)