5 ms·

That's the usual response I get when I bring this issue up. "file watching is actually very complicated" or "if you avoided deps, you'd just reimplement million

That's the usual response I get when I bring this issue up. "file watching is actually very complicated" or "if you avoided deps, you'd just reimplement millions of loc yourself. Forgive me if I'm making a very bold claim, but I think cross-platform file watching should not require this much code. It's 32x larger than the Linux memory management subsystem.

selfmodruntime 2y ago
Eh. The standard library is also a gigantic dependency written entirely by volunteers.
- dist1ll 2y ago
  I don't have a problem with dependencies in principle. There's a good reason for standard libraries to contain a decent amount of code. It is a vector for supply chain attacks, but I also have a lot of trust in the Rust maintainers. The Rust standard library is exceptionally well-written from what I've seen, so I'm not too worried about it. FWIW I checked out the nightly toolchain, and it looks like the stdlib is less than 400k SLoC. So literally 10x smaller.
- DanielHB 2y ago
  I am not a Rust expert but the thing with the standard libraries is that it only has peer dependencies with itself and they are all synced to the same version. Meaning if you only use the std lib you: 1) Will never include two different versions of the same peer dependency because of incompatible version requirements. 2) Will usually not have two dependencies relying on two different peer-dependencies that do the same thing. This can still happen for deprecated std lib features, but tends to be a much lesser issue. These two issues are usually the ones that cause dependency size explosion in projects.
  - deleted 2y ago
    [deleted]
DanielHB 2y ago
I think the issue of file-watching is that the libs usually support multiple implementations (with different tradeoffs and with multiple fallbacks) for file-watching with a lot of them being platform specific.
joatmon-snoo 2y ago
Good file watching that provides flexible primitives absolutely requires: - ok, a single ext4 file inode changes, and its filename matches my hardcoded string - oh, you don’t want to match against just changes to “package.json” but you want to match against a regex? voila, now you need a regex engine - what about handling a directory rename? should that trigger matches on all files in the renamed directory? - should the file watcher be triggered once per file, or just every 5ms? turns out this depends on your use case - how do symlinks fit into this story? - let’s say i want to handle once every 5ms- how do i actually wait for 5ms? do i yield the thread? do i allow other async contexts to execute while i’m waiting? how do those contexts know when to execute and when to yield back to me? now you have an async runtime with timers - how does buffering work? are there limits on how many file change events can be buffered? do i dynamically allocate more memory as more file changes get buffered? now you need a vector/arraylist implementation And this is before you look at what this looks like on different platforms, or if you want polling fallbacks. Can you do it with less dependencies? Probably, if you start making hard tradeoffs and adding even more complexity about what features you activate - but that only adds lines of code, it doesn’t remove them. What you describe is ideologically nice, but in practice it’s over-optimizing for a goal that most people don’t really care about.
- varjag 2y ago
  Are you 100% sure all these cases are handled by cargo-watch?
  - cogman10 2y ago
    Yup. Every user facing case mentioned has a corresponding flag. The non-user facing stuff, like being cross platform, is common sense. https://crates.io/crates/cargo-watch/8.5.2 https://crates.io/crates/cargo-watch/8.5.2
- lostmsu 2y ago
  Regex one stands out as a negative example here. Why does it have to be built-in instead of exposing a str -> bool filter lambda?
SkiFire13 2y ago
It's not just file watching, that would be the watchexec crate, while cargo-watch properly integrate with cargo. Moreover cargo-watch also includes: - proper CLI support, with help messages, subcommands and so on - support for reading cargo's metadata - logging - support for dotenv files - proper shell escaping support - and it seems also support for colored terminal writing. Moreover both watchexec and cargo-watch end up depending on winapi, which includes binding for a lot of windows API, some which might be needed and some which not be. This could also be worse if the offial windows crate by Microsoft was used (or maybe it's already used due to some dependency, I haven't checked), since that's gigantic.

selfmodruntime 2y ago

Eh. The standard library is also a gigantic dependency written entirely by volunteers.

dist1ll 2y ago
I don't have a problem with dependencies in principle. There's a good reason for standard libraries to contain a decent amount of code. It is a vector for supply chain attacks, but I also have a lot of trust in the Rust maintainers. The Rust standard library is exceptionally well-written from what I've seen, so I'm not too worried about it. FWIW I checked out the nightly toolchain, and it looks like the stdlib is less than 400k SLoC. So literally 10x smaller.
DanielHB 2y ago
I am not a Rust expert but the thing with the standard libraries is that it only has peer dependencies with itself and they are all synced to the same version. Meaning if you only use the std lib you: 1) Will never include two different versions of the same peer dependency because of incompatible version requirements. 2) Will usually not have two dependencies relying on two different peer-dependencies that do the same thing. This can still happen for deprecated std lib features, but tends to be a much lesser issue. These two issues are usually the ones that cause dependency size explosion in projects.
- deleted 2y ago
  [deleted]

DanielHB 2y ago

I think the issue of file-watching is that the libs usually support multiple implementations (with different tradeoffs and with multiple fallbacks) for file-watching with a lot of them being platform specific.

joatmon-snoo 2y ago

Good file watching that provides flexible primitives absolutely requires: - ok, a single ext4 file inode changes, and its filename matches my hardcoded string - oh, you don’t want to match against just changes to “package.json” but you want to match against a regex? voila, now you need a regex engine - what about handling a directory rename? should that trigger matches on all files in the renamed directory? - should the file watcher be triggered once per file, or just every 5ms? turns out this depends on your use case - how do symlinks fit into this story? - let’s say i want to handle once every 5ms- how do i actually wait for 5ms? do i yield the thread? do i allow other async contexts to execute while i’m waiting? how do those contexts know when to execute and when to yield back to me? now you have an async runtime with timers - how does buffering work? are there limits on how many file change events can be buffered? do i dynamically allocate more memory as more file changes get buffered? now you need a vector/arraylist implementation And this is before you look at what this looks like on different platforms, or if you want polling fallbacks. Can you do it with less dependencies? Probably, if you start making hard tradeoffs and adding even more complexity about what features you activate - but that only adds lines of code, it doesn’t remove them. What you describe is ideologically nice, but in practice it’s over-optimizing for a goal that most people don’t really care about.

varjag 2y ago
Are you 100% sure all these cases are handled by cargo-watch?
- cogman10 2y ago
  Yup. Every user facing case mentioned has a corresponding flag. The non-user facing stuff, like being cross platform, is common sense. https://crates.io/crates/cargo-watch/8.5.2 https://crates.io/crates/cargo-watch/8.5.2
lostmsu 2y ago
Regex one stands out as a negative example here. Why does it have to be built-in instead of exposing a str -> bool filter lambda?

SkiFire13 2y ago

It's not just file watching, that would be the watchexec crate, while cargo-watch properly integrate with cargo. Moreover cargo-watch also includes: - proper CLI support, with help messages, subcommands and so on - support for reading cargo's metadata - logging - support for dotenv files - proper shell escaping support - and it seems also support for colored terminal writing. Moreover both watchexec and cargo-watch end up depending on winapi, which includes binding for a lot of windows API, some which might be needed and some which not be. This could also be worse if the offial windows crate by Microsoft was used (or maybe it's already used due to some dependency, I haven't checked), since that's gigantic.