4 ms·

While the idea that there can be a lot of small and composable tools sounds good in theory, in practice the implementation is very poor. To use tools in a scrip

While the idea that there can be a lot of small and composable tools sounds good in theory, in practice the implementation is very poor. To use tools in a script, they should have standartized machine-friendly input and output formats, but instead they have arbitrary and difficult to parse formats. It means that they are not intended for use in scripts. - spaces or newlines in file names are often not handled properly. Some linux software based on shell scripts requires that a user install it to a path that has no spaces. - the output format is often arbitrary and not machine-friendly. For example, `ls -l` output is difficult to parse properly. Some programs might output a file size with different suffixes, like 10k or 10M, that you have to parse. - output format often depends on locale settings and scripts can break if used with different locale So the programs that rely on line-oriented inputs/outputs like `grep`, `wc`, `ls` (without -l), `sort` work relatively good, but other programs that produce more complex output, are unusable for scripting. Scripts often use ugly hacks like parsing output with `sed` or, even worse, `awk`, a 40 years old cryptic legacy tool. Instead they could use something like JSONpath, for example `parse $.*.size -- ls -l` to get sizes of all files (although I don't think that using JSON is a good idea; it's hard to read by human. I think something indentation-based like YAML would be better; however, YAML itself is overengineered and I recommend against it). Also the shells are an awful choice for scripting. For example, there is no syntax to write a glob pattern to select all files including dot-files, but excluding pseudo-files like . and .. To conclude, "Unix philosophy" of composing utilities using shell scripts is often advertised as a superior and innovative approach but in practice it is a bunch of inconvenient to use programs glued together with unrealiable buggy scripts written in outdated language. It is easier to use Python and ignore all this "philosophy". By the way, `/proc` filesystem also is not scripting-friendly and not standartized. So it is usable only as a debugging tool for humans.

unsafecast 4y ago
I've always thought the same, with one exception: You can (and should) use lightweight formats that are hard to read by a human and have the shell format it to look nice. You can even format the same datum differently depending on context. I don't like the idea of using the same format for machine and human consumption, they have different requirements.
tpoacher 4y ago
those examples seem a bit contrived to me. i could say the same about python if I mandated no imports or sth
Fnoord 4y ago
Because ls -l is long format, meant for human readable instead of machine readable. It might be ls -s is suffice. Instead of parsing ls you may be better off with find or the quick Rust equivalent fd. Instead of awk you can use sed or cut + tr + grep (but I find awk easier to use awk). Nice tools to finish it off are sort and uniq. Though I suppose nowadays everything becomes json.

unsafecast 4y ago

I've always thought the same, with one exception: You can (and should) use lightweight formats that are hard to read by a human and have the shell format it to look nice. You can even format the same datum differently depending on context. I don't like the idea of using the same format for machine and human consumption, they have different requirements.

tpoacher 4y ago

those examples seem a bit contrived to me. i could say the same about python if I mandated no imports or sth

Fnoord 4y ago

Because ls -l is long format, meant for human readable instead of machine readable. It might be ls -s is suffice. Instead of parsing ls you may be better off with find or the quick Rust equivalent fd. Instead of awk you can use sed or cut + tr + grep (but I find awk easier to use awk). Nice tools to finish it off are sort and uniq. Though I suppose nowadays everything becomes json.