3 ms·

> "I've never experienced warts with Unix tools such as `cat`, `cut`, `uniq`, etc." Literally the first time of using uniq and finding duplicates in the output

> "I've never experienced warts with Unix tools such as `cat`, `cut`, `uniq`, etc." Literally the first time of using uniq and finding duplicates in the output is a wart. Having to realise it's not doing "unique values" it's doing "drop consecutive duplicates" and the reason it's doing it is because 1970s computers didn't have enough memory to dedupe a whole dataset and could only do it streaming. cat being "catenate" intended to catenate multiple files together into one pipeline stream, but always being used to read single files instead of a pager. "cat words.txt | less" is a wart seen all over the place. cut being needed to separate fields because we couldn't all come together and agree to use ASCII Field Separator character to separate fields is arguably warty. Or the find command where you can find files by different sizes and you can specify +100G for finding files with sizes 100 Gibibytes or larger, or M for Mebibytes, K for Kibibytes, and b for bytes. No just kidding b is 512-byte blocks, it's nothing at all for bytes just +100. No just kidding again that's also 512-byte blocks. It's c for bytes. Because the first version of Unix happened to use 512-byte blocks on its filesystem[1]. And the c stands for characters so you have to know that it's all pre-Unicode and using 8-bit characters. [1] https://unix.stackexchange.com/questions/259208/purpose-of-find-commands-default-size-unit-512-bytes https://unix.stackexchange.com/questions/259208/purpose-of-f...

imiric 1y ago
I think you're misunderstanding what a "wart" is. None of those examples are warts. They're the way those programs work, which you can easily understand by reading their documentation. Your point about `cat` is an example of misuse, and `cut` is needed because of the deliberate design decision to make text streams the universal interface, which has far greater benefits than forcing any specific separator. A wart is not surprising or unintuitive behavior. It is a design quirk—a feature that sticks out from the program's overall design, usually meant to handle edge cases, or to preserve backwards compatibility. For example, some GNU programs like `grep` and `find` support a `POSIXLY_CORRECT` env var, which changes their interface and behavior. My point is that the complexity of the software and its interface is directly proportional to the likelihood of it developing "warts". `find` is a much more complex program than `cut` or `uniq`, therefore it has developed warts. Next time, please address the point of the comment you're replying to, instead of forcing some counterargument that happens to be wrong.
- jodrellblank 1y ago
  The context of the blog post is that "warts" are the way something was originally designed or built, which is now outdated and you would change it if you could, but you don't want to break backwards compatibility so you don't. > "For example, some GNU programs like `grep` and `find` support a `POSIXLY_CORRECT` env var, which changes their interface and behavior." By your own logic that's not a wart, that's the way the programs work and they are documented to work like that. Twisting yourself into knots to pretend Linux/Unix utilities are perfectly designed isn't good logic. There's no excuse "it was designed that way" or "it's documented to work like this" which makes something not warty. Something might have been a good design originally and become a bad design as the world changed around it. For example unserialised byte streams where you have to fight whether \n or 0 (null byte) are data or record separators, is a wart. How do we know? Because when people design newer tools they don't keep that, they design it away. Serialised JSON and jq, .NET objects in PowerShell, structured data in nushell[1] and oilshell YSH[2]. [1] https://www.nushell.sh/ https://www.nushell.sh/ [2] https://oils.pub/ysh.html https://oils.pub/ysh.html
  - imiric 1y ago
    Ugh, you're exhausting. Look, here's a program: print "Hello" Do you see any warts in it? No? Well, my point is that the likelihood of a program developing warts is partly related to its complexity. That's it. I mentioned Unix tools as an example of this, since they're by far the most reliable tools I use, particularly the simpler ones. I never said that they were "perfectly designed". Don't put words in my mouth. > By your own logic that's not a wart, that's the way the programs work and they are documented to work like that. A wart can be documented. I mentioned the case of `POSIXLY_CORRECT` since GNU tools predate the POSIX standard by a few years, so presumably that env var and the change in behavior was introduced when POSIX was first established. I'm not familiar with the detailed history, but that example is beside my point. > Something might have been a good design originally and become a bad design as the world changed around it. Again, you're misunderstanding what a wart is. It has nothing to do with bad design, and it doesn't appear on its own over time. It is a deliberate quirk that deviates from the program's original design—whatever that may be—in order to support some functionality without breaking backwards compatibility. Just because you don't like the design doesn't mean that the software has warts. Read that again. Those fancy shells you mention have at least one major drawback: interop between tools only works if structured I/O is supported by each tool, whether that is because they were written from scratch, or because it was tacked on via shims. In practice, this can only be done if the entire ecosystem is maintained by a single organization. And if that strict interface needs to be changed, good luck updating every single tool while maintaining backwards compatibility. Talk about warts... In contrast, using unstructured text streams as the universal interface means that programs written 40 years ago can interoperate with programs written today, without either author having to add explicit support for it, and without requiring centralized maintenance. Most well-designed programs do support flags like `-0` to indicate a null separator, which has become somewhat of a standard, but otherwise the user is in control of how that interop happens. This freedom is a good thing, and is precisely what makes programs infinitely composable. In any case, this discussion has veered off from my original point and the article, because you don't like how Unix programs are designed, and misunderstand what warts are. So for that reason, I'm out.

imiric 1y ago

I think you're misunderstanding what a "wart" is. None of those examples are warts. They're the way those programs work, which you can easily understand by reading their documentation. Your point about `cat` is an example of misuse, and `cut` is needed because of the deliberate design decision to make text streams the universal interface, which has far greater benefits than forcing any specific separator. A wart is not surprising or unintuitive behavior. It is a design quirk—a feature that sticks out from the program's overall design, usually meant to handle edge cases, or to preserve backwards compatibility. For example, some GNU programs like `grep` and `find` support a `POSIXLY_CORRECT` env var, which changes their interface and behavior. My point is that the complexity of the software and its interface is directly proportional to the likelihood of it developing "warts". `find` is a much more complex program than `cut` or `uniq`, therefore it has developed warts. Next time, please address the point of the comment you're replying to, instead of forcing some counterargument that happens to be wrong.

jodrellblank 1y ago
The context of the blog post is that "warts" are the way something was originally designed or built, which is now outdated and you would change it if you could, but you don't want to break backwards compatibility so you don't. > "For example, some GNU programs like `grep` and `find` support a `POSIXLY_CORRECT` env var, which changes their interface and behavior." By your own logic that's not a wart, that's the way the programs work and they are documented to work like that. Twisting yourself into knots to pretend Linux/Unix utilities are perfectly designed isn't good logic. There's no excuse "it was designed that way" or "it's documented to work like this" which makes something not warty. Something might have been a good design originally and become a bad design as the world changed around it. For example unserialised byte streams where you have to fight whether \n or 0 (null byte) are data or record separators, is a wart. How do we know? Because when people design newer tools they don't keep that, they design it away. Serialised JSON and jq, .NET objects in PowerShell, structured data in nushell[1] and oilshell YSH[2]. [1] https://www.nushell.sh/ https://www.nushell.sh/ [2] https://oils.pub/ysh.html https://oils.pub/ysh.html
- imiric 1y ago
  Ugh, you're exhausting. Look, here's a program: print "Hello" Do you see any warts in it? No? Well, my point is that the likelihood of a program developing warts is partly related to its complexity. That's it. I mentioned Unix tools as an example of this, since they're by far the most reliable tools I use, particularly the simpler ones. I never said that they were "perfectly designed". Don't put words in my mouth. > By your own logic that's not a wart, that's the way the programs work and they are documented to work like that. A wart can be documented. I mentioned the case of `POSIXLY_CORRECT` since GNU tools predate the POSIX standard by a few years, so presumably that env var and the change in behavior was introduced when POSIX was first established. I'm not familiar with the detailed history, but that example is beside my point. > Something might have been a good design originally and become a bad design as the world changed around it. Again, you're misunderstanding what a wart is. It has nothing to do with bad design, and it doesn't appear on its own over time. It is a deliberate quirk that deviates from the program's original design—whatever that may be—in order to support some functionality without breaking backwards compatibility. Just because you don't like the design doesn't mean that the software has warts. Read that again. Those fancy shells you mention have at least one major drawback: interop between tools only works if structured I/O is supported by each tool, whether that is because they were written from scratch, or because it was tacked on via shims. In practice, this can only be done if the entire ecosystem is maintained by a single organization. And if that strict interface needs to be changed, good luck updating every single tool while maintaining backwards compatibility. Talk about warts... In contrast, using unstructured text streams as the universal interface means that programs written 40 years ago can interoperate with programs written today, without either author having to add explicit support for it, and without requiring centralized maintenance. Most well-designed programs do support flags like `-0` to indicate a null separator, which has become somewhat of a standard, but otherwise the user is in control of how that interop happens. This freedom is a good thing, and is precisely what makes programs infinitely composable. In any case, this discussion has veered off from my original point and the article, because you don't like how Unix programs are designed, and misunderstand what warts are. So for that reason, I'm out.