12 ms·

Unix Is Not an Acceptable Unix (2015)

123 points by l1n 10y ago

faragon 10y ago
Unix is OK. That article is bullshit, in my opinion.
bigger_cheese 10y ago
I think in some ways using flags to modify the behavior of a command is an elegant solution. The alternative is to rewrite a new version of "ls" for each sort of functionality you'd like. Without flags you'd need "ls1", "ls2" etc. like you do with system calls ala. dup()->dup2()->dup3(), pipe() -> pipe2() etc. Linux has gone further than some *nixs and started using flags to extend system calls as well see: https://lwn.net/Articles/585415/ https://lwn.net/Articles/585415/
- JasonSage 10y ago
  The alternative is not to write multiple `ls` commands. The alternative is to let `ls` list directory contents and to leave filtering, sorting, or customizing the output display to a separate utility which specializes in those.
  - Rotten194 10y ago
    ls | filter-hidden | sort | add-directory-slashes | mark-symlinks | mark-executables | colorize | display-in-grid This seems really inconvenient, but who cares because it's ideologically sound!
    - JasonSage 10y ago
      You could make it as convenient as existing flags are, only you wouldn't have to learn a different set of flags for every cli. No need for sarcasm.
      - flukus 10y ago
        How would you make it as convenient as existing flags?
        JasonSage 10y ago
        Not sure if you're being serious or not, but if I had to do it today, instead of: `ls | filter-hidden | sort | add-directory-slashes | mark-symlinks | mark-executables | colorize | display-in-grid` I'd start with this: `ls | nodot | sort | slash | dec | c | grid` I imagine there's an even better way, I just haven't spent very much time thinking about it. There's an argument that this could be even more convenient than existing flags—`slash` could work on output from other commands, like `find`—only I don't have to pull up a man page for `find` since I'm not as familiar with it as I am with `ls`. That's the idea behind this kind of composability. If this is actually too much typing or you're not happy with the defaults, a few bash aliases would go a long way towards making this shorter. For instance, you could alias `ls` such that it colorizes, sorts, and hides dotfiles by default.
        flukus 10y ago
        That doesn't seem more convenient than "ls -abc", whatever the flags are.
        pjc50 10y ago
        How many of those filter systems would be useful for things that aren't ls? Also, these days the cost of doing an extra fs lookup + stat call for each file in each filter is pretty small, but if you want to run that pipeline in a directory of >100k files it might be noticeable.
        JdeBP 10y ago
        The cost of the extra lookups goes away entirely if one then extends the design in the ways already discussed elsewhere on this very page. Instead of the pipeline passing along a stream of bytes, which every program then parses into a stream of filenames, the pipeline passes along a stream of objects. Then "sort" becomes a utility that sorts a stream of objects by a chosen set of fields; "slash" becomes a utility that inspects the "file type" and "file name" field of an object and makes an appropriate "display name" field; and "ls" becomes a utility that outputs a stream of objects that have "file type" and "file name" fields, straight out of the directory entry. "ls" is the only part of the pipeline that looks at the disc.
    - nottorp 10y ago
      You could improve it further - on every ls spin up several hadoop instances to do your map/reduce to process the output! Timeless reference: https://www.joelonsoftware.com/2001/04/21/dont-let-architecture-astronauts-scare-you/ https://www.joelonsoftware.com/2001/04/21/dont-let-architect...
  - bigger_cheese 10y ago
    I think it's the old micro vs monolitihc argument. If you rely on communication between external utilities you have to be pretty clear where you draw the line - I don't want to end up with javascript like leftpad debacle on my command line apps.
MikeTaylor 10y ago
It seems that the lessons of Rob Pike's 1983 talk "cat -v Considered Harmful" have still not been absorbed 34 years later. Here is the abstract of the talk, together with the link to the paper (co-authored with Brian Kernighan) that evolved from it: http://gaul.org/files/cat_-v_considered_harmful.html http://gaul.org/files/cat_-v_considered_harmful.html
- deleted 10y ago
  [deleted]
- pjmlp 10y ago
  I is kind of interesting that FOSS community argues for the spirit of UNIX, when its creators all agree that UNIX as OS model is long overdue.
  - flukus 10y ago
    Where are the better options? There are plenty of warts, as you'd expect from something that old, but dealing with those warts (or beauty spots) is still the best option I know of.
    - pjmlp 10y ago
      Yes, the school of though from Xerox PARC and ETHZ workstation systems and safe systems programming languages, with development environment deeply integrated with the OS. Even Plan 9 and Inferno are more close to that design model than UNIX, hence Rob Pike's complaints about UNIX, how ACME works or Limbo's design.
    - mfukar 10y ago
      The better options are struggling for support while the rest of us pour our money on the problematic UNIX, hoping we can 'deal' with its warts. In one word, inertia.
charles-salvia 10y ago
What's funny is that some modern scripting languages like Python, with things like the "os" module, are almost passable as a general purpose shell scripting environment. I actually tend to write Python scripts instead of bash shell scripts for one-off file manipulation scripting, mostly because even after a decade-plus long career, I've never become much of a wizard at bash shell staples like sed or awk. But it's just that a general purpose programming language like Python doesn't treat executing subprocesses in enough of a first-class manner, in the way the bash shell does, to make it feel like a seamless shell-scripting experience. You still have to go through the motions of "import os", etc. just to get started, and you can't directly execute other programs, instead having to use "system" or the more cumbersome subprocess module. But still, it's.... tantalizingly close to something like a "better" shell that includes support for lists and other data structures instead of just "everything is a string".
- lgas 10y ago
  You might find ipython more compelling as an interactive shell: https://ipython.org/ipython-doc/3/interactive/shell.html https://ipython.org/ipython-doc/3/interactive/shell.html
  - jhasse 10y ago
    ptpython is also nice: https://github.com/jonathanslenders/ptpython https://github.com/jonathanslenders/ptpython
- thomastjeffery 10y ago
  For python, Xonsh makes a good attempt at bridging this gap. Ruby also does this well, since everything in backticks runs in a shell (just like in bash).
- theonemind 10y ago
  The near-miss of Python is why I write perl scripts. It supports the 'backtick' notation, and pretty much kills the problem domain of awk, sed, and bash.
thomastjeffery 10y ago
Unix is a poor implementation of the Unix philosophy. Just like QWERTY is an awful keyboard layout, etc. This is yet another case where we are steeped in tradition. It would be fantastic if we could just instantly overcome these shortcomings, but it's difficult enough to make an OS compatible with the ones we already have. With manufacturers like Nvidia and Broadcom keeping such a tight control over their drivers, it's difficult enough to even make an OS! It would be fantastic to have a more cohesive environment, especially in (and including) the shell, but most people are happy with the status quo. Even worse than *nix tools, there is really no environment outside the realm of text-based UI that follows the Unix philosophy. A GUI that does just one thing, let alone well? Ha. AFAIK, there is no existing way to create GUI software that can cohesively inter-operate with the expressive power that pipes and subshells give command-line tools.
- alkonaut 10y ago
  With unix that's exactly the mistake - what stops us from changing how some of the standard tools work in unix? They are used in existing scripts! Making the same set of tools be interactive (so we need them to be concise, simple) and used in scripting (so we need them to be constant over time) has led us here. They now form a programming API instead of a set of tools that each do one thing well.
coffeemug 10y ago
Deciding whether a program does "one thing well" is a matter of granularity. `ls` with its 38 flags is absolutely tiny compared to, say, finder, which displays multimedia previews, allows you to connect to remote servers, and probably does a bazillion other things. Yes, `ls` isn't perfectly clean in a mathematical sense, but it's still tiny compared to modern software, and passing a couple of flags for common tasks now and then is far more convenient than writing `ls | map ... | sort ...`. Unix is not a mathematically perfect Unix, but it is most definitely an acceptable one.
- vacri 10y ago
  The article hinges far, far too strongly on the "do one thing" part of the definition, and largely ignores the "do it well". The optional flags to 'ls' are there to help "do it well". If we're going to use the overzealous puritism of the article's definition, why not go further? My keyboard driver isn't unix-ey, because I can use it to input a 'k'. Or a 'K'. Or a lot of other characters. The argument in the article would mean that for a keyboard driver to be unix-ey, it should only output a single character. Nevermind the fact that I have 104 options on the HID at my fingertips, if my driver allowed that, it wouldn't be "doing one thing"!
  - skierscott 10y ago
    > The optional flags to 'ls' are there to help "do it well" And by default `ls` does it's job well.
    - mfukar 10y ago
      By default, `ls` doesn't even list all files. So no, it does not do its job, let alone 'well'.
      - oneeyedpigeon 10y ago
        It hides hidden files by default. It would be bizarre if it didn't do that.
quanticle 10y ago
Unix permits programs to communicate with one another, and with the user, exclusively through character streams. You can’t write a function that returns a list of files because the shell doesn’t know what a “list” is, doesn’t know what “files” are, and couldn’t tell you the difference between a “function” and a “program” if its life depended on it. Programs don’t “take arguments” and “return values”, they read characters from stdin and print characters to stdout! And this is why I consider PowerShell a more Unix-y shell than bash. Powershell commands don't just operate on character streams. They operate on CLR objects. For example, the PowerShell equivalent of grep, Select-String, doesn't return a string, or even a list of strings. It returns a list of match objects, which contain the regexp used, the matching string, and the file that matched. PowerShell often gets dinged for its verbosity, but honestly, I find it much easier to compose PowerShell commands because I know exactly what each command does, and the command-line flags are mostly intuitive. Meanwhile, Unix commands are shorter, yes, but it's a lot more difficult for me to remember, e.g. that if I want to run a command across a number of files, I have to pass the command to find.
- jhbadger 10y ago
  I remember VMS fans praising their proprietary ecosystem of commands. How's that working out for them?
  - nailer 10y ago
    Powershell is Open Source. https://github.com/PowerShell/PowerShell/blob/master/LICENSE.txt https://github.com/PowerShell/PowerShell/blob/master/LICENSE...
  - ksk 10y ago
    You only say that because SCO lost a lawsuit. UNIX was a closed source proprietary OS.
    - cestith 10y ago
      SCO lost a lawsuit because all claims brought in the suit were entirely baseless.
MichaelMoser123 10y ago
I think the unix design philosophy is great in theory, but in practice you get a lot of parts that have to be combined, this combination then gets tricky, so you will want a utility that just covers all the edges. Lets see, I often use ls -tr * the good thing about this is that it is very fast to type * the bad part is that you have to memorize the incantation. You could probably do it by a pipe (ls | sort <now it gets complicated here>) * good: the good part is that every program does its job * bad: also two programs have a larger footprint than one * good: you can have it as an alias to make it short again * bad: aliases can't be used in bash scripts, unless you explicitly source the file that defines the alias * good/bad: you just shifted complexity from ls to sort ....
- nailer 10y ago
  > ls | sort <now it gets complicated here> In Powershell, from memory: ls | sort -property time https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.utility/sort-object https://msdn.microsoft.com/en-us/powershell/reference/5.1/mi...
  - flukus 10y ago
    "man ls" in bash showed me what I need, doing the same in powershell gave me some nonsense about Get-ChildItem.
    - sha666sum 10y ago
      In PowerShell, ls is an alias for Get-ChildItem.
      - flukus 10y ago
        That doesn't really help me though does it? "get-help Get-ChildItem" isn't any better. "get-help Get-ChildItem -examples" is a bit better, "get-help Get-ChildItem -detailed" is what I'm after, why on earth doesn't "man ls" map to that? It's the same story with sort. Know how am I supposed to know what type ls returns and what properties are available? For that matter, how am I supposed to know what type of object I'm dealing with? Time isn't displayed by the command and is different to LastWriteTime, how do I discover that it's a property? So far in this ultra simple example powershell is looking over complicated and under featured.
f2f 10y ago
Back in 1983 the authors of UNIX were already complaining about the proliferation of incompatibilities. As far as "true unix" is concerned nothing this paper refers to is considered "true unix". It seems that UNIX has become the victim of cancerous growth at the hands of organizations such as UCB. 4.2BSD is an order of magnitude larger than Version 5, but, Pike claims, not ten times better. http://harmful.cat-v.org/cat-v/ http://harmful.cat-v.org/cat-v/
deleted 10y ago
[deleted]
Animats 10y ago
There was a time (PWB Research Unix) when one typed: ls | sort | mc to get an alphabetical list in multiple columns. That didn't last. This reads like someone discovered functional programming and now wants everything to be functional.
- jff 10y ago
  Plan 9's 'ls' still behaves in the old fashion, and if you want it formatted nicely you run 'lc', which is just a shell script that does ls | mc
- oneeyedpigeon 10y ago
  Isn't that kinda the point? Why would that be a bad thing? Seemingly, my version of ls doesn't provide a 'sort alphabetically' flag, but if it did, the equivalent of ls | sort | mc would be something like ls -C --sort=name and, as discussed, the ls program has to be much, much more complicated.
  - Animats 10y ago
    UNIX pipeline data is mostly raw strings. To do anything non-trivial, you have to know the format of the previous pipeline item and have a parser for it. It would be amusing to have a system where all commands output JSON, so that pipelined programs can process the previous input. Then you could have ls | sort --key=modification_time | mc For display, you'd have a convention that if there's a "text" field in the JSON, that's what's displayed in the console window. Otherwise you see the raw JSON. So "ls" would output an array of JSON records with all the usual fields, but if you just wrote ls the user would see a list of file names. You could have "format" and "select" programs: ls | select --exclude="owner=root" | sort --key=modification_time | format --fields=name,modification_time,size This would be quite powerful and would allow the creation of complex, functional pipelines. The end result is basically equivalent to SQL, but it's functional!
    - sbierwagen 10y ago
      This was the point of Powershell, where programs pass around objects, rather than plain text: https://en.wikipedia.org/wiki/PowerShell#Pipeline https://en.wikipedia.org/wiki/PowerShell#Pipeline
tempodox 10y ago
Lisp machines once had all it took to make better interfaces but were wiped out by the worse-is-better logic behind Unix. Now the weight of all the stuff we built on top of that makes it harder to change. And almost everyone still subscribes to worse-is-better, albeit under a different name.
- progman 10y ago
  McCLIM is an attempt to revive the awesome GUI of the Lisp Machines. There is no need to invent special data interchange formats like XML or native blobs since Lisp uses simple lists as AST. https://common-lisp.net/project/mcclim/excite.html https://common-lisp.net/project/mcclim/excite.html
TheOtherHobbes 10y ago
"Unix considered harmful" I think "a demonstrably inhumane environment" may be my favourite description to date. There are many terrible assumptions in the design of the shell. Not least: Use short commands (because they're quicker to type in theory, even though you'll waste time huge amounts of time in practice looking up commands and switches, because even the average god-like greybeard can't remember more than a few tens, and beginners have no chance) Use terse switches (ditto) Text is universal and computers are text processors (not true back then, even less true now) Make no attempt at standardisation (switches are effectively random between different commands) Quick hacks are better than a smart and structured environment (this was always bullshit and has caused endless grief, not just in Unix, but in everything touched by Unix) Software forks are good (not in the shell, they're not) And now computing is buried under the dead weight of accumulated cruft and poor theory and practice - probably forever. But that isn't even the real problem. The real problem is that research into humane OS design seems to have stalled. CS academics could band together to design something smarter, better, and friendlier than the current shell. But there seems to be zero interest in starting a project that would generate huge productivity and reliability gains and also make low-level computing more accessible to ordinary users. I find that baffling, and actually rather disturbing.
- adrianN 10y ago
  There are shells other than SH and derivatives. They're not terribly popular.
  - oblio 10y ago
    That's the IE/Safari/Android/vi problem: defaults.
bjourne 10y ago
> Unix permits programs to communicate with one another, and with the user, exclusively through character streams. It communicates using byte streams. For the original authors of Unix where ascii ruled supreme, it didn't make a difference but now it does. For example, see this stack question: http://unix.stackexchange.com/questions/167814/get-first-x-characters-from-the-cat-command http://unix.stackexchange.com/questions/167814/get-first-x-c...
nsb1 10y ago
The arguments of ls are simply an optimization for things which are done so frequently that simply typing them over and over becomes a huge waste of time. In the early days, you can imagine people first typing everything out manually, then creating macros like 'ls | awk {stuff} | sort -k2 |..." and then just getting fed up with your .xxxrc file and modifying the ls source to trim it down to a couple of arguments. Taken to extremes, this article seems to suggest that all command line flags for every executable ought to be replaced with separate executables and pipes. Just imagine how many files would be in /usr/bin, and how many of them you would have to remember just to do useful work every day. We'd have to increase inode limits just to hold all the manpages. An OS like this would not be one I'd like to use.
glangdale 10y ago
What always gets me about this class of articles is that they are a classic case of bikeshedding. Is anyone really having that much trouble with ls? Yes, things like ls could be better - but is switching between "ls --plus -a --pile --of --options" and "ls | one | damn | filter | after | another" a particularly core concern? I tend to agree with this bit... "Rather than re-evaluating the Unix command line with an eye towards improving its usability under the greatly relaxed technological constraints of modern hardware, we’ve written terminal emulators that faithfully reproduce the constraints of the mid-1970s", ... but feel that the author decided that it was a lot more fun and easy to complain about ls instead of properly exploring this idea.
- ygra 10y ago
  Unless Unix dies a swift death for some reason or another, going forward there will be many more people who have to learn this than there are that have learned it already. Surely, once you're using those things on a daily basis it doesn't matter how inconvenient, illogical, or badly designed things are, you just know how to do them right. Is that a good reason to scoff at anyone who points out that things are badly-designed? Backwards-compatibility seems a far better argument in favor of keeping things like they are than »it doesn't bother me much in practice«.
progman 10y ago
What's the problem with all those options? IMHO ls is not bloated at all. It simply follows the Unix philosophy: Do just one thing, and do it "right" which means: do it as well as possible. ls actually does that. I never had any desire for another ls because ls already gives me almost all possible functionality. I am glad that I have all those options so that I don't have to write my own tool to get a special function if I really need that. If you don't need all the options, ignore them. If ls is too complicated for you then do just as the author recommends: "write a simpler alternative to ls", and name it lf ("list files") or the like.
huherto 10y ago
I do a lot of things from the command line. But as soon as I have a few lines and I need to write a script I just switch to Perl(which I know better than Python). It just seems easier. I still call unix commands but Perl seems better to glue them together.
upofadown 10y ago
>...it was developed by and for a small group of highly specialized experts in an environment where every keystroke, every character displayed on the screen, came at a real and meaningful cost. Have keyboards gotten better/faster? How does the existence of a 80 character line length change the best way to specify a directory listing? This seems to be based on the logical fallacy that since the old thing is bad that a better new thing must be possible.

123 points by l1n 10y ago

Unix is OK. That article is bullshit, in my opinion.

I think in some ways using flags to modify the behavior of a command is an elegant solution. The alternative is to rewrite a new version of "ls" for each sort of functionality you'd like. Without flags you'd need "ls1", "ls2" etc. like you do with system calls ala. dup()->dup2()->dup3(), pipe() -> pipe2() etc. Linux has gone further than some *nixs and started using flags to extend system calls as well see: https://lwn.net/Articles/585415/ https://lwn.net/Articles/585415/

JasonSage 10y ago
The alternative is not to write multiple `ls` commands. The alternative is to let `ls` list directory contents and to leave filtering, sorting, or customizing the output display to a separate utility which specializes in those.
- Rotten194 10y ago
  ls | filter-hidden | sort | add-directory-slashes | mark-symlinks | mark-executables | colorize | display-in-grid This seems really inconvenient, but who cares because it's ideologically sound!
  - JasonSage 10y ago
    You could make it as convenient as existing flags are, only you wouldn't have to learn a different set of flags for every cli. No need for sarcasm.
    - flukus 10y ago
      How would you make it as convenient as existing flags?
      - JasonSage 10y ago
        Not sure if you're being serious or not, but if I had to do it today, instead of: `ls | filter-hidden | sort | add-directory-slashes | mark-symlinks | mark-executables | colorize | display-in-grid` I'd start with this: `ls | nodot | sort | slash | dec | c | grid` I imagine there's an even better way, I just haven't spent very much time thinking about it. There's an argument that this could be even more convenient than existing flags—`slash` could work on output from other commands, like `find`—only I don't have to pull up a man page for `find` since I'm not as familiar with it as I am with `ls`. That's the idea behind this kind of composability. If this is actually too much typing or you're not happy with the defaults, a few bash aliases would go a long way towards making this shorter. For instance, you could alias `ls` such that it colorizes, sorts, and hides dotfiles by default.
        flukus 10y ago
        That doesn't seem more convenient than "ls -abc", whatever the flags are.
        pjc50 10y ago
        How many of those filter systems would be useful for things that aren't ls? Also, these days the cost of doing an extra fs lookup + stat call for each file in each filter is pretty small, but if you want to run that pipeline in a directory of >100k files it might be noticeable.
        JdeBP 10y ago
        The cost of the extra lookups goes away entirely if one then extends the design in the ways already discussed elsewhere on this very page. Instead of the pipeline passing along a stream of bytes, which every program then parses into a stream of filenames, the pipeline passes along a stream of objects. Then "sort" becomes a utility that sorts a stream of objects by a chosen set of fields; "slash" becomes a utility that inspects the "file type" and "file name" field of an object and makes an appropriate "display name" field; and "ls" becomes a utility that outputs a stream of objects that have "file type" and "file name" fields, straight out of the directory entry. "ls" is the only part of the pipeline that looks at the disc.
  - nottorp 10y ago
    You could improve it further - on every ls spin up several hadoop instances to do your map/reduce to process the output! Timeless reference: https://www.joelonsoftware.com/2001/04/21/dont-let-architecture-astronauts-scare-you/ https://www.joelonsoftware.com/2001/04/21/dont-let-architect...
- bigger_cheese 10y ago
  I think it's the old micro vs monolitihc argument. If you rely on communication between external utilities you have to be pretty clear where you draw the line - I don't want to end up with javascript like leftpad debacle on my command line apps.

MikeTaylor 10y ago

It seems that the lessons of Rob Pike's 1983 talk "cat -v Considered Harmful" have still not been absorbed 34 years later. Here is the abstract of the talk, together with the link to the paper (co-authored with Brian Kernighan) that evolved from it: http://gaul.org/files/cat_-v_considered_harmful.html http://gaul.org/files/cat_-v_considered_harmful.html

deleted 10y ago
[deleted]
pjmlp 10y ago
I is kind of interesting that FOSS community argues for the spirit of UNIX, when its creators all agree that UNIX as OS model is long overdue.
- flukus 10y ago
  Where are the better options? There are plenty of warts, as you'd expect from something that old, but dealing with those warts (or beauty spots) is still the best option I know of.
  - pjmlp 10y ago
    Yes, the school of though from Xerox PARC and ETHZ workstation systems and safe systems programming languages, with development environment deeply integrated with the OS. Even Plan 9 and Inferno are more close to that design model than UNIX, hence Rob Pike's complaints about UNIX, how ACME works or Limbo's design.
  - mfukar 10y ago
    The better options are struggling for support while the rest of us pour our money on the problematic UNIX, hoping we can 'deal' with its warts. In one word, inertia.

charles-salvia 10y ago

What's funny is that some modern scripting languages like Python, with things like the "os" module, are almost passable as a general purpose shell scripting environment. I actually tend to write Python scripts instead of bash shell scripts for one-off file manipulation scripting, mostly because even after a decade-plus long career, I've never become much of a wizard at bash shell staples like sed or awk. But it's just that a general purpose programming language like Python doesn't treat executing subprocesses in enough of a first-class manner, in the way the bash shell does, to make it feel like a seamless shell-scripting experience. You still have to go through the motions of "import os", etc. just to get started, and you can't directly execute other programs, instead having to use "system" or the more cumbersome subprocess module. But still, it's.... tantalizingly close to something like a "better" shell that includes support for lists and other data structures instead of just "everything is a string".

lgas 10y ago
You might find ipython more compelling as an interactive shell: https://ipython.org/ipython-doc/3/interactive/shell.html https://ipython.org/ipython-doc/3/interactive/shell.html
- jhasse 10y ago
  ptpython is also nice: https://github.com/jonathanslenders/ptpython https://github.com/jonathanslenders/ptpython
thomastjeffery 10y ago
For python, Xonsh makes a good attempt at bridging this gap. Ruby also does this well, since everything in backticks runs in a shell (just like in bash).
theonemind 10y ago
The near-miss of Python is why I write perl scripts. It supports the 'backtick' notation, and pretty much kills the problem domain of awk, sed, and bash.

thomastjeffery 10y ago

Unix is a poor implementation of the Unix philosophy. Just like QWERTY is an awful keyboard layout, etc. This is yet another case where we are steeped in tradition. It would be fantastic if we could just instantly overcome these shortcomings, but it's difficult enough to make an OS compatible with the ones we already have. With manufacturers like Nvidia and Broadcom keeping such a tight control over their drivers, it's difficult enough to even make an OS! It would be fantastic to have a more cohesive environment, especially in (and including) the shell, but most people are happy with the status quo. Even worse than *nix tools, there is really no environment outside the realm of text-based UI that follows the Unix philosophy. A GUI that does just one thing, let alone well? Ha. AFAIK, there is no existing way to create GUI software that can cohesively inter-operate with the expressive power that pipes and subshells give command-line tools.

alkonaut 10y ago
With unix that's exactly the mistake - what stops us from changing how some of the standard tools work in unix? They are used in existing scripts! Making the same set of tools be interactive (so we need them to be concise, simple) and used in scripting (so we need them to be constant over time) has led us here. They now form a programming API instead of a set of tools that each do one thing well.

coffeemug 10y ago

Deciding whether a program does "one thing well" is a matter of granularity. `ls` with its 38 flags is absolutely tiny compared to, say, finder, which displays multimedia previews, allows you to connect to remote servers, and probably does a bazillion other things. Yes, `ls` isn't perfectly clean in a mathematical sense, but it's still tiny compared to modern software, and passing a couple of flags for common tasks now and then is far more convenient than writing `ls | map ... | sort ...`. Unix is not a mathematically perfect Unix, but it is most definitely an acceptable one.

vacri 10y ago
The article hinges far, far too strongly on the "do one thing" part of the definition, and largely ignores the "do it well". The optional flags to 'ls' are there to help "do it well". If we're going to use the overzealous puritism of the article's definition, why not go further? My keyboard driver isn't unix-ey, because I can use it to input a 'k'. Or a 'K'. Or a lot of other characters. The argument in the article would mean that for a keyboard driver to be unix-ey, it should only output a single character. Nevermind the fact that I have 104 options on the HID at my fingertips, if my driver allowed that, it wouldn't be "doing one thing"!
- skierscott 10y ago
  > The optional flags to 'ls' are there to help "do it well" And by default `ls` does it's job well.
  - mfukar 10y ago
    By default, `ls` doesn't even list all files. So no, it does not do its job, let alone 'well'.
    - oneeyedpigeon 10y ago
      It hides hidden files by default. It would be bizarre if it didn't do that.

quanticle 10y ago

Unix permits programs to communicate with one another, and with the user, exclusively through character streams. You can’t write a function that returns a list of files because the shell doesn’t know what a “list” is, doesn’t know what “files” are, and couldn’t tell you the difference between a “function” and a “program” if its life depended on it. Programs don’t “take arguments” and “return values”, they read characters from stdin and print characters to stdout! And this is why I consider PowerShell a more Unix-y shell than bash. Powershell commands don't just operate on character streams. They operate on CLR objects. For example, the PowerShell equivalent of grep, Select-String, doesn't return a string, or even a list of strings. It returns a list of match objects, which contain the regexp used, the matching string, and the file that matched. PowerShell often gets dinged for its verbosity, but honestly, I find it much easier to compose PowerShell commands because I know exactly what each command does, and the command-line flags are mostly intuitive. Meanwhile, Unix commands are shorter, yes, but it's a lot more difficult for me to remember, e.g. that if I want to run a command across a number of files, I have to pass the command to find.

jhbadger 10y ago
I remember VMS fans praising their proprietary ecosystem of commands. How's that working out for them?
- nailer 10y ago
  Powershell is Open Source. https://github.com/PowerShell/PowerShell/blob/master/LICENSE.txt https://github.com/PowerShell/PowerShell/blob/master/LICENSE...
- ksk 10y ago
  You only say that because SCO lost a lawsuit. UNIX was a closed source proprietary OS.
  - cestith 10y ago
    SCO lost a lawsuit because all claims brought in the suit were entirely baseless.

MichaelMoser123 10y ago

I think the unix design philosophy is great in theory, but in practice you get a lot of parts that have to be combined, this combination then gets tricky, so you will want a utility that just covers all the edges. Lets see, I often use ls -tr * the good thing about this is that it is very fast to type * the bad part is that you have to memorize the incantation. You could probably do it by a pipe (ls | sort <now it gets complicated here>) * good: the good part is that every program does its job * bad: also two programs have a larger footprint than one * good: you can have it as an alias to make it short again * bad: aliases can't be used in bash scripts, unless you explicitly source the file that defines the alias * good/bad: you just shifted complexity from ls to sort ....

nailer 10y ago
> ls | sort <now it gets complicated here> In Powershell, from memory: ls | sort -property time https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.utility/sort-object https://msdn.microsoft.com/en-us/powershell/reference/5.1/mi...
- flukus 10y ago
  "man ls" in bash showed me what I need, doing the same in powershell gave me some nonsense about Get-ChildItem.
  - sha666sum 10y ago
    In PowerShell, ls is an alias for Get-ChildItem.
    - flukus 10y ago
      That doesn't really help me though does it? "get-help Get-ChildItem" isn't any better. "get-help Get-ChildItem -examples" is a bit better, "get-help Get-ChildItem -detailed" is what I'm after, why on earth doesn't "man ls" map to that? It's the same story with sort. Know how am I supposed to know what type ls returns and what properties are available? For that matter, how am I supposed to know what type of object I'm dealing with? Time isn't displayed by the command and is different to LastWriteTime, how do I discover that it's a property? So far in this ultra simple example powershell is looking over complicated and under featured.

f2f 10y ago

Back in 1983 the authors of UNIX were already complaining about the proliferation of incompatibilities. As far as "true unix" is concerned nothing this paper refers to is considered "true unix". It seems that UNIX has become the victim of cancerous growth at the hands of organizations such as UCB. 4.2BSD is an order of magnitude larger than Version 5, but, Pike claims, not ten times better. http://harmful.cat-v.org/cat-v/ http://harmful.cat-v.org/cat-v/

deleted 10y ago

[deleted]

Animats 10y ago

There was a time (PWB Research Unix) when one typed: ls | sort | mc to get an alphabetical list in multiple columns. That didn't last. This reads like someone discovered functional programming and now wants everything to be functional.

jff 10y ago
Plan 9's 'ls' still behaves in the old fashion, and if you want it formatted nicely you run 'lc', which is just a shell script that does ls | mc
oneeyedpigeon 10y ago
Isn't that kinda the point? Why would that be a bad thing? Seemingly, my version of ls doesn't provide a 'sort alphabetically' flag, but if it did, the equivalent of ls | sort | mc would be something like ls -C --sort=name and, as discussed, the ls program has to be much, much more complicated.
- Animats 10y ago
  UNIX pipeline data is mostly raw strings. To do anything non-trivial, you have to know the format of the previous pipeline item and have a parser for it. It would be amusing to have a system where all commands output JSON, so that pipelined programs can process the previous input. Then you could have ls | sort --key=modification_time | mc For display, you'd have a convention that if there's a "text" field in the JSON, that's what's displayed in the console window. Otherwise you see the raw JSON. So "ls" would output an array of JSON records with all the usual fields, but if you just wrote ls the user would see a list of file names. You could have "format" and "select" programs: ls | select --exclude="owner=root" | sort --key=modification_time | format --fields=name,modification_time,size This would be quite powerful and would allow the creation of complex, functional pipelines. The end result is basically equivalent to SQL, but it's functional!
  - sbierwagen 10y ago
    This was the point of Powershell, where programs pass around objects, rather than plain text: https://en.wikipedia.org/wiki/PowerShell#Pipeline https://en.wikipedia.org/wiki/PowerShell#Pipeline

tempodox 10y ago

Lisp machines once had all it took to make better interfaces but were wiped out by the worse-is-better logic behind Unix. Now the weight of all the stuff we built on top of that makes it harder to change. And almost everyone still subscribes to worse-is-better, albeit under a different name.

progman 10y ago
McCLIM is an attempt to revive the awesome GUI of the Lisp Machines. There is no need to invent special data interchange formats like XML or native blobs since Lisp uses simple lists as AST. https://common-lisp.net/project/mcclim/excite.html https://common-lisp.net/project/mcclim/excite.html

TheOtherHobbes 10y ago

"Unix considered harmful" I think "a demonstrably inhumane environment" may be my favourite description to date. There are many terrible assumptions in the design of the shell. Not least: Use short commands (because they're quicker to type in theory, even though you'll waste time huge amounts of time in practice looking up commands and switches, because even the average god-like greybeard can't remember more than a few tens, and beginners have no chance) Use terse switches (ditto) Text is universal and computers are text processors (not true back then, even less true now) Make no attempt at standardisation (switches are effectively random between different commands) Quick hacks are better than a smart and structured environment (this was always bullshit and has caused endless grief, not just in Unix, but in everything touched by Unix) Software forks are good (not in the shell, they're not) And now computing is buried under the dead weight of accumulated cruft and poor theory and practice - probably forever. But that isn't even the real problem. The real problem is that research into humane OS design seems to have stalled. CS academics could band together to design something smarter, better, and friendlier than the current shell. But there seems to be zero interest in starting a project that would generate huge productivity and reliability gains and also make low-level computing more accessible to ordinary users. I find that baffling, and actually rather disturbing.

adrianN 10y ago
There are shells other than SH and derivatives. They're not terribly popular.
- oblio 10y ago
  That's the IE/Safari/Android/vi problem: defaults.

bjourne 10y ago

> Unix permits programs to communicate with one another, and with the user, exclusively through character streams. It communicates using byte streams. For the original authors of Unix where ascii ruled supreme, it didn't make a difference but now it does. For example, see this stack question: http://unix.stackexchange.com/questions/167814/get-first-x-characters-from-the-cat-command http://unix.stackexchange.com/questions/167814/get-first-x-c...

nsb1 10y ago

The arguments of ls are simply an optimization for things which are done so frequently that simply typing them over and over becomes a huge waste of time. In the early days, you can imagine people first typing everything out manually, then creating macros like 'ls | awk {stuff} | sort -k2 |..." and then just getting fed up with your .xxxrc file and modifying the ls source to trim it down to a couple of arguments. Taken to extremes, this article seems to suggest that all command line flags for every executable ought to be replaced with separate executables and pipes. Just imagine how many files would be in /usr/bin, and how many of them you would have to remember just to do useful work every day. We'd have to increase inode limits just to hold all the manpages. An OS like this would not be one I'd like to use.

glangdale 10y ago

What always gets me about this class of articles is that they are a classic case of bikeshedding. Is anyone really having that much trouble with ls? Yes, things like ls could be better - but is switching between "ls --plus -a --pile --of --options" and "ls | one | damn | filter | after | another" a particularly core concern? I tend to agree with this bit... "Rather than re-evaluating the Unix command line with an eye towards improving its usability under the greatly relaxed technological constraints of modern hardware, we’ve written terminal emulators that faithfully reproduce the constraints of the mid-1970s", ... but feel that the author decided that it was a lot more fun and easy to complain about ls instead of properly exploring this idea.

ygra 10y ago
Unless Unix dies a swift death for some reason or another, going forward there will be many more people who have to learn this than there are that have learned it already. Surely, once you're using those things on a daily basis it doesn't matter how inconvenient, illogical, or badly designed things are, you just know how to do them right. Is that a good reason to scoff at anyone who points out that things are badly-designed? Backwards-compatibility seems a far better argument in favor of keeping things like they are than »it doesn't bother me much in practice«.

progman 10y ago

What's the problem with all those options? IMHO ls is not bloated at all. It simply follows the Unix philosophy: Do just one thing, and do it "right" which means: do it as well as possible. ls actually does that. I never had any desire for another ls because ls already gives me almost all possible functionality. I am glad that I have all those options so that I don't have to write my own tool to get a special function if I really need that. If you don't need all the options, ignore them. If ls is too complicated for you then do just as the author recommends: "write a simpler alternative to ls", and name it lf ("list files") or the like.

huherto 10y ago

I do a lot of things from the command line. But as soon as I have a few lines and I need to write a script I just switch to Perl(which I know better than Python). It just seems easier. I still call unix commands but Perl seems better to glue them together.

upofadown 10y ago

>...it was developed by and for a small group of highly specialized experts in an environment where every keystroke, every character displayed on the screen, came at a real and meaningful cost. Have keyboards gotten better/faster? How does the existence of a 80 character line length change the best way to specify a directory listing? This seems to be based on the logical fallacy that since the old thing is bad that a better new thing must be possible.