18 ms·

Unix doesn’t follow the Unix philosophy

50 points by prajjwal 14y ago

udp 14y ago
> Why does ls have dozens of tangled options? Because users, even Unix users, are not overly fond of the first two points of the Unix philosophy. No, ls(1) is flexible because the philosophy is to do one thing and do it well. ls(1) lists information about files. That's what `man ls` says it will do, and whichever flags you pass, that's what it's going to do. Providing a lot of control over how exactly it does that is (imho) doing it well.
- betawolf33 14y ago
  In a sense, yes. However, some of the things ls does are 'borrowed functionality'. That the displayed listing can be sorted by a flag to ls is a bit of an aberration in the Unix philosophy. We're meant to have a separate tool for sorting - sort. I agree that a certain flexibility is useful in the Unix tools, but a well-founded argument can be made that flexibility costs you simplicity.
  - pilif 14y ago
    The fact that ls can produce output that can't be reliably sorted using the sort utility shows that ls is actually doing three things instead of just one: Listing files, sorting the output and then displaying the output in a human readable format. This is not to criticize ls though - I would imagine that no matter how small the operation (well aside of operations executed directly by the cpu), there's still the possibility to break it up into smaller parts. At some point convenience just trumps modularity. ls -lt is just more convenient than a (hypotetical) ls -t | sort | xargs fileinfo | display-fileinfo On a related note: Windows PowerShell comes closer to this ideal by being able to send arbitrary objects between pipes. This still has the side-effect though that single commands become much more awkward to use as they don't produce output that's usable on its own.
    - cturner 14y ago
      I would imagine that no matter how small the operation (well aside of operations executed directly by the cpu), there's still the possibility to break it up into smaller parts. The scripting system could have been designed to make layering easier. Bourne descendants are ugly and awkward. Hence, unix evolved into a situation where ls had lots of features, whereas were there a better scripting system, we might have had a standard environment which you could introspect deeper and extend with more confidence.
      - ori_b 14y ago
        > The scripting system could have been designed to make layering easier. How? sh sucks for anything involved, but I can't think of a way to simplify pipelines much.
        jcmhn 14y ago
        A line oriented xargs could have been built in from day one. A pair of shell variables honored by commands that let you specify output wrappers or separators would have been nice too. Think "-Q" in gnu ls, but universally supported and user-controlled.
  - reddit_clone 14y ago
    How would you make 'sort' do all the things that 'ls' can easily do? How do you sort a multi-column output by 3rd column?. You have to write some scripts for that.
    - elwin 14y ago
      sort -k 3
      - tedunangst 14y ago
        That works for the trivial case of the 3rd column, but not for everything ls can produce. There's no way to make sort work on dates as formatted by ls (which may or may not have years or hours, depending on how old) or sizes (which you may want to pretty print as human readable.) What sort option puts 1.0K, 1.0M, and 1.0G in the right order?
        betawolf33 14y ago
        While I do appreciate the point that ls is doing sorting that sort would struggle to manage, isn't this an argument for better functionality in sort? Someone pointed out that ls is really doing three things - listing files, sorting them and displaying them. That seems fairly accurate, and indeed it's convenient that it does. But it's not 'doing one thing'. I'm not saying that's necessarily bad (as I just said, it's convenient), but it's not hardline Unix-way purity. For practical reasons like those you raise, it makes sense for ls to handle its own sorting rather than leave it to the sort command, just as it handles its own formatting rather than let the user manage it. But those features _are_ something other than just reporting file listings, whatever the motivation for the feature creep. This convenient flexibility makes ls less single-task, less Unix-adherent but more generally useful on its own.
        masklinn 14y ago
        > There's no way to make sort work on dates as formatted by ls (which may or may not have years or hours, depending on how old) or sizes (which you may want to pretty print as human readable.) Both of which wouldn't be the job of a "do one thing and do it well" `ls`. That one would display dates as a UNIX timestamp and sizes as bytes. Aside from that, surely if ls can format information it's possible to reverse that formatting.
- telemachos 14y ago
  > No, ls has dozens of tangled options because the philosophy is to "do one thing and do it well". Do you mean that in order to truly do well its one thing (list information about files), that ls had to incorporate things like these? -o List in long format, but omit the group id. -T When used with the -l (lowercase letter ``ell'') option, display complete time information for the file, including month, day, hour, minute, second, and year. -x The same as -C, except that the multi-column output is produced with entries sorted across, rather than down, the columns. Some degree of feature creep is probably inevitable, but a lot of the common utilities now seem to lean towards "offer a flag for everything" rather than "do a small number of permutations on one thing; create pipelines for the rest". (Note: These three options are all from OSX's BSD-derived ls. GNU's version is almost certainly more complex still.)
  - masklinn 14y ago
    > that ls had to incorporate things like these? Or this bunch: -p Write a slash (`/') after each filename if that file is a directory. -r Reverse the order of the sort to get reverse lexicographical order or the oldest entries first (or largest files last, if combined with sort by size -S Sort files by size * -p's information is redundant with the first character of the long format * -S (and -r) are redundant with `sort` (ls -lSr -> ls -l | sort -k5; ls -lS -> ls -l | sort -rk5)
    - lmm 14y ago
      -p is immensely useful. Of course it's redundant in long format; it's for using in short format. -r is also very useful (I use -tr all the time, which I can't immediately see how I'd replace with sort), and while I could replace ls -lS with piping into sort, you seem to be assuming ls will always be outputting one file per line, which is far from its most useful output mode when running it in a terminal.
      - masklinn 14y ago
        > you seem to be assuming ls will always be outputting one file per line No, I am assuming a "do one thing" `ls` would be used in long format (or as inode numbers) whenever it need to be worked with, and would be massaged into form by further commands. For `t`, `r`, `S` and a bunch of others you'd ls, sort and then cut/awk/whatever to format into a "short form". Not that it matters a bit. Also, I didn't claim these weren't useful, I claimed they were redundant. Which they are.
- reddit_clone 14y ago
  I agree. It is not like 'ls' is predicting weather and telling the fortune. It lists files and directories. It has a zillion options to control its behaviour and output. I see nothing wrong with those options if they will be of use to somebody. I use 4-5 options myself and know them by heart. I do admit that a new user can get overwhelmed if he looks at the man page.
  - quesera 14y ago
    I agree. ls is primarily a user-facing utility, and has acquired a ton of display and formatting switches. This is not a bad thing. Some of the display options are superfluous but convenient. Others are awkward, very hard, or impossible to duplicate in a pipeline. New users don't need to be overwhelmed by the man page. -a and -l should be enough for >99% of use cases. ls shows up in shell scripts too, but that's almost always a mistake -- or a dependency on one of the harder-to-duplicate formatting options.
romaniv 14y ago
It’s a lot easier to maintain an orthogonal design when your software isn’t being used. Mentally, I'm translating this to "it's okay to have messy design if your software is used". This is probably the most common excuse for writing bad software. Some things in Unix really should follow Unix philosophy more closely.
- raldi 14y ago
  I'd rephrase it more like, "Clean design is a means to pleasing users; it's not an end in itself." When is it okay to get messy? When it's a net win for the user experience.
  - romaniv 14y ago
    That's exactly what I'm talking about. "Pleasing users" is often used as an excuse for horrible internal design. People cite examples like ls and proceed to conclude that the Unix philosophy is a concept of abstract philosophy not applicable to real life, when they should conclude that there are simply some issues with ls design. Nearly every time I hear "but it's okay to be messy, because it's real life" or "I must do this bad thing to please the users" the person saying it advocated something that is an insignificant short-term gain for a tiny subset of users at the expense of very significant long-term losses for all the users. Internal design usually does not affect end users directly, but it has tremendous indirect effects in terms of performance and reliability, which people do care about. LS is a good example of this. Having some obscure flag might have solved a particular problem in the past, but now everyone has to deal with extra documentation, complexity and bugs.
    - raldi 14y ago
      That may be the case sometimes, but not always. Here are some examples of what I think were good uses of messiness: ls output can be sorted in all sorts of ways with just a few characters of typing, even though it would be cleaner to make people use sort. grep has a flag for counting lines that match, even though wc is supposed to be used for counting lines. rm can remove directories with the right flags, even though rmdir is supposed to be the tool for removing directories. I think those were all net wins for the user, both in the short term and the long term.
dexen 14y ago
http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs follows the Unix philosophy. A lot of legacy has been shed. I can count 13 options to ls, 11 options to sed and just 5 to sed. The standard Plan 9 shell, Rc, is described in mere ~500 lines of manpage, while Bash takes whooping ~5400 lines. Oh, and there is no `dll hell' in P9 :-)
- telemachos 14y ago
  > A lot of legacy has been shed. An example I remember from the UNIX to Plan 9 command translation page[1]: UNIX Plan 9 head sed 10q [1]: http://www.plan9.bell-labs.com/wiki/plan9/unix_to_plan_9_command_translation/index.html http://www.plan9.bell-labs.com/wiki/plan9/unix_to_plan_9_com...
  - DeepDuh 14y ago
    Looking at this list I can't really tell why this is supposed to be better. Shedding complexity ok - but getting rid of many shorthands that are there for a reason? It reminds me of how Java wanted to replace printf with a ridiculously complex string formatting system.
    - sp332 14y ago
      The point is that "head" on your system is not the same as "sed 10q". There are two different codebases. By default head shows the first 10 lines but you can change it with flags etc. Well why bother when you already have sed to do that? head -n 5 sed 5q For most cases, head is not a shortcut. 10 is just a special case.
      - DeepDuh 14y ago
        I agree with that change. I don't with many others.
    - telemachos 14y ago
      Getting rid of good things would be bad, but is sed 10q really "ridiculously complex"?
papsosouid 14y ago
Unix followed the unix philosophy just fine. Looking at current "unix-like" software and the huge amount of bloat and cruft isn't a reasonable way to criticize unix. Unix has been obsolete for decades, and the systems that took over (BSDs, linux, GNU) didn't follow the unix philosophy very much (or at all in the case of GNU).
- unimpressive 14y ago
  > (or at all in the case of GNU). What did you expect? RMS was a MIT Hacker. Which means that his background was using Lisp, ITS, and Lisp machines.
  - papsosouid 14y ago
    I didn't say I expected GNU to follow the unix philosophy. I pointed out that it doesn't, and that saying "unix doesn't follow unix philosophy" while pointing at GNU tools for proof is absurd.
dmethvin 14y ago
> It’s a lot easier to maintain an orthogonal design when your software isn’t being used. Software that gets used becomes less orthogonal and develops diagonal shortcuts. Yep, and that's not necessarily a bad thing. Human language is exactly the same way. Irregular verbs, contractions, and slang are all diagonal shortcuts. There may be a few unneeded adornments to some Unix utilities, I agree. But remember that the `ls -l` that you might have typed in 1980 on a minicomputer running BSD still has the same output as the `ls -l` from a PC running Linux in 2012. That's just awesome backcompat right there.
- andrewem 14y ago
  I really liked the phrase "diagonal shortcuts" from the OP. It brought to mind a physical metaphor for what's meant by software having only orthogonal functions: on a college campus they usually pave orthogonal sidewalks, and inevitably you find diagonal paths where students have cut through the grass and trampled it down to bare dirt. It's ugly in some sense, but it's what everyone wants.
  - dmethvin 14y ago
    Yeah, a literal case of voting with your feet.
rjknight 14y ago
I agree with some of OP's points, but I think he could explain the mechanisms at work better. OP quotes the example of the 35+ command-line switches supported by ls, some of which are clear violations of the 'Unix philosophy'. However, he moves on from this to suggest that users actively dislike using multiple tools to complete a task, and prefer a single all-purpose tool even if it results in that tool becoming very complex. So users would rather pass another flag to ls than learn how to achieve the same effect by piping the output from ls to grep, or sed, or what-have-you. I don't think this is the right way to look at it. Passing in additional flags is the preferred option not because users can't comprehend multiple tools, but because there's no discovery mechanism for finding out that using ls and grep together can achieve a particular outcome. The user's primary reference, the man page, provides no guidance here. Googling might help, as might asking a colleague, but by this point the cost of finding the answer begins to exceed the benefit. What Unix misses right now is a system to help users figure out how to combine operations. Yes, it's obvious once you know how, but for beginners it's really not obvious at all. The point is that there's nothing inherently difficult about doing things the Unix way, it's mostly a problem of documentation and discoverability. There's nothing that can compete with the single man page listing all of the options for a single tool in one place. This problem exists at higher levels too. Often web API documentation fails because it only tells you how to use the API in question, when a tiny bit of extra documentation on how to mash that API up with other popular APIs would be much more beneficial. If you want people to adopt your tool, you really need to tell them how to make it work with their other tools. I guess this is another example of the same principle behind the success of PHP - being 'right' is nowhere near as important as having documentation that tells people how to solve relevant problems.
- lubutu 14y ago
  I think I agree with you, that it's (for those who actually do like Unix) a discoverability problem, but then I think that can be helped by simply reducing the number of command line tools. If you have a small set of tools you know extremely well, the pipelines necessary become immediately obvious whenever you encounter a task... I think part of the problem is that many modern Unix commands are non-orthogonal. It would be interesting to look at ways to improve the discoverability of manpages, though, or at least have some kind of well-structured "if you want to X, you could use Y" index.
blt 14y ago
Linkbait article with hardly any meat on its bones.
- deleted 14y ago
  [deleted]
lsd5you 14y ago
In my opinion this is a superficial view of the problem which misses the fundamental point. The Unix shell tries to be 2 things and suffers as a result. It is a) an interactive shell, optimised for quick navigation and b) a programming api. The programming api should be as simple as possible, but no simpler, everything should be well structured and in long form and with typed arguments. e.g. 'ls' would be 'listDirectory' and it would take arguments as named parameters. It would also be directly bindable into programming languages (rather than messing around with strings to 'execute'. The interactive shell would be a program built on top of the programming api (an api which does everything, not posix), and would contain all the abbreviations and implicit type conversions. This would probably create more choice/fragmentation/innovation than currently exists it would be more practical to deprecate commands and arguments.
adamtj 14y ago
The idea that Unix is or should be a collection of small tools is only a corollary to the real point, which is that Unix is not a machine, but a language. A machine can only do what it has buttons for. A language can express ideas that have yet to be thought. They are greater than the sums of their parts. As with all languages, words get expanded and used in ways they weren't intended. This is good. English is messier than Esperanto, and better for it. Unix is a language where merely speaking an idea makes it reality. We call this magic.
kamaal 14y ago
Unix is basically a REPL for the C programming language. Its more than a operating system, its actually a development environment. And most of the Unix utilities that you see today are sum total of people trying to solve problems they faced repeateadly. There fore what you see is tiny utilities with a lot options each designed to solve both a operating system and a programming problem. That is precisely what helps people to write almost real looking programs by just sewing commands using the pipe. I love the all the three points the author mentions, because together they form a a very powerful platform to work with and solve problems. >>Why does ls have dozens of tangled options? Because users, even Unix users, are not overly fond of the first two points of the Unix philosophy. They don’t want to chain little programs together. They’d rather do more with the tool at hand than put it down to pick up new tools. They do appreciate the ideal of single-purpose tools that work well together, but not too strictly. I don't agree with this. I can tell you that I just love Unix the way it is now. People feel they are messy because,as I said they always keep looking at them as commands and not as small programs that can be used together to solve problems. Most people don't realize this, because they don't know how to use it. It takes a lot of time to mature on the Unix platform. You have to learn what the commands do, then you need to go out and learn how to use them in combinations. Then you need to start learning the Unix way of representing data and transforming data to solve problems. Like always most people don't realize the fact that data is a actually 'Text'. And not learning how to work with Text/Strings/Data generally creates all sorts of other problems. Although projects which are RDBMS heavy don't encounter them usually. But a lot of projects that are not DB heavy really pass through frustrating patches of time trying to figure out easy ways of representing, transforming and working with Data(Text). The most common format which is heavily abused are XML's. Followed by JSON these days. But in my experience if only they had learned how to use a little grep, cat, head, tail, sed, awk and learned to represent their data properly they would save themselves writing several hundreds to thousand lines of code. This problem is especially prevalent in the Java world. They are plagued with writing endless programs that actually do a subset of what sed or awk do. Its sad, but most people express shock and surprise while shown how their 500 line Java program is actually a one line sed command. Or how a simple awk command actually does what they spent a whole day writing. Not understanding Unix forces you just re implement Unix utilities badly! Unix utilities were precisely invented to solve these kind of problems. In fact if you find yourself solving a problem a particular way, then ideally you should use a tool that can be programmed that particular way to solve the problem. A tool like sed is implicitly designed to open files, iterate line by line, provide you with matching, manipulating, counting, printing abilities. It also contains variables to hold data, and do all other kinds of manipulations. If you were to keep writing programs in permutations of logic to perform each of these facilities specifically as needed(Which is pretty common, if your data is in files), you might as well just go ahead and learn sed. Because that will ensure you not just save on writing code, but will also save you time. Now this is just sed, Awk can go further. It can help you do a lot more stuff. It gives you conditionals, more sharpened regexes, better ways of handling field separated data, variables, functions and much more. Combined with stuff like wc, less, cat, tr, xargs, grep, column etc you really get a full fledged programming language(Without even realizing that you have one :)) in which you can hack real fast without worrying about bugs. The thing is 'Text' is actually data. And knowing how to play with Text and learning how to manipulate it well helps you solve problems elegantly, quickly and in a far more easier way than writing a specific program for each task. As a final leap, you mature as a Unix programmer when you learn Perl. Its the King of scripting languages. You master Perl, you begin to truly appreciate the beauty of what it means to be a backend Unix programmer. Its a world whose beauty and power is known to those who program in it. It completes the total Unix equation and suddenly you recieve your enlightenement and realize why despite Unix's messiness it still is the best thing in the world. Worse is Indeed, Better!
zhiel 14y ago
The basic flaw in the Unix philosophy is that the rules 1. and 2. are incorrect. The rules should be as follows: 1. Write libraries that do one thing and do it well. 2. Write libraries to work together. The third rule is correct, because that is what programs essentially are: black boxes to modify input streams to output streams. I think this correction should be followed in the future.
conformal 14y ago
this is a troll with very little substance. summarized as "ls does not follow the unix philosophy", it points out that an example of a tool that doesn't follow the philosophy as close as it could or should. there are exceptions to every rule, what a substantive exposition. ORANGE ALERT, send out the bubble!

50 points by prajjwal 14y ago

udp 14y ago

> Why does ls have dozens of tangled options? Because users, even Unix users, are not overly fond of the first two points of the Unix philosophy. No, ls(1) is flexible because the philosophy is to do one thing and do it well. ls(1) lists information about files. That's what `man ls` says it will do, and whichever flags you pass, that's what it's going to do. Providing a lot of control over how exactly it does that is (imho) doing it well.

betawolf33 14y ago
In a sense, yes. However, some of the things ls does are 'borrowed functionality'. That the displayed listing can be sorted by a flag to ls is a bit of an aberration in the Unix philosophy. We're meant to have a separate tool for sorting - sort. I agree that a certain flexibility is useful in the Unix tools, but a well-founded argument can be made that flexibility costs you simplicity.
- pilif 14y ago
  The fact that ls can produce output that can't be reliably sorted using the sort utility shows that ls is actually doing three things instead of just one: Listing files, sorting the output and then displaying the output in a human readable format. This is not to criticize ls though - I would imagine that no matter how small the operation (well aside of operations executed directly by the cpu), there's still the possibility to break it up into smaller parts. At some point convenience just trumps modularity. ls -lt is just more convenient than a (hypotetical) ls -t | sort | xargs fileinfo | display-fileinfo On a related note: Windows PowerShell comes closer to this ideal by being able to send arbitrary objects between pipes. This still has the side-effect though that single commands become much more awkward to use as they don't produce output that's usable on its own.
  - cturner 14y ago
    I would imagine that no matter how small the operation (well aside of operations executed directly by the cpu), there's still the possibility to break it up into smaller parts. The scripting system could have been designed to make layering easier. Bourne descendants are ugly and awkward. Hence, unix evolved into a situation where ls had lots of features, whereas were there a better scripting system, we might have had a standard environment which you could introspect deeper and extend with more confidence.
    - ori_b 14y ago
      > The scripting system could have been designed to make layering easier. How? sh sucks for anything involved, but I can't think of a way to simplify pipelines much.
      - jcmhn 14y ago
        A line oriented xargs could have been built in from day one. A pair of shell variables honored by commands that let you specify output wrappers or separators would have been nice too. Think "-Q" in gnu ls, but universally supported and user-controlled.
- reddit_clone 14y ago
  How would you make 'sort' do all the things that 'ls' can easily do? How do you sort a multi-column output by 3rd column?. You have to write some scripts for that.
  - elwin 14y ago
    sort -k 3
    - tedunangst 14y ago
      That works for the trivial case of the 3rd column, but not for everything ls can produce. There's no way to make sort work on dates as formatted by ls (which may or may not have years or hours, depending on how old) or sizes (which you may want to pretty print as human readable.) What sort option puts 1.0K, 1.0M, and 1.0G in the right order?
      - betawolf33 14y ago
        While I do appreciate the point that ls is doing sorting that sort would struggle to manage, isn't this an argument for better functionality in sort? Someone pointed out that ls is really doing three things - listing files, sorting them and displaying them. That seems fairly accurate, and indeed it's convenient that it does. But it's not 'doing one thing'. I'm not saying that's necessarily bad (as I just said, it's convenient), but it's not hardline Unix-way purity. For practical reasons like those you raise, it makes sense for ls to handle its own sorting rather than leave it to the sort command, just as it handles its own formatting rather than let the user manage it. But those features _are_ something other than just reporting file listings, whatever the motivation for the feature creep. This convenient flexibility makes ls less single-task, less Unix-adherent but more generally useful on its own.
      - masklinn 14y ago
        > There's no way to make sort work on dates as formatted by ls (which may or may not have years or hours, depending on how old) or sizes (which you may want to pretty print as human readable.) Both of which wouldn't be the job of a "do one thing and do it well" `ls`. That one would display dates as a UNIX timestamp and sizes as bytes. Aside from that, surely if ls can format information it's possible to reverse that formatting.
telemachos 14y ago
> No, ls has dozens of tangled options because the philosophy is to "do one thing and do it well". Do you mean that in order to truly do well its one thing (list information about files), that ls had to incorporate things like these? -o List in long format, but omit the group id. -T When used with the -l (lowercase letter ``ell'') option, display complete time information for the file, including month, day, hour, minute, second, and year. -x The same as -C, except that the multi-column output is produced with entries sorted across, rather than down, the columns. Some degree of feature creep is probably inevitable, but a lot of the common utilities now seem to lean towards "offer a flag for everything" rather than "do a small number of permutations on one thing; create pipelines for the rest". (Note: These three options are all from OSX's BSD-derived ls. GNU's version is almost certainly more complex still.)
- masklinn 14y ago
  > that ls had to incorporate things like these? Or this bunch: -p Write a slash (`/') after each filename if that file is a directory. -r Reverse the order of the sort to get reverse lexicographical order or the oldest entries first (or largest files last, if combined with sort by size -S Sort files by size * -p's information is redundant with the first character of the long format * -S (and -r) are redundant with `sort` (ls -lSr -> ls -l | sort -k5; ls -lS -> ls -l | sort -rk5)
  - lmm 14y ago
    -p is immensely useful. Of course it's redundant in long format; it's for using in short format. -r is also very useful (I use -tr all the time, which I can't immediately see how I'd replace with sort), and while I could replace ls -lS with piping into sort, you seem to be assuming ls will always be outputting one file per line, which is far from its most useful output mode when running it in a terminal.
    - masklinn 14y ago
      > you seem to be assuming ls will always be outputting one file per line No, I am assuming a "do one thing" `ls` would be used in long format (or as inode numbers) whenever it need to be worked with, and would be massaged into form by further commands. For `t`, `r`, `S` and a bunch of others you'd ls, sort and then cut/awk/whatever to format into a "short form". Not that it matters a bit. Also, I didn't claim these weren't useful, I claimed they were redundant. Which they are.
reddit_clone 14y ago
I agree. It is not like 'ls' is predicting weather and telling the fortune. It lists files and directories. It has a zillion options to control its behaviour and output. I see nothing wrong with those options if they will be of use to somebody. I use 4-5 options myself and know them by heart. I do admit that a new user can get overwhelmed if he looks at the man page.
- quesera 14y ago
  I agree. ls is primarily a user-facing utility, and has acquired a ton of display and formatting switches. This is not a bad thing. Some of the display options are superfluous but convenient. Others are awkward, very hard, or impossible to duplicate in a pipeline. New users don't need to be overwhelmed by the man page. -a and -l should be enough for >99% of use cases. ls shows up in shell scripts too, but that's almost always a mistake -- or a dependency on one of the harder-to-duplicate formatting options.

romaniv 14y ago

It’s a lot easier to maintain an orthogonal design when your software isn’t being used. Mentally, I'm translating this to "it's okay to have messy design if your software is used". This is probably the most common excuse for writing bad software. Some things in Unix really should follow Unix philosophy more closely.

raldi 14y ago
I'd rephrase it more like, "Clean design is a means to pleasing users; it's not an end in itself." When is it okay to get messy? When it's a net win for the user experience.
- romaniv 14y ago
  That's exactly what I'm talking about. "Pleasing users" is often used as an excuse for horrible internal design. People cite examples like ls and proceed to conclude that the Unix philosophy is a concept of abstract philosophy not applicable to real life, when they should conclude that there are simply some issues with ls design. Nearly every time I hear "but it's okay to be messy, because it's real life" or "I must do this bad thing to please the users" the person saying it advocated something that is an insignificant short-term gain for a tiny subset of users at the expense of very significant long-term losses for all the users. Internal design usually does not affect end users directly, but it has tremendous indirect effects in terms of performance and reliability, which people do care about. LS is a good example of this. Having some obscure flag might have solved a particular problem in the past, but now everyone has to deal with extra documentation, complexity and bugs.
  - raldi 14y ago
    That may be the case sometimes, but not always. Here are some examples of what I think were good uses of messiness: ls output can be sorted in all sorts of ways with just a few characters of typing, even though it would be cleaner to make people use sort. grep has a flag for counting lines that match, even though wc is supposed to be used for counting lines. rm can remove directories with the right flags, even though rmdir is supposed to be the tool for removing directories. I think those were all net wins for the user, both in the short term and the long term.

dexen 14y ago

http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs follows the Unix philosophy. A lot of legacy has been shed. I can count 13 options to ls, 11 options to sed and just 5 to sed. The standard Plan 9 shell, Rc, is described in mere ~500 lines of manpage, while Bash takes whooping ~5400 lines. Oh, and there is no `dll hell' in P9 :-)

telemachos 14y ago
> A lot of legacy has been shed. An example I remember from the UNIX to Plan 9 command translation page[1]: UNIX Plan 9 head sed 10q [1]: http://www.plan9.bell-labs.com/wiki/plan9/unix_to_plan_9_command_translation/index.html http://www.plan9.bell-labs.com/wiki/plan9/unix_to_plan_9_com...
- DeepDuh 14y ago
  Looking at this list I can't really tell why this is supposed to be better. Shedding complexity ok - but getting rid of many shorthands that are there for a reason? It reminds me of how Java wanted to replace printf with a ridiculously complex string formatting system.
  - sp332 14y ago
    The point is that "head" on your system is not the same as "sed 10q". There are two different codebases. By default head shows the first 10 lines but you can change it with flags etc. Well why bother when you already have sed to do that? head -n 5 sed 5q For most cases, head is not a shortcut. 10 is just a special case.
    - DeepDuh 14y ago
      I agree with that change. I don't with many others.
  - telemachos 14y ago
    Getting rid of good things would be bad, but is sed 10q really "ridiculously complex"?

papsosouid 14y ago

Unix followed the unix philosophy just fine. Looking at current "unix-like" software and the huge amount of bloat and cruft isn't a reasonable way to criticize unix. Unix has been obsolete for decades, and the systems that took over (BSDs, linux, GNU) didn't follow the unix philosophy very much (or at all in the case of GNU).

unimpressive 14y ago
> (or at all in the case of GNU). What did you expect? RMS was a MIT Hacker. Which means that his background was using Lisp, ITS, and Lisp machines.
- papsosouid 14y ago
  I didn't say I expected GNU to follow the unix philosophy. I pointed out that it doesn't, and that saying "unix doesn't follow unix philosophy" while pointing at GNU tools for proof is absurd.

dmethvin 14y ago

> It’s a lot easier to maintain an orthogonal design when your software isn’t being used. Software that gets used becomes less orthogonal and develops diagonal shortcuts. Yep, and that's not necessarily a bad thing. Human language is exactly the same way. Irregular verbs, contractions, and slang are all diagonal shortcuts. There may be a few unneeded adornments to some Unix utilities, I agree. But remember that the `ls -l` that you might have typed in 1980 on a minicomputer running BSD still has the same output as the `ls -l` from a PC running Linux in 2012. That's just awesome backcompat right there.

andrewem 14y ago
I really liked the phrase "diagonal shortcuts" from the OP. It brought to mind a physical metaphor for what's meant by software having only orthogonal functions: on a college campus they usually pave orthogonal sidewalks, and inevitably you find diagonal paths where students have cut through the grass and trampled it down to bare dirt. It's ugly in some sense, but it's what everyone wants.
- dmethvin 14y ago
  Yeah, a literal case of voting with your feet.

rjknight 14y ago

I agree with some of OP's points, but I think he could explain the mechanisms at work better. OP quotes the example of the 35+ command-line switches supported by ls, some of which are clear violations of the 'Unix philosophy'. However, he moves on from this to suggest that users actively dislike using multiple tools to complete a task, and prefer a single all-purpose tool even if it results in that tool becoming very complex. So users would rather pass another flag to ls than learn how to achieve the same effect by piping the output from ls to grep, or sed, or what-have-you. I don't think this is the right way to look at it. Passing in additional flags is the preferred option not because users can't comprehend multiple tools, but because there's no discovery mechanism for finding out that using ls and grep together can achieve a particular outcome. The user's primary reference, the man page, provides no guidance here. Googling might help, as might asking a colleague, but by this point the cost of finding the answer begins to exceed the benefit. What Unix misses right now is a system to help users figure out how to combine operations. Yes, it's obvious once you know how, but for beginners it's really not obvious at all. The point is that there's nothing inherently difficult about doing things the Unix way, it's mostly a problem of documentation and discoverability. There's nothing that can compete with the single man page listing all of the options for a single tool in one place. This problem exists at higher levels too. Often web API documentation fails because it only tells you how to use the API in question, when a tiny bit of extra documentation on how to mash that API up with other popular APIs would be much more beneficial. If you want people to adopt your tool, you really need to tell them how to make it work with their other tools. I guess this is another example of the same principle behind the success of PHP - being 'right' is nowhere near as important as having documentation that tells people how to solve relevant problems.

lubutu 14y ago
I think I agree with you, that it's (for those who actually do like Unix) a discoverability problem, but then I think that can be helped by simply reducing the number of command line tools. If you have a small set of tools you know extremely well, the pipelines necessary become immediately obvious whenever you encounter a task... I think part of the problem is that many modern Unix commands are non-orthogonal. It would be interesting to look at ways to improve the discoverability of manpages, though, or at least have some kind of well-structured "if you want to X, you could use Y" index.

blt 14y ago

Linkbait article with hardly any meat on its bones.

deleted 14y ago
[deleted]

lsd5you 14y ago

In my opinion this is a superficial view of the problem which misses the fundamental point. The Unix shell tries to be 2 things and suffers as a result. It is a) an interactive shell, optimised for quick navigation and b) a programming api. The programming api should be as simple as possible, but no simpler, everything should be well structured and in long form and with typed arguments. e.g. 'ls' would be 'listDirectory' and it would take arguments as named parameters. It would also be directly bindable into programming languages (rather than messing around with strings to 'execute'. The interactive shell would be a program built on top of the programming api (an api which does everything, not posix), and would contain all the abbreviations and implicit type conversions. This would probably create more choice/fragmentation/innovation than currently exists it would be more practical to deprecate commands and arguments.

adamtj 14y ago

The idea that Unix is or should be a collection of small tools is only a corollary to the real point, which is that Unix is not a machine, but a language. A machine can only do what it has buttons for. A language can express ideas that have yet to be thought. They are greater than the sums of their parts. As with all languages, words get expanded and used in ways they weren't intended. This is good. English is messier than Esperanto, and better for it. Unix is a language where merely speaking an idea makes it reality. We call this magic.

kamaal 14y ago

Unix is basically a REPL for the C programming language. Its more than a operating system, its actually a development environment. And most of the Unix utilities that you see today are sum total of people trying to solve problems they faced repeateadly. There fore what you see is tiny utilities with a lot options each designed to solve both a operating system and a programming problem. That is precisely what helps people to write almost real looking programs by just sewing commands using the pipe. I love the all the three points the author mentions, because together they form a a very powerful platform to work with and solve problems. >>Why does ls have dozens of tangled options? Because users, even Unix users, are not overly fond of the first two points of the Unix philosophy. They don’t want to chain little programs together. They’d rather do more with the tool at hand than put it down to pick up new tools. They do appreciate the ideal of single-purpose tools that work well together, but not too strictly. I don't agree with this. I can tell you that I just love Unix the way it is now. People feel they are messy because,as I said they always keep looking at them as commands and not as small programs that can be used together to solve problems. Most people don't realize this, because they don't know how to use it. It takes a lot of time to mature on the Unix platform. You have to learn what the commands do, then you need to go out and learn how to use them in combinations. Then you need to start learning the Unix way of representing data and transforming data to solve problems. Like always most people don't realize the fact that data is a actually 'Text'. And not learning how to work with Text/Strings/Data generally creates all sorts of other problems. Although projects which are RDBMS heavy don't encounter them usually. But a lot of projects that are not DB heavy really pass through frustrating patches of time trying to figure out easy ways of representing, transforming and working with Data(Text). The most common format which is heavily abused are XML's. Followed by JSON these days. But in my experience if only they had learned how to use a little grep, cat, head, tail, sed, awk and learned to represent their data properly they would save themselves writing several hundreds to thousand lines of code. This problem is especially prevalent in the Java world. They are plagued with writing endless programs that actually do a subset of what sed or awk do. Its sad, but most people express shock and surprise while shown how their 500 line Java program is actually a one line sed command. Or how a simple awk command actually does what they spent a whole day writing. Not understanding Unix forces you just re implement Unix utilities badly! Unix utilities were precisely invented to solve these kind of problems. In fact if you find yourself solving a problem a particular way, then ideally you should use a tool that can be programmed that particular way to solve the problem. A tool like sed is implicitly designed to open files, iterate line by line, provide you with matching, manipulating, counting, printing abilities. It also contains variables to hold data, and do all other kinds of manipulations. If you were to keep writing programs in permutations of logic to perform each of these facilities specifically as needed(Which is pretty common, if your data is in files), you might as well just go ahead and learn sed. Because that will ensure you not just save on writing code, but will also save you time. Now this is just sed, Awk can go further. It can help you do a lot more stuff. It gives you conditionals, more sharpened regexes, better ways of handling field separated data, variables, functions and much more. Combined with stuff like wc, less, cat, tr, xargs, grep, column etc you really get a full fledged programming language(Without even realizing that you have one :)) in which you can hack real fast without worrying about bugs. The thing is 'Text' is actually data. And knowing how to play with Text and learning how to manipulate it well helps you solve problems elegantly, quickly and in a far more easier way than writing a specific program for each task. As a final leap, you mature as a Unix programmer when you learn Perl. Its the King of scripting languages. You master Perl, you begin to truly appreciate the beauty of what it means to be a backend Unix programmer. Its a world whose beauty and power is known to those who program in it. It completes the total Unix equation and suddenly you recieve your enlightenement and realize why despite Unix's messiness it still is the best thing in the world. Worse is Indeed, Better!

zhiel 14y ago

The basic flaw in the Unix philosophy is that the rules 1. and 2. are incorrect. The rules should be as follows: 1. Write libraries that do one thing and do it well. 2. Write libraries to work together. The third rule is correct, because that is what programs essentially are: black boxes to modify input streams to output streams. I think this correction should be followed in the future.

conformal 14y ago

this is a troll with very little substance. summarized as "ls does not follow the unix philosophy", it points out that an example of a tool that doesn't follow the philosophy as close as it could or should. there are exceptions to every rule, what a substantive exposition. ORANGE ALERT, send out the bubble!