5 ms·

The UNIX Way

22 points by techdog 17y ago

drhowarddrfine 17y ago
It amazes me that The Way of Unix is not known and understood, and most certainly applied, by everyone everywhere.
- weaksauce 17y ago
  If you amend that statement to "..., and most certainly applied, by everyone where it makes sense." I would be 100% on board. Engineering a solution is all about tradeoffs and sometimes the tradeoff needs to break a long standing convention to be fast enough / have a cohesive UI / whatever. I love the UNIX way but it is not the solution for Life, the Universe, and Everything either. That is 42.
wwalker3 17y ago
These rules don't seem to apply to the modern age very well. The rules "Avoid captive user interfaces" and "Make every program a filter" mean that you can't write a text editor. The rule "Store data in flat text files" means you can't write a (reasonable) MP3 player or video editing program. "Build a prototype as soon as possible" is pretty general, but most of these rules are just good guidelines for writing command shells and compilers -- not coincidentally, two of the main concerns of early Unix developers.
- makecheck 17y ago
  But note that the overarching guideline is to structure programs in ways that are generally useful (particularly to other programmers). I don't think the point is that there should be no bigger programs, but rather that those programs should take advantage of as many small parts as possible. An MP3 player could have done this. For example, is there a small program that (only) extracts MP3 tags, to allow song lists to be easily constructed/sorted based on attributes such as Artist? Why should a giant GUI be the only way to perform these tasks? When one believes that the GUI is the only way, suddenly it seems "necessary" for the GUI to have its own scripting language to allow automation of these kinds of tasks. Instead, iTunes could run the "create song list based on these tags" program whenever it needs to perform that function, and use the results to update its GUI.
- SwellJoe 17y ago
  s/rule/guideline/ and maybe you'll be able to see the forest past the trees. The rules "Avoid captive user interfaces" and "Make every program a filter" mean that you can't write a text editor. Not true. Some text editors are of the UNIX nature, and some are not. But, the fact is that you probably shouldn't be writing a text editor today. Editing text is a solved problem, and you'll almost certainly do it worse than emacs or vim if you try. The rule "Store data in flat text files" means you can't write a (reasonable) MP3 player or video editing program. Sure you can. Your playlists can be flat text. Your video queue list can be flat text. Your preferences can be flat text. MP3s and video can also be processed in a filter style (and often are conceptually so within the UI). Only problem is compressed data isn't in a form that can be filtered without first decompressing it. But, I regularly batch process music and audio files. If I did a lot of video work, I'm sure I'd also do the same with video. Your program can certainly be a filter for those types of files. Sox, lame, and the ImageMagick convert utility are good examples of media programs that embrace the UNIX philosophy. The point is that you shouldn't use a database if you don't need the things that databases provide beyond storage of blobs of data. Don't over-architect your file formats, don't obfuscate them, don't make it harder for users to use your code in ways you didn't envision. This is the UNIX philosophy...not that you aren't allowed to operate on non-text data. Programs have been working with messy real world problems since before UNIX and the UNIX philosophy existed. The need to interact with non-text data hasn't sprung into existence in the past ten years. Before there were MP3 and video, there was typesetting, and scientific data analysis, and moon landing calculations, and neural networks, and telephone switching, and so on and so forth. Programming in the UNIX philosophy is always possible, and, as far as I can see from my many years of software development, always wise. but most of these rules are just good guidelines for writing command shells and compilers -- not coincidentally, two of the main concerns of early Unix developers Every single problem of note that I can think of would be well-served by breaking it into reusable pieces. Sometimes this just means libraries, rather than small standalone programs, particularly on the desktop...but sometimes separate programs that happen to be used by a "main" program are the right way to go. Things like video and audio compression can be, and often are, performed by separate programs. This separation is a win on many levels...on a multi-core machine, the new process can have its own resources, while you continue to work away on your media. It can be upgraded or maintained at a specific version more readily. It can be used by other programs or in a batch mode from a script or batch compression utility. I think it takes someone who doesn't use UNIX on a regular basis (actually uses UNIX; not merely a GUI shell on top) to think these concepts are outdated and not generally applicable.
makecheck 17y ago
This blog post contains mostly a small list of design principles. But those ideas are from ESR's book "The Art of UNIX Programming", where they are expanded upon in great detail, with excellent examples. I am someone who practices the Unix philosophy on a regular basis, and cringes when I see how twisted and inelegant Windows has become (even on the programming side). I consider this book a must-read for people who know only Windows. It is also an interesting book for those who know Unix already.
- kqr2 17y ago
  The blog post attributes the list of design principles to Mike Gancarz's "Unix Philosophy" (1994) whereas ESR's book "The Art of UNIX Programming" was published in 2003. I'm sure those ideas had been floating around in the community for a long while but was written down more formally in those books.
- nailer 17y ago
  'Text as the universal interface' clashes quite badly with separating content from presentation. This means you're forever relying on regex's to rip data out of other applications, and relying on those other applications not to change their output. Compare finding processes starting within the last hour on Unix and Windows: ps | grep 'regex representing the current amount of columns output by ps where start time is less than 2 hours' Windows (ps is an alias for get-process): ps | where COLUMN <= value The latter is much more robust. As Powershell gets more cmdlets it's going to become increasingly useful. That said, I prefer Unix for other reasons.
  - cagey 17y ago
    But are these 'additional cmdlets' going to be available in a default O.S. install? If not, will they be automatically brought in by Windows Updater?
  - makecheck 17y ago
    True, though "ps" isn't the best example, as it's one of the least portable commands I've ever seen. :) Its output is not even consistent from one Unix variant to another. I kind of hope most Unix variants follow the /proc filesystem model of Linux. It basically cleans up these kinds of queries considerably.
leif 17y ago
Too much of desktop linux has strayed from his #5, and it bothers me to no end (yes this means you, hal and dbus and gconf).
- nailer 17y ago
  GConf is just text files. Open your .gconf folder.
  - leif 17y ago
    Just xml files in a horribly tall folder hierarchy, you mean. Hand-editing xml (especially with a foreign schema) is retarded.

The UNIX Way

22 points by techdog 17y ago

drhowarddrfine 17y ago

It amazes me that The Way of Unix is not known and understood, and most certainly applied, by everyone everywhere.

weaksauce 17y ago
If you amend that statement to "..., and most certainly applied, by everyone where it makes sense." I would be 100% on board. Engineering a solution is all about tradeoffs and sometimes the tradeoff needs to break a long standing convention to be fast enough / have a cohesive UI / whatever. I love the UNIX way but it is not the solution for Life, the Universe, and Everything either. That is 42.

wwalker3 17y ago

These rules don't seem to apply to the modern age very well. The rules "Avoid captive user interfaces" and "Make every program a filter" mean that you can't write a text editor. The rule "Store data in flat text files" means you can't write a (reasonable) MP3 player or video editing program. "Build a prototype as soon as possible" is pretty general, but most of these rules are just good guidelines for writing command shells and compilers -- not coincidentally, two of the main concerns of early Unix developers.

makecheck 17y ago
But note that the overarching guideline is to structure programs in ways that are generally useful (particularly to other programmers). I don't think the point is that there should be no bigger programs, but rather that those programs should take advantage of as many small parts as possible. An MP3 player could have done this. For example, is there a small program that (only) extracts MP3 tags, to allow song lists to be easily constructed/sorted based on attributes such as Artist? Why should a giant GUI be the only way to perform these tasks? When one believes that the GUI is the only way, suddenly it seems "necessary" for the GUI to have its own scripting language to allow automation of these kinds of tasks. Instead, iTunes could run the "create song list based on these tags" program whenever it needs to perform that function, and use the results to update its GUI.
SwellJoe 17y ago
s/rule/guideline/ and maybe you'll be able to see the forest past the trees. The rules "Avoid captive user interfaces" and "Make every program a filter" mean that you can't write a text editor. Not true. Some text editors are of the UNIX nature, and some are not. But, the fact is that you probably shouldn't be writing a text editor today. Editing text is a solved problem, and you'll almost certainly do it worse than emacs or vim if you try. The rule "Store data in flat text files" means you can't write a (reasonable) MP3 player or video editing program. Sure you can. Your playlists can be flat text. Your video queue list can be flat text. Your preferences can be flat text. MP3s and video can also be processed in a filter style (and often are conceptually so within the UI). Only problem is compressed data isn't in a form that can be filtered without first decompressing it. But, I regularly batch process music and audio files. If I did a lot of video work, I'm sure I'd also do the same with video. Your program can certainly be a filter for those types of files. Sox, lame, and the ImageMagick convert utility are good examples of media programs that embrace the UNIX philosophy. The point is that you shouldn't use a database if you don't need the things that databases provide beyond storage of blobs of data. Don't over-architect your file formats, don't obfuscate them, don't make it harder for users to use your code in ways you didn't envision. This is the UNIX philosophy...not that you aren't allowed to operate on non-text data. Programs have been working with messy real world problems since before UNIX and the UNIX philosophy existed. The need to interact with non-text data hasn't sprung into existence in the past ten years. Before there were MP3 and video, there was typesetting, and scientific data analysis, and moon landing calculations, and neural networks, and telephone switching, and so on and so forth. Programming in the UNIX philosophy is always possible, and, as far as I can see from my many years of software development, always wise. but most of these rules are just good guidelines for writing command shells and compilers -- not coincidentally, two of the main concerns of early Unix developers Every single problem of note that I can think of would be well-served by breaking it into reusable pieces. Sometimes this just means libraries, rather than small standalone programs, particularly on the desktop...but sometimes separate programs that happen to be used by a "main" program are the right way to go. Things like video and audio compression can be, and often are, performed by separate programs. This separation is a win on many levels...on a multi-core machine, the new process can have its own resources, while you continue to work away on your media. It can be upgraded or maintained at a specific version more readily. It can be used by other programs or in a batch mode from a script or batch compression utility. I think it takes someone who doesn't use UNIX on a regular basis (actually uses UNIX; not merely a GUI shell on top) to think these concepts are outdated and not generally applicable.

makecheck 17y ago

This blog post contains mostly a small list of design principles. But those ideas are from ESR's book "The Art of UNIX Programming", where they are expanded upon in great detail, with excellent examples. I am someone who practices the Unix philosophy on a regular basis, and cringes when I see how twisted and inelegant Windows has become (even on the programming side). I consider this book a must-read for people who know only Windows. It is also an interesting book for those who know Unix already.

kqr2 17y ago
The blog post attributes the list of design principles to Mike Gancarz's "Unix Philosophy" (1994) whereas ESR's book "The Art of UNIX Programming" was published in 2003. I'm sure those ideas had been floating around in the community for a long while but was written down more formally in those books.
nailer 17y ago
'Text as the universal interface' clashes quite badly with separating content from presentation. This means you're forever relying on regex's to rip data out of other applications, and relying on those other applications not to change their output. Compare finding processes starting within the last hour on Unix and Windows: ps | grep 'regex representing the current amount of columns output by ps where start time is less than 2 hours' Windows (ps is an alias for get-process): ps | where COLUMN <= value The latter is much more robust. As Powershell gets more cmdlets it's going to become increasingly useful. That said, I prefer Unix for other reasons.
- cagey 17y ago
  But are these 'additional cmdlets' going to be available in a default O.S. install? If not, will they be automatically brought in by Windows Updater?
- makecheck 17y ago
  True, though "ps" isn't the best example, as it's one of the least portable commands I've ever seen. :) Its output is not even consistent from one Unix variant to another. I kind of hope most Unix variants follow the /proc filesystem model of Linux. It basically cleans up these kinds of queries considerably.

leif 17y ago

Too much of desktop linux has strayed from his #5, and it bothers me to no end (yes this means you, hal and dbus and gconf).

nailer 17y ago
GConf is just text files. Open your .gconf folder.
- leif 17y ago
  Just xml files in a horribly tall folder hierarchy, you mean. Hand-editing xml (especially with a foreign schema) is retarded.