7 ms·

This whole article is a bunch of strawman arguments. It points out historical mistakes by unix developers, but those mistakes aren't inherent to or a result of

This whole article is a bunch of strawman arguments. It points out historical mistakes by unix developers, but those mistakes aren't inherent to or a result of the Unix Philosophy: (1) Write programs that do one thing and do it well. (2) Write programs to work together. (3) Write programs to handle text streams, because that is a universal interface.

TeMPOraL 10y ago
The arguments in the article indeed don't have much to say about Unix Philosophy per se - they're just a list of various fuckups and idiocies Unix accumulated for some reasons or others. As for Unix Philosophy, the point (3) in your summary is something that's a) dumb, and b) back when it was created, they already had better solutions. Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc. Think of it this way: if, per Unix Philosophy (points (1) and (2) of your summary), programs are kind of like function calls, and your OS is kind of like the running image, then (3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed. I kind of understand how it is people got used to it and don't see a problem anymore (Stockholm syndrome). What shocked me was learning that back before UNIX they already knew how to do it better, but UNIX just ignored it.
- em3rgent0rdr 10y ago
  > "The arguments in the article indeed don't have much to say about Unix Philosophy per se - they're just a list of various fuckups and idiocies Unix accumulated for some reasons or others." Right. The title should have been reflective of that "Various idiocies Unix has accumulated to this day" but since the article mentions Unix Philosophy, my point is that the article should have criticised the philosophy and not the practice. > "Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc." But this has actually proved to be very useful as it provided a standard medium of communication between programs that is both human readable and computer understandable. And ahead of its time since it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded. > "(3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed." That may be a performance downside in some cases, but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits.
  - TeMPOraL 10y ago
    > And ahead of its time It wasn't ahead of its time. By the time Unix was created, people were already aware of the benefits of structured data. > it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded. That's orthogonal to the issue. The simple solution to Unix problems would be to put a standard parser for JSON/SEXP/whatever into libc or OS libraries and have people use it for stdin/stdout communication. This can still take advantage of multiprocessor systems and whatnot, with an added benefit of program authors not having to each write their own buggy parser anymore. > but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits. I'd say it's exactly the opposite. Unstructured text is not an universally-agreeable format. In fact, it's non-agreeable, since anyone can output anything however they like (and they do), and as a user you're forced to transform data from one program into another via more ad-hoc parsers, usually written in form of sed, awk or Perl invocations. You lose time doing that, each of those parsing steps introduces vulnerabilities, and the whole thing will eventually fall apart anyway because of million reasons that can fuck up the output of Unix commands, including things like your system distribution and your locale settings. As an example of what I'm talking about, imagine that your "ls" invocation would return a list of named rows in some structured format, instead of an ASCII table. E.g. ((:columns :type :permissions :no-links :owner :group :size :modification-time :name) (:data (:directory 775 8 temporal temporal 4096 1488506415 ".git") (:file 664 1 temporal temporal 4 1488506415 ".gitignore") ... (:file 755 1 temporal temporal 69337136 1488506415 "hju"))) With such a format you could trivially issue commands like: ls | filter ':modification-time < 1 month ago' | cp --to '/home/otheruser/oldfiles/' find :name LIKE ".git%" | select (:name :permissions) | format-list > git_perms_audit.log Hell, you could display the usual Unix "ls -la" table for the user trivially too, but you wouldn't have to parse it manually. BTW. This is exactly what PowerShell does (except it sends .NET objects), which is why it's awesome.
    - deleted 10y ago
      [deleted]
    - 10y ago
- astrange 10y ago
  > (3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed. I think this is great. There's slightly more principled ways to do it, but having to convert everything to one single format at the end of the day keeps you humble. Let's go back to the previous decade's Hacker News: http://wiki.c2.com/?AlternateHardAndSoftLayers http://wiki.c2.com/?AlternateHardAndSoftLayers
  - TeMPOraL 10y ago
    It's not converting to "one single format", it's converting to "any and all possible formats", because with unstructured text, you're literally throwing away the structure and semantics inherent in the data, instead relying on users to glue things together with ad-hoc parsers.
    - jstimpfle 10y ago
      Please stop spreading bs. It's not throwing away structure. Piping text doesn't even preclude sexps. It's just that they are seldom needed. Simpler encodings like space-separated are sufficient for many use cases, and better for interoperation. It's misguided and inefficient to encode everything in the same way. Would you prefer to have your JPG or MP4 encoded in sexps? And I say that as someone who is working on a serialization format for relational databases.
      - TeMPOraL 10y ago
        Piping isn't the culprit. It's what you pipe that is, and Unix Philosophy says "pipe whatever the hell you want, and let the users upstream sort it out". It's not about encoding everything in exactly the same way. It's about providing the basic, shared protocol for representing structure. With typical Unix tools, you don't have "simpler encodings", you have no encoding at all. Each tool outputs whatever its particular author felt like (and this changes between Unix systems), each tool parses things in whatever way its author felt like, and as a user your job is to keep transforming unstructured blobs of text to glue them together.

TeMPOraL 10y ago

The arguments in the article indeed don't have much to say about Unix Philosophy per se - they're just a list of various fuckups and idiocies Unix accumulated for some reasons or others. As for Unix Philosophy, the point (3) in your summary is something that's a) dumb, and b) back when it was created, they already had better solutions. Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc. Think of it this way: if, per Unix Philosophy (points (1) and (2) of your summary), programs are kind of like function calls, and your OS is kind of like the running image, then (3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed. I kind of understand how it is people got used to it and don't see a problem anymore (Stockholm syndrome). What shocked me was learning that back before UNIX they already knew how to do it better, but UNIX just ignored it.

em3rgent0rdr 10y ago
> "The arguments in the article indeed don't have much to say about Unix Philosophy per se - they're just a list of various fuckups and idiocies Unix accumulated for some reasons or others." Right. The title should have been reflective of that "Various idiocies Unix has accumulated to this day" but since the article mentions Unix Philosophy, my point is that the article should have criticised the philosophy and not the practice. > "Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc." But this has actually proved to be very useful as it provided a standard medium of communication between programs that is both human readable and computer understandable. And ahead of its time since it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded. > "(3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed." That may be a performance downside in some cases, but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits.
- TeMPOraL 10y ago
  > And ahead of its time It wasn't ahead of its time. By the time Unix was created, people were already aware of the benefits of structured data. > it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded. That's orthogonal to the issue. The simple solution to Unix problems would be to put a standard parser for JSON/SEXP/whatever into libc or OS libraries and have people use it for stdin/stdout communication. This can still take advantage of multiprocessor systems and whatnot, with an added benefit of program authors not having to each write their own buggy parser anymore. > but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits. I'd say it's exactly the opposite. Unstructured text is not an universally-agreeable format. In fact, it's non-agreeable, since anyone can output anything however they like (and they do), and as a user you're forced to transform data from one program into another via more ad-hoc parsers, usually written in form of sed, awk or Perl invocations. You lose time doing that, each of those parsing steps introduces vulnerabilities, and the whole thing will eventually fall apart anyway because of million reasons that can fuck up the output of Unix commands, including things like your system distribution and your locale settings. As an example of what I'm talking about, imagine that your "ls" invocation would return a list of named rows in some structured format, instead of an ASCII table. E.g. ((:columns :type :permissions :no-links :owner :group :size :modification-time :name) (:data (:directory 775 8 temporal temporal 4096 1488506415 ".git") (:file 664 1 temporal temporal 4 1488506415 ".gitignore") ... (:file 755 1 temporal temporal 69337136 1488506415 "hju"))) With such a format you could trivially issue commands like: ls | filter ':modification-time < 1 month ago' | cp --to '/home/otheruser/oldfiles/' find :name LIKE ".git%" | select (:name :permissions) | format-list > git_perms_audit.log Hell, you could display the usual Unix "ls -la" table for the user trivially too, but you wouldn't have to parse it manually. BTW. This is exactly what PowerShell does (except it sends .NET objects), which is why it's awesome.
  - deleted 10y ago
    [deleted]
  - 10y ago
astrange 10y ago
> (3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed. I think this is great. There's slightly more principled ways to do it, but having to convert everything to one single format at the end of the day keeps you humble. Let's go back to the previous decade's Hacker News: http://wiki.c2.com/?AlternateHardAndSoftLayers http://wiki.c2.com/?AlternateHardAndSoftLayers
- TeMPOraL 10y ago
  It's not converting to "one single format", it's converting to "any and all possible formats", because with unstructured text, you're literally throwing away the structure and semantics inherent in the data, instead relying on users to glue things together with ad-hoc parsers.
  - jstimpfle 10y ago
    Please stop spreading bs. It's not throwing away structure. Piping text doesn't even preclude sexps. It's just that they are seldom needed. Simpler encodings like space-separated are sufficient for many use cases, and better for interoperation. It's misguided and inefficient to encode everything in the same way. Would you prefer to have your JPG or MP4 encoded in sexps? And I say that as someone who is working on a serialization format for relational databases.
    - TeMPOraL 10y ago
      Piping isn't the culprit. It's what you pipe that is, and Unix Philosophy says "pipe whatever the hell you want, and let the users upstream sort it out". It's not about encoding everything in exactly the same way. It's about providing the basic, shared protocol for representing structure. With typical Unix tools, you don't have "simpler encodings", you have no encoding at all. Each tool outputs whatever its particular author felt like (and this changes between Unix systems), each tool parses things in whatever way its author felt like, and as a user your job is to keep transforming unstructured blobs of text to glue them together.