9 ms·

> This would be a great opportunity to avoid the classic Unix mistake of using text files as the lowest common denominator everywhere There are a great many wh

> This would be a great opportunity to avoid the classic Unix mistake of using text files as the lowest common denominator everywhere There are a great many who do not consider this to be a mistake. The fact that after half a century, Unix-like operating systems and programming environments are de rigueur is plain enough evidence of this. I can't think of much else in the whole field of computing that has had this kind of staying power over such a long period of time, excepting perhaps the idea of binary digital computers themselves. Fun history note: Part of the reason the Unix userspace and development chain lean so heavily on text processing is because its inventors were tasked with designing a system for more powerful tools for the writing and production of technical documentation. But they were dyed-in-the-wool systems and programming nerds so stand-alone documentation tools were not very interesting. So their solution was to invent a new OS, borrowing the features they liked from other OSes they had been exposed to, under the guise of delivering a documentation production system. If it turned out that this Unix thing they came up with happened to be an excellent programming environment as well, that was just a nice bonus as far as management was to be informed. One of the greatest hacks of computing history, if you ask me. Edit to add: As others have pointed out, you are hardly restricted to piping around text. And if you do, the power of text is that you can impose arbitrary structure on top of it, e.g. tabular data, shell scripts, XML, YAML, JSON, etc. Saying it's too bad Unix abstracts everything as text files is like saying it's too bad microprocessors abstract everything down to either a one or a zero when far more interesting number exist.

deleted 6y ago
[deleted]
pwdisswordfish2 6y ago
Wasn't UNIX originally used to format patent applications at Bell Labs/AT&T? That was purportedly the selling point of UNIX to the business people. When someone suggests using something other than "text" (usually it's some binary format, I consider S-Expressions to be text), I always wonder, why is this person writing their suggestion using text? Then we have the suggestions that "everything is a file" is also a "mistake". That's great but if a "file" is not a good enough abstraction then what is?
- wtetzner 6y ago
  > When someone suggests using something other than "text" (usually it's some binary format, I consider S-Expressions to be text), I always wonder, why is this person writing their suggestion using text? Of course Unix actually operations on streams of bytes, which often represent text, but not always.
- megameter 6y ago
  Text is a human language. Computers do not really run on text. You can't talk to Unix like it's the bridge computer in Star Trek. No, Unix runs on bytes that sort of look like text because we have encoded and rendered it that way. What computers need most for structured processing are data types. At instruction set level we had a period of experimentation with byte and word lengths which was still in effect when Unix was born. At that time we also had experimentation with text encodings such as EBCDIC and the various ASCII codepages. Since then we have pretty much settled on standards based on multiples of an 8-bit byte, and on a common text encoding - UTF-8. At the time of Unix's inception the byte stream was a relatively bold move because it enforced variable-length encodings of bytes as the building block of interoperability, versus some kind of static record. Now it is mundane, and the world has moved on to always parsing the bytes for some other purpose. We desire standards that encode data containers, strings and numeric primitives, so that we do less parsing. XML and S-expressions offer two methods of encoding hierarchies of strings. More recently, JSON has become a very common encoding. It encodes strings, numeric values, and two types of hierarchical containers(key-value and array). We also have SQL databases as an older example, which encode various primitives and a variety of tabular data relationships. With each data language we get one or more associated query languages to express how we select data. Unix's handling of files is incomplete because of the ambiguity of queries on files. We rely heavily on relationships between directories and on glob syntax to perform selection in a Unix filesystem. Unix paths and glob syntax are not really a byte stream or a file, but they are most certainly part of Unix, and you would be suffering in short order without those concepts. Yet Unix does not respect the existence of its own query language, and never declares a basic system representation for queries and their results akin to the results of a SQL query. Instead we parse the query as bytes and output the results of queries to bytes and parse those bytes, which means there is no guarantee of support at any of the boundaries where your program touches paths. As long as you only deal with one file at a time you don't have an issue, but at scale, the way we work with our applications needs more complex selections on files; the workaround is to "bottle up" the data into one file that has some other structure, which then necessitates special tools to move around the data. And if you look at what people complain about when working with the shell, the behavior of file selection is a major complaint, and adds a lot of complexity.
seagreen 6y ago
> The fact that after half a century, Unix-like operating systems and programming environments are de rigueur is plain enough evidence of this. Unfortunately Unix blew a 28-3 lead against Netscape Navigator and now 99% of people spend their time in web browsers instead. I think this speaks to the weakness of the experience using Unix, not its strength.
- pantalaimon 6y ago
  > Unfortunately Unix blew a 28-3 lead against Netscape Navigator and now 99% of people spend their time in web browsers instead. There might be exceptions, but people generally don't develop software in their browser.
  - seagreen 6y ago
    Thanks for the serious response to a somewhat snarky comment. I see software development as only one among the many things a computing system should enable. It's definitely an important one. But computers should enable creativity and power for all kinds of tasks, not just programming. It's sad to me that people in all these different fields besides ours get stuck with webapps. I understand why they use them, they're very convenient. But it would have been great if we could have provided them an even better OS/desktop environment so that they never would have had to switch to such an unempowering tool. Additionally, programmers being one of the last holdouts of actually engaging with Unix-as-a-way-of-doing-things is a precarious place to be. Microsoft/GitHub is on full "embrace" mode again, and it looks like they're going to make another run at slurping everyone into their ecosystem. Same with tools like Repl.it. It seems to me that the proportion of people who are engaged with the Unix way of doing things (piping text around, etc.) is shrinking, not growing. EDIT: To get myself back on topic, for these reasons I see people experimenting with new (and crucially local) environments as really exciting. Traditional Unix environments don't need to be the last word, we can do better.
    - linguae 6y ago
      Apple went down a very interesting path during the interregnum years from 1985 through 1996. It was a beacon of creativity where some of its engineers worked on projects that offered new visions of personal computing that empowered the user without requiring them to be full-fledged programmers. HyperCard and AppleScript are examples of such projects, and the Knowledge Navigator, while not an actual product, was one of Apple's visions. I also find OpenDoc to be quite intriguing; OpenDoc would have introduced component-based GUI software to the mass public. Users could perform tasks by mixing-and-matching components and applying them to a document. It would have been really cool to have seen Copland and Gershwin in its completed forms, but sadly as creative Apple was in the 1990s, its management was a mess until Steve Jobs returned, and so Copland never saw the light of day other than a few developer releases. I am grateful for the work that Steve Jobs did at NeXT and for his amazing work returning to Apple, saving it from possible bankruptcy, and making it the juggernaut that it is today. I've long been a fan of Mac OS X. However, parts of me miss the Apple of the 1980s and 1990s, where the engineers and researchers there explored ideas of how to make personal computing better. I feel that the personal computing experience for desktops have been stuck in a rut for nearly 20 years, with some aspects actually degrading rather than improving. But projects such as Snakeware give me hope that we'll see desktop computing move forward. I'm actually working on a side project where I'm creating a Common Lisp-based desktop environment inspired by Lisp machines and OpenDoc that can run on Linux and the BSDs, but I'm in the design phase and I'm far from writing any code (I've been studying up on graphics programming since part of this project involves writing a new GUI toolkit). There is still a lot of room for innovation on the desktop, and as you said, Unix isn't the last word, even for programming.
franga2000 6y ago
Piping data from one command to the next is one of those things that I love showing to new users and it gets them all excited, only to then have to disappoint them when they think of a cool idea because they'd need to learn regex, sed and awk first to do anything non-trivial. It's a shame Powershell is a disaster, because having a structured-by-default pipeline would've really bridged the gap between power users and programmers.
- majkinetor 6y ago
  > It's a shame Powershell is a disaster Yeah, lets get back to bash forever :S
  - franga2000 6y ago
    Bash is a good shell with a fairly unimpressive pipeline. Powershell is bad shell with a good pipeline. For me, as a programmer, the shell part of a shell is more important, as I can do the rest in a more powerful language. Powershell is too verbose for shell usage (see New-Item), brings nothing new to the table in terms of scripting and is ultimately not worth using just for the better pipeline. (another commenter pointed me to Nushell, which seems to be a very good middle ground, and I've heard of a few more similar efforts, so we hopefully won't be stuck with just bash for too long)
    - majkinetor 6y ago
      Oh, here we go again with verbosity like aliases are non existent. For realz, New-Item is ni for example or mkdir or whatever... > as I can do the rest in a more powerful language Powershell is like repl for dotNet. You can't really get more powerfull. > brings nothing new to the table in terms of scripting Are you serious ? I guess somebody has a lot of learning to do. > (another commenter pointed me to Nushell, which seems to be a very good middle ground, and I've heard of a few more similar efforts, so we hopefully won't be stuck with just bash for too long) We were on that road number of times before...
      - 6y ago

deleted 6y ago

[deleted]

pwdisswordfish2 6y ago

Wasn't UNIX originally used to format patent applications at Bell Labs/AT&T? That was purportedly the selling point of UNIX to the business people. When someone suggests using something other than "text" (usually it's some binary format, I consider S-Expressions to be text), I always wonder, why is this person writing their suggestion using text? Then we have the suggestions that "everything is a file" is also a "mistake". That's great but if a "file" is not a good enough abstraction then what is?

wtetzner 6y ago
> When someone suggests using something other than "text" (usually it's some binary format, I consider S-Expressions to be text), I always wonder, why is this person writing their suggestion using text? Of course Unix actually operations on streams of bytes, which often represent text, but not always.
megameter 6y ago
Text is a human language. Computers do not really run on text. You can't talk to Unix like it's the bridge computer in Star Trek. No, Unix runs on bytes that sort of look like text because we have encoded and rendered it that way. What computers need most for structured processing are data types. At instruction set level we had a period of experimentation with byte and word lengths which was still in effect when Unix was born. At that time we also had experimentation with text encodings such as EBCDIC and the various ASCII codepages. Since then we have pretty much settled on standards based on multiples of an 8-bit byte, and on a common text encoding - UTF-8. At the time of Unix's inception the byte stream was a relatively bold move because it enforced variable-length encodings of bytes as the building block of interoperability, versus some kind of static record. Now it is mundane, and the world has moved on to always parsing the bytes for some other purpose. We desire standards that encode data containers, strings and numeric primitives, so that we do less parsing. XML and S-expressions offer two methods of encoding hierarchies of strings. More recently, JSON has become a very common encoding. It encodes strings, numeric values, and two types of hierarchical containers(key-value and array). We also have SQL databases as an older example, which encode various primitives and a variety of tabular data relationships. With each data language we get one or more associated query languages to express how we select data. Unix's handling of files is incomplete because of the ambiguity of queries on files. We rely heavily on relationships between directories and on glob syntax to perform selection in a Unix filesystem. Unix paths and glob syntax are not really a byte stream or a file, but they are most certainly part of Unix, and you would be suffering in short order without those concepts. Yet Unix does not respect the existence of its own query language, and never declares a basic system representation for queries and their results akin to the results of a SQL query. Instead we parse the query as bytes and output the results of queries to bytes and parse those bytes, which means there is no guarantee of support at any of the boundaries where your program touches paths. As long as you only deal with one file at a time you don't have an issue, but at scale, the way we work with our applications needs more complex selections on files; the workaround is to "bottle up" the data into one file that has some other structure, which then necessitates special tools to move around the data. And if you look at what people complain about when working with the shell, the behavior of file selection is a major complaint, and adds a lot of complexity.

seagreen 6y ago

> The fact that after half a century, Unix-like operating systems and programming environments are de rigueur is plain enough evidence of this. Unfortunately Unix blew a 28-3 lead against Netscape Navigator and now 99% of people spend their time in web browsers instead. I think this speaks to the weakness of the experience using Unix, not its strength.

pantalaimon 6y ago
> Unfortunately Unix blew a 28-3 lead against Netscape Navigator and now 99% of people spend their time in web browsers instead. There might be exceptions, but people generally don't develop software in their browser.
- seagreen 6y ago
  Thanks for the serious response to a somewhat snarky comment. I see software development as only one among the many things a computing system should enable. It's definitely an important one. But computers should enable creativity and power for all kinds of tasks, not just programming. It's sad to me that people in all these different fields besides ours get stuck with webapps. I understand why they use them, they're very convenient. But it would have been great if we could have provided them an even better OS/desktop environment so that they never would have had to switch to such an unempowering tool. Additionally, programmers being one of the last holdouts of actually engaging with Unix-as-a-way-of-doing-things is a precarious place to be. Microsoft/GitHub is on full "embrace" mode again, and it looks like they're going to make another run at slurping everyone into their ecosystem. Same with tools like Repl.it. It seems to me that the proportion of people who are engaged with the Unix way of doing things (piping text around, etc.) is shrinking, not growing. EDIT: To get myself back on topic, for these reasons I see people experimenting with new (and crucially local) environments as really exciting. Traditional Unix environments don't need to be the last word, we can do better.
  - linguae 6y ago
    Apple went down a very interesting path during the interregnum years from 1985 through 1996. It was a beacon of creativity where some of its engineers worked on projects that offered new visions of personal computing that empowered the user without requiring them to be full-fledged programmers. HyperCard and AppleScript are examples of such projects, and the Knowledge Navigator, while not an actual product, was one of Apple's visions. I also find OpenDoc to be quite intriguing; OpenDoc would have introduced component-based GUI software to the mass public. Users could perform tasks by mixing-and-matching components and applying them to a document. It would have been really cool to have seen Copland and Gershwin in its completed forms, but sadly as creative Apple was in the 1990s, its management was a mess until Steve Jobs returned, and so Copland never saw the light of day other than a few developer releases. I am grateful for the work that Steve Jobs did at NeXT and for his amazing work returning to Apple, saving it from possible bankruptcy, and making it the juggernaut that it is today. I've long been a fan of Mac OS X. However, parts of me miss the Apple of the 1980s and 1990s, where the engineers and researchers there explored ideas of how to make personal computing better. I feel that the personal computing experience for desktops have been stuck in a rut for nearly 20 years, with some aspects actually degrading rather than improving. But projects such as Snakeware give me hope that we'll see desktop computing move forward. I'm actually working on a side project where I'm creating a Common Lisp-based desktop environment inspired by Lisp machines and OpenDoc that can run on Linux and the BSDs, but I'm in the design phase and I'm far from writing any code (I've been studying up on graphics programming since part of this project involves writing a new GUI toolkit). There is still a lot of room for innovation on the desktop, and as you said, Unix isn't the last word, even for programming.

franga2000 6y ago

Piping data from one command to the next is one of those things that I love showing to new users and it gets them all excited, only to then have to disappoint them when they think of a cool idea because they'd need to learn regex, sed and awk first to do anything non-trivial. It's a shame Powershell is a disaster, because having a structured-by-default pipeline would've really bridged the gap between power users and programmers.

majkinetor 6y ago
> It's a shame Powershell is a disaster Yeah, lets get back to bash forever :S
- franga2000 6y ago
  Bash is a good shell with a fairly unimpressive pipeline. Powershell is bad shell with a good pipeline. For me, as a programmer, the shell part of a shell is more important, as I can do the rest in a more powerful language. Powershell is too verbose for shell usage (see New-Item), brings nothing new to the table in terms of scripting and is ultimately not worth using just for the better pipeline. (another commenter pointed me to Nushell, which seems to be a very good middle ground, and I've heard of a few more similar efforts, so we hopefully won't be stuck with just bash for too long)
  - majkinetor 6y ago
    Oh, here we go again with verbosity like aliases are non existent. For realz, New-Item is ni for example or mkdir or whatever... > as I can do the rest in a more powerful language Powershell is like repl for dotNet. You can't really get more powerfull. > brings nothing new to the table in terms of scripting Are you serious ? I guess somebody has a lot of learning to do. > (another commenter pointed me to Nushell, which seems to be a very good middle ground, and I've heard of a few more similar efforts, so we hopefully won't be stuck with just bash for too long) We were on that road number of times before...
    - 6y ago