3 ms·

What you want is tagged and/or typed pipes. Once a pipe declares how its data is encoded, you can build all kinds of modern tooling and high-level automation on

by hhas01 6y ago

What you want is tagged and/or typed pipes. Once a pipe declares how its data is encoded, you can build all kinds of modern tooling and high-level automation on top of that. The whole “But everything’s a text file!” is an outrageous lie, and always has been: just an excuse to cover up a massive omission in the design of the Unix file system (untyped, untagged “file” resources). Such egregious corner-cutting might have been understandable—even forgivable—back in K&R’s day, when they were bootstrapping C and Unix on hardware with less processing power than a modern microwave oven controller. But we’re decades on from that now and there’s no excuse for perpetuating it: we know how to build safe powerful modern static and dynamic type systems, and we’re spoiled rotten for the hardware resources to support it. Fifty years ago, K&R built C and Unix, in a cave, with scraps. Nowadays you can bootstrap a complete hardware stack for less than the cost of a 360, and there are modern kernels (L4) and systems language (Rust) already available for free off the shelf. So pick a target market and build the userland for that. All that’s missing is a modern-day Kernighan and Ritchie to step up and get on with it, so it’d be a sorry indictment of this generation if they do not exist.

gen220 6y ago
> untyped, untagged “file” resources I think the problem then, and the problem today, is that these “types” are a very very mobile target. How many implementations of JSON are there? XML? HTML? CSV? It doesn’t make sense to put these types in the operating system, or else you’ll start TypeError’s every time you download anything. Not to mention the Computational costs of parsing the entire structure of a document every time you edit it to verify that it’s “still of type X”. I’m not at all sold on the idea that categorizing files and streams in this way is a step forward (in the shell, at least). This is what applications are for: to define their own formats, parsers, and generators. The point of the operating system is not to be one of these applications, but to facilitate their development and use. IMO, typed files and streams would actually impede more than enable this work.
- hhas01 6y ago
  “It doesn’t make sense to put these types in the operating system” The OS doesn’t need to hardcode individual types, and to do so would defeat growth. It only has to provide a standard mechanism by which files can declare the type of data they contain, and executables can declare the type(s) of data they consume and/or produce. BeOS, for instance, tagged files with MIME types. The original Mac OS used “four-char codes”. Even DOS had file name extensions, as awful as that particular design choice was. Storing arbitrary data with no way to indicate how that data is encoded is rank insanity. Saying “it’s the app’s job to deal with encodings” is not wrong, but if apps have no way to know what encoding a given file was written in, how can they know how to decode it? Please do not rationalize away the longstanding and well-known deficiencies of Unix. These deficiencies do not aid progress; they hinder it. As do apologetics.
  - gen220 6y ago
    > files can declare the type of data they contain [...] executables can declare the type(s) of data they consume and/or produce. I think you're conceding that the OS would not be responsible for enforcing data's compliance with it's declared type. I'd be OK with this world, but I am not convinced that this hypothetical OS is substantially better than the UNIX descendants we have today. > Storing arbitrary data with no way to indicate how that data is encoded is rank insanity. I think we all can agree on this... it's just a matter of how we encode the data type. UNIX descendants follow the pattern of encoding data types with filename endings. While I agree that this approach feels primitive, it's hard to argue that it doesn't work, because it does. This is fundamentally identical to MIME-types, imo, because I believe extensions (and mime-types!) are fundamentally "names", and don't have much semantic above that, given the diversity of "name"-implementations, as stated earlier. I don't believe that we gain a lot of practical value from compelling executables to register their accepted "names" with the Operating System: I'm trying to imagine what big value this gives you, beyond transforming `some_program --out_type=json | jq` to `some_program | jq`. I hope you understand I'm not trying to apologize for UNIX. I agree that many things about it should be better. But I do think they got "everything is plaintext, from the OS's point of view" right. (I think it's good discussion and I want to learn from it)
    - 6y ago

by hhas01 6y ago

gen220 6y ago

> untyped, untagged “file” resources I think the problem then, and the problem today, is that these “types” are a very very mobile target. How many implementations of JSON are there? XML? HTML? CSV? It doesn’t make sense to put these types in the operating system, or else you’ll start TypeError’s every time you download anything. Not to mention the Computational costs of parsing the entire structure of a document every time you edit it to verify that it’s “still of type X”. I’m not at all sold on the idea that categorizing files and streams in this way is a step forward (in the shell, at least). This is what applications are for: to define their own formats, parsers, and generators. The point of the operating system is not to be one of these applications, but to facilitate their development and use. IMO, typed files and streams would actually impede more than enable this work.

hhas01 6y ago
“It doesn’t make sense to put these types in the operating system” The OS doesn’t need to hardcode individual types, and to do so would defeat growth. It only has to provide a standard mechanism by which files can declare the type of data they contain, and executables can declare the type(s) of data they consume and/or produce. BeOS, for instance, tagged files with MIME types. The original Mac OS used “four-char codes”. Even DOS had file name extensions, as awful as that particular design choice was. Storing arbitrary data with no way to indicate how that data is encoded is rank insanity. Saying “it’s the app’s job to deal with encodings” is not wrong, but if apps have no way to know what encoding a given file was written in, how can they know how to decode it? Please do not rationalize away the longstanding and well-known deficiencies of Unix. These deficiencies do not aid progress; they hinder it. As do apologetics.
- gen220 6y ago
  > files can declare the type of data they contain [...] executables can declare the type(s) of data they consume and/or produce. I think you're conceding that the OS would not be responsible for enforcing data's compliance with it's declared type. I'd be OK with this world, but I am not convinced that this hypothetical OS is substantially better than the UNIX descendants we have today. > Storing arbitrary data with no way to indicate how that data is encoded is rank insanity. I think we all can agree on this... it's just a matter of how we encode the data type. UNIX descendants follow the pattern of encoding data types with filename endings. While I agree that this approach feels primitive, it's hard to argue that it doesn't work, because it does. This is fundamentally identical to MIME-types, imo, because I believe extensions (and mime-types!) are fundamentally "names", and don't have much semantic above that, given the diversity of "name"-implementations, as stated earlier. I don't believe that we gain a lot of practical value from compelling executables to register their accepted "names" with the Operating System: I'm trying to imagine what big value this gives you, beyond transforming `some_program --out_type=json | jq` to `some_program | jq`. I hope you understand I'm not trying to apologize for UNIX. I agree that many things about it should be better. But I do think they got "everything is plaintext, from the OS's point of view" right. (I think it's good discussion and I want to learn from it)
  - 6y ago