4 ms·

This is an interesting idea. I don't know whether it will take off. One thought that occurred to me several times in the last year or so is that roughly the l

This is an interesting idea. I don't know whether it will take off. One thought that occurred to me several times in the last year or so is that roughly the level of abstraction offered by NestedText might make sense as part of a hierarchy of abstractions that could be built on. We already have that with text files. Because end-of-line character combinations are special, text files in a given encoding are already more structured than streams of characters. So, assuming UTF-8 character encoding: CharacterStream Text (EOL character combinations) NestedText MyNestedTextFormat (domain specific semantics) With non-text files, this has already happened more than once. For example, both Zip files and Sqlite files are used as base formats for specifying other formats.

rswail 6y ago
ASCII has always had separator and message delineation characters [1][2] that can be used instead of the CR, LF, SP, TAB, "|", "," etc. They are not "human readable", but are very easy to parse. They are even UTF-8 transparent and can be easily converted to/from CSV, TSV, PSV, and, with a definition of the equivalent of a "close brace" could allow for a multi-level hierarchy. [1] https://en.wikipedia.org/wiki/Control_character#Data_structuring https://en.wikipedia.org/wiki/Control_character#Data_structu... [2] https://en.wikipedia.org/wiki/C0_and_C1_control_codes https://en.wikipedia.org/wiki/C0_and_C1_control_codes
- delhanty 6y ago
  I don't know whether that's an "aside" comment, but it's interesting and informative anyway! Another variant of the idea that I was mulling over was to base a roughly NestedText level of abstraction on UTF-8 transparent characters, and then combine that with what Animats was talking about as a standardized GUI for trees, dictionaries etc. A recent trend is that programming languages have more than one bijectively equivalent syntax. For example, ReasonML [0] and OCaml are two bijectively text based syntaxes. That idea could be extended to a NestedText like syntax being bijectively equivalent to a text based syntax. Editors like Visual Studio Code infer that sort of information continually on the fly, but it sort of gets lost in the toolchain. Compilers could operate at a higher-level of abstraction than lexing/scanning. Git merge might also work better if could operate at a NestedText like level of abstraction. [0] https://en.wikipedia.org/wiki/Reason_(syntax_extension_for_OCaml) https://en.wikipedia.org/wiki/Reason_(syntax_extension_for_O...
- samatman 6y ago
  Yep. We were recently storing tokens in a database, and I chose to use SOH for the metadata and SOX for the text. One byte width, no collisions with printable text, and that's what they're there for. I'd love to see a CSV replacement that used SOH ... SOX for headers, RS as "commas", and GS as "newlines". You'd be able to cleanly concatenate multiple files, since the first line is no longer special, and you'd be able to have commas, newlines, and in fact any printable text whatsoever inside the data. And the semantics are perfectly clean. Again, that's what they're there for. Some small challenges for hand-editing that a competent text editor can easily rise to.
  - delhanty 6y ago
    There's a lot of value to the ecosystem here if it could be standardized - possibly it should be an RFC. Combined with what rswail wrote about encoding hierarchies, with careful design, those CSV sections could be embedded as tables. If that was used as a base format for other formats, then objection that encoding for boolean and numerics wasn't standardized might go away.
  - mongol 6y ago
    Me too. I think this could be achieved. But there would need to be good editor support to make it successful. And good keyboard support too.
  - mongol 6y ago
    Looking closer at it.. I think US would be commas, and RS the newlines? Leaves the imagination what GS could be used for..
    - samatman 6y ago
      Hmm I think it would be nice to reserve US for when you have multiple entries in a single column. Like if the column was phone numbers and occasionally there's more than one, that sort of thing. Thinking of each cell as a "record" and allowing it to have more than one "unit" makes sense to me. But anything would be better than ever having to whip up a script to fix a CSV with comma-separated dollar values in it, ever again.

rswail 6y ago

ASCII has always had separator and message delineation characters [1][2] that can be used instead of the CR, LF, SP, TAB, "|", "," etc. They are not "human readable", but are very easy to parse. They are even UTF-8 transparent and can be easily converted to/from CSV, TSV, PSV, and, with a definition of the equivalent of a "close brace" could allow for a multi-level hierarchy. [1] https://en.wikipedia.org/wiki/Control_character#Data_structuring https://en.wikipedia.org/wiki/Control_character#Data_structu... [2] https://en.wikipedia.org/wiki/C0_and_C1_control_codes https://en.wikipedia.org/wiki/C0_and_C1_control_codes

delhanty 6y ago
I don't know whether that's an "aside" comment, but it's interesting and informative anyway! Another variant of the idea that I was mulling over was to base a roughly NestedText level of abstraction on UTF-8 transparent characters, and then combine that with what Animats was talking about as a standardized GUI for trees, dictionaries etc. A recent trend is that programming languages have more than one bijectively equivalent syntax. For example, ReasonML [0] and OCaml are two bijectively text based syntaxes. That idea could be extended to a NestedText like syntax being bijectively equivalent to a text based syntax. Editors like Visual Studio Code infer that sort of information continually on the fly, but it sort of gets lost in the toolchain. Compilers could operate at a higher-level of abstraction than lexing/scanning. Git merge might also work better if could operate at a NestedText like level of abstraction. [0] https://en.wikipedia.org/wiki/Reason_(syntax_extension_for_OCaml) https://en.wikipedia.org/wiki/Reason_(syntax_extension_for_O...
samatman 6y ago
Yep. We were recently storing tokens in a database, and I chose to use SOH for the metadata and SOX for the text. One byte width, no collisions with printable text, and that's what they're there for. I'd love to see a CSV replacement that used SOH ... SOX for headers, RS as "commas", and GS as "newlines". You'd be able to cleanly concatenate multiple files, since the first line is no longer special, and you'd be able to have commas, newlines, and in fact any printable text whatsoever inside the data. And the semantics are perfectly clean. Again, that's what they're there for. Some small challenges for hand-editing that a competent text editor can easily rise to.
- delhanty 6y ago
  There's a lot of value to the ecosystem here if it could be standardized - possibly it should be an RFC. Combined with what rswail wrote about encoding hierarchies, with careful design, those CSV sections could be embedded as tables. If that was used as a base format for other formats, then objection that encoding for boolean and numerics wasn't standardized might go away.
- mongol 6y ago
  Me too. I think this could be achieved. But there would need to be good editor support to make it successful. And good keyboard support too.
- mongol 6y ago
  Looking closer at it.. I think US would be commas, and RS the newlines? Leaves the imagination what GS could be used for..
  - samatman 6y ago
    Hmm I think it would be nice to reserve US for when you have multiple entries in a single column. Like if the column was phone numbers and occasionally there's more than one, that sort of thing. Thinking of each cell as a "record" and allowing it to have more than one "unit" makes sense to me. But anything would be better than ever having to whip up a script to fix a CSV with comma-separated dollar values in it, ever again.