3 ms·

Related thread: CSVs: The Good, the Bad, and the Ugly (2020) https://news.ycombinator.com/item?id=25014721 https://news.ycombinator.com/item?id=25014721 FWIW

by chubot 5y ago

Related thread: CSVs: The Good, the Bad, and the Ugly (2020) https://news.ycombinator.com/item?id=25014721 https://news.ycombinator.com/item?id=25014721 FWIW I have designed an upgrade to TSV as part of https://www.oilshell.org https://www.oilshell.org. Oil actually emits it now from 'pp proc' to pretty print "functions" and their docstrings as a table. It will be used in many parts of Oil, most of which aren't implemented yet. It's called QTT -- Quoted, Typed Tables (formerly QTSV) It's built on top of QSN, which is just Rust string literal notation -- https://www.oilshell.org/release/latest/doc/qsn.html https://www.oilshell.org/release/latest/doc/qsn.html There is a stub doc which links to a wiki proposal -- http://www.oilshell.org/release/latest/doc/qtt.html http://www.oilshell.org/release/latest/doc/qtt.html ------ I think the only way that such a format becomes popular is if Oil itself becomes popular, so I haven't really emphasized it as a spec. (Similar to how JSON wouldn't be popular if the browser / JS weren't popular) But it is a pretty easy spec -- just take TSV, add types to the column header, and specify that any cell that begins with a single quote is QSN. A nice property is that every almost TSV file is a valid QTT file -- the exception being TSV files where a cell just a single quote, or some malformed QSN. Note that TSV officially can't represent fields with tabs, but QTT can with '\t'. As always feel free to contact me if you want to work on any of this ---- The ASCII field separators don't make sense -- they break tools and take you off the narrow waist of Unix. https://www.oilshell.org/blog/2022/02/diagrams.html https://www.oilshell.org/blog/2022/02/diagrams.html A key design point of QTT is that it's a special case of the "lines of text" narrow waist. CSV does not have that property, because fields can contain embedded newlines. And unlike TSV, QTT cells can contain arbitrary values.

earthboundkid 5y ago
You should make the connection of QTT and QSN to Rust more clear by coming up with a backronym for RSON.
- chubot 5y ago
  Hm I wasn't aware of RSON. https://github.com/rson-rs/rson https://github.com/rson-rs/rson QSN isn't intended to be tied to Rust in any way (and isn't), while RSON says it uses the Serde data model. This gets at an issue I have been having a hard time explaining, mentioned here: http://www.oilshell.org/blog/2022/03/backlog-arch.html http://www.oilshell.org/blog/2022/03/backlog-arch.html That is, narrow waists are necessarily a COMPROMISE. JSON is a compromise, and Rust users will be equally unhappy as Lua or Erlang users. That is a feature and not a bug for something meant of interoperability. You are "stuck with" the lowest common denominator, but that's what enables interop. (The lowest common denominator of them all is Unix byte streams.) I contrast "monoglot" serialization formats like Python pickle an Go .gob with language-independent formats like JSON, TSV, and HTML. The wisdom of JSON is that Crockford specified it independently of JavaScript. But both are useful. It's not clear if RSON is meant to be monoglot or polyglot, but it's a huge difference and it seems more monoglot. QSN on the other hand is definitely a polyglot design like JSON, despite being derived from Rust.

by chubot 5y ago

earthboundkid 5y ago

You should make the connection of QTT and QSN to Rust more clear by coming up with a backronym for RSON.

chubot 5y ago
Hm I wasn't aware of RSON. https://github.com/rson-rs/rson https://github.com/rson-rs/rson QSN isn't intended to be tied to Rust in any way (and isn't), while RSON says it uses the Serde data model. This gets at an issue I have been having a hard time explaining, mentioned here: http://www.oilshell.org/blog/2022/03/backlog-arch.html http://www.oilshell.org/blog/2022/03/backlog-arch.html That is, narrow waists are necessarily a COMPROMISE. JSON is a compromise, and Rust users will be equally unhappy as Lua or Erlang users. That is a feature and not a bug for something meant of interoperability. You are "stuck with" the lowest common denominator, but that's what enables interop. (The lowest common denominator of them all is Unix byte streams.) I contrast "monoglot" serialization formats like Python pickle an Go .gob with language-independent formats like JSON, TSV, and HTML. The wisdom of JSON is that Crockford specified it independently of JavaScript. But both are useful. It's not clear if RSON is meant to be monoglot or polyglot, but it's a huge difference and it seems more monoglot. QSN on the other hand is definitely a polyglot design like JSON, despite being derived from Rust.