8 ms·

All downsides, no upsides. You cannot edit it in regular editor, like csv/tsv/jsonlines. There is no schema or efficient storage, like binary formats. There

by theamk 3y ago

All downsides, no upsides. You cannot edit it in regular editor, like csv/tsv/jsonlines. There is no schema or efficient storage, like binary formats. There is no wide library support. Not all data is representable.

mavhc 3y ago
1 Editors can be improved 2 Same as CSV etc then 3 Libraries can be improved 4 Escaping characters exists ASCII 1963 had 8 separators, 1965 reduced it to 4, and named them. See 6.3.12 of https://dl.acm.org/doi/pdf/10.1145/363831.363839 https://dl.acm.org/doi/pdf/10.1145/363831.363839
- cxr 3y ago
  The task here is to explain why one should use this over CSV. By your own admissions, there is no reason to prefer this over CSV.
  - mavhc 3y ago
    There's no standard for CSV files, thus no one can parse them properly The only time you need to escape a character is if it's a control character that's rarely used, unlike the " and , characters
    - cxr 3y ago
      > There's no standard for CSV files Literally untrue. (And were it true, it still wouldn't be a reason why one should use this over CSV—not sure what's so hard to grasp about the conversational/contextual premise here.)
ixwt 3y ago
> You cannot edit it in regular editor, like csv/tsv/jsonlines. If only there were shortcuts on modern operating systems to allow us to do things that aren't readily on our keyboards. Like upper case characters. Or copy and paste. Or close windows. Our lives would be so much better. If ASV had caught on, there could be common shared shortcuts to type them, and fonts would regularly display them (just like the unicode characters proposed). But CSV was simple enough and readily type-able. > There is no schema or efficient storage, like binary formats. I'm not quite certain where you're trying to go with this. Binary formats aren't really meant to be human readable in an average text editor. It doesn't know to differentiate 1, 2, 4, or 8 bytes as an integer or a float. Even current hex editors to make it easier to navigate these formats don't really know unless you are able to tell it somehow. > There is no wide library support. It's a critical mass problem. Not enough people are using them, so no libraries are being made. > Not all data is representable. I'm not quite certain what data couldn't be represented. f you can represent your data in CSV, you can represent it in ASV. It's all plain text that gets interpreted based on what you need. They're nearly a 1:1 replacement. Commas get replaced by unit separators, new lines get replaced by group separators. Then you have record and file separators to do with for further levels of abstraction if you need.
- theamk 3y ago
  Re editors: The problem with USV is not that it's hard to type the characters, but rather than the newlines are completely optional. Which means that in general case, most line-based tools are not going to work with USV. Now, the readme actually has that optional newline separator thing, but the optionality of it makes it completely useless, it seems like an after-thought. Fr example the first "real" USV writer I found, the "csv-to-usv", does not put them [0] and thus makes uneditable files. And if we are going to end up with uneditable files, might as well go with something schema-full, like parquet or avro. You are going to have the same "critical mass problem", but at least the tooling is much better and you have neat features like schemas. [0] https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/src/lib.rs#L11 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...
  - jph 3y ago
    Good catch! I just fixed csv-to-usv so it prints newlines now. You're right, Parquet and Avro are both great, for use with schemas.
- LegionMammal978 3y ago
  > I'm not quite certain what data couldn't be represented. What do you do if you receive data already containing a unit separator, or a group separator, and you need to put it into a field? The whole value proposition of ASV over, say, TSV is that you should never need to escape anything, but that's only possible by rejecting some input data.

by theamk 3y ago

mavhc 3y ago

1 Editors can be improved 2 Same as CSV etc then 3 Libraries can be improved 4 Escaping characters exists ASCII 1963 had 8 separators, 1965 reduced it to 4, and named them. See 6.3.12 of https://dl.acm.org/doi/pdf/10.1145/363831.363839 https://dl.acm.org/doi/pdf/10.1145/363831.363839

cxr 3y ago
The task here is to explain why one should use this over CSV. By your own admissions, there is no reason to prefer this over CSV.
- mavhc 3y ago
  There's no standard for CSV files, thus no one can parse them properly The only time you need to escape a character is if it's a control character that's rarely used, unlike the " and , characters
  - cxr 3y ago
    > There's no standard for CSV files Literally untrue. (And were it true, it still wouldn't be a reason why one should use this over CSV—not sure what's so hard to grasp about the conversational/contextual premise here.)

ixwt 3y ago

> You cannot edit it in regular editor, like csv/tsv/jsonlines. If only there were shortcuts on modern operating systems to allow us to do things that aren't readily on our keyboards. Like upper case characters. Or copy and paste. Or close windows. Our lives would be so much better. If ASV had caught on, there could be common shared shortcuts to type them, and fonts would regularly display them (just like the unicode characters proposed). But CSV was simple enough and readily type-able. > There is no schema or efficient storage, like binary formats. I'm not quite certain where you're trying to go with this. Binary formats aren't really meant to be human readable in an average text editor. It doesn't know to differentiate 1, 2, 4, or 8 bytes as an integer or a float. Even current hex editors to make it easier to navigate these formats don't really know unless you are able to tell it somehow. > There is no wide library support. It's a critical mass problem. Not enough people are using them, so no libraries are being made. > Not all data is representable. I'm not quite certain what data couldn't be represented. f you can represent your data in CSV, you can represent it in ASV. It's all plain text that gets interpreted based on what you need. They're nearly a 1:1 replacement. Commas get replaced by unit separators, new lines get replaced by group separators. Then you have record and file separators to do with for further levels of abstraction if you need.

theamk 3y ago
Re editors: The problem with USV is not that it's hard to type the characters, but rather than the newlines are completely optional. Which means that in general case, most line-based tools are not going to work with USV. Now, the readme actually has that optional newline separator thing, but the optionality of it makes it completely useless, it seems like an after-thought. Fr example the first "real" USV writer I found, the "csv-to-usv", does not put them [0] and thus makes uneditable files. And if we are going to end up with uneditable files, might as well go with something schema-full, like parquet or avro. You are going to have the same "critical mass problem", but at least the tooling is much better and you have neat features like schemas. [0] https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/src/lib.rs#L11 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...
- jph 3y ago
  Good catch! I just fixed csv-to-usv so it prints newlines now. You're right, Parquet and Avro are both great, for use with schemas.
LegionMammal978 3y ago
> I'm not quite certain what data couldn't be represented. What do you do if you receive data already containing a unit separator, or a group separator, and you need to put it into a field? The whole value proposition of ASV over, say, TSV is that you should never need to escape anything, but that's only possible by rejecting some input data.