3 ms·

It won't wrap at the record separator, so you'll get a very long line.

by tgv 3y ago

bryanlarsen 3y ago
The example seems to use `␞\n` as a separator rather than just `␞`. I assume their proposed standard is more definitive.
- vidarh 3y ago
  Their ABNF uses RS, defined as U+241E, not U+241E + '\n' as the record separator. They seem to add an "USV escape" in front of the linefeeds. My bet is that this will lead to implementations that wrongly treats "␞␛\n" (RS ESC \m) as the real record separator, the same way lots of "CSV" implementations just split on comma and LF. Seems to me if you're going to add support for something like that you should just bite the bullet and declare an LF immediately following an RS as part of the record separator, or you're falling in the same trap as CSV of being "close enough" to naively splittable that people will do it because it works often enough.
  - shawnz 3y ago
    The escape symbol lets you ignore any non-special character, not just newlines: https://github.com/sixarm/usv?tab=readme-ov-file#escape-esc https://github.com/sixarm/usv?tab=readme-ov-file#escape-esc
    - vidarh 3y ago
      I'm aware. I don't think that serves a useful purpose - I think the way they've done it is likely to make people more likely to get the parsing wrong for pretty much zero benefit. My guess is you'll end up seeing a lot of "pseudo-USV" parsers the same way we have a ton of "pseudo-CSV" parsers that breaks on escapes or quoted strings with commas, and so I think they fundamentally failed to learn the lessons of CSV.
- theamk 3y ago
  that's a lie as far as I can see, the csv-to-usv tools does not add any newlines: [0] https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/src/lib.rs#L11 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...
  - bryanlarsen 3y ago
    The examples here have them: https://github.com/SixArm/usv/tree/main/examples https://github.com/SixArm/usv/tree/main/examples
    - theamk 3y ago
      the submitted tool does not have produce them, check out the tests - note there is no \n anywhere in USV https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/tests/integration_test.rs#L7 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...

It won't wrap at the record separator, so you'll get a very long line.

by tgv 3y ago

It won't wrap at the record separator, so you'll get a very long line.

The example seems to use `␞\n` as a separator rather than just `␞`. I assume their proposed standard is more definitive.

vidarh 3y ago
Their ABNF uses RS, defined as U+241E, not U+241E + '\n' as the record separator. They seem to add an "USV escape" in front of the linefeeds. My bet is that this will lead to implementations that wrongly treats "␞␛\n" (RS ESC \m) as the real record separator, the same way lots of "CSV" implementations just split on comma and LF. Seems to me if you're going to add support for something like that you should just bite the bullet and declare an LF immediately following an RS as part of the record separator, or you're falling in the same trap as CSV of being "close enough" to naively splittable that people will do it because it works often enough.
- shawnz 3y ago
  The escape symbol lets you ignore any non-special character, not just newlines: https://github.com/sixarm/usv?tab=readme-ov-file#escape-esc https://github.com/sixarm/usv?tab=readme-ov-file#escape-esc
  - vidarh 3y ago
    I'm aware. I don't think that serves a useful purpose - I think the way they've done it is likely to make people more likely to get the parsing wrong for pretty much zero benefit. My guess is you'll end up seeing a lot of "pseudo-USV" parsers the same way we have a ton of "pseudo-CSV" parsers that breaks on escapes or quoted strings with commas, and so I think they fundamentally failed to learn the lessons of CSV.
theamk 3y ago
that's a lie as far as I can see, the csv-to-usv tools does not add any newlines: [0] https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/src/lib.rs#L11 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...
- bryanlarsen 3y ago
  The examples here have them: https://github.com/SixArm/usv/tree/main/examples https://github.com/SixArm/usv/tree/main/examples
  - theamk 3y ago
    the submitted tool does not have produce them, check out the tests - note there is no \n anywhere in USV https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324d8ebd1086bae52ab2fc8d3c9abda53c0a/tests/integration_test.rs#L7 https://github.com/SixArm/csv-to-usv-rust-crate/blob/30a0324...