4 ms·
That isn't solving anything, just changing the problem. If I want to store a string containing 0x1C - 0x1F in one of the columns then we're back in the exact sa
by craftkiller 2y ago
That isn't solving anything, just changing the problem. If I want to store a string containing 0x1C - 0x1F in one of the columns then we're back in the exact same situation while also losing the human readable/manually typeable aspect people seem to love about CSV. The real solution is a strict spec with mandatory escaping.
- SoftTalker 2y agoNot for text data. Those values are not text characters like , or " are, and have only one meaning. It would be like arguing that 0x41 isn't always the letter "A". For binary files, yeah but you don't see CSV used there anyway.
- Hackbraten 2y agoThere are several ways how a control character might inadvertently end up inside a text corpus. Given enough millions of lines, it’s bound to happen, and you absolutely don’t want it to trip up your whole export because of that one occurrence. So yes, you have to account for it in text data, too.
- yellowapple 2y agoSure, but that's orders of magnitude less likely to happen than a comma ending up inside a text corpus.
- craftkiller 2y agoSo in addition to losing human readability, we are also throwing away the ability to nest (pseudo-)CSVs? With comma delimiters, I can take an entire CSV document and put it in 1 column, but with 0x1C-0x1F delimiters and banning non-text valid utf-8 in columns I no longer can. This continues to be a step backwards.
- thayne 2y agoNo reason you can't escape those special characters.
- craftkiller 2y agoThen we're back to my original response of it doesn't solve anything: https://news.ycombinator.com/item?id=43495217 https://news.ycombinator.com/item?id=43495217
- mjw_byrne 2y agoThe idea that binary data doesn't go in CSVs is debatable; people do all sorts of weird stuff. Part of the robustness of a format is coping with abuse. But putting that aside, if the control chars are not text, then you sacrifice human-readability and human-writability. In which case, you may as well just use a binary format.
- Gormo 2y agoI've dealt with a few cases of CSVs including base64-encoded binary data. It's an unusual scenario, but the tools for working with CSVs are robust enough that it was never an issue.
- SoftTalker 2y agoTrue, but very few people compose or edit CSV data in Notepad. You can, but it's very error-prone. Most people will use a spreadsheet and save as CSV, so field and record separator characters are not anything they would ever deal with.
- kragen 2y agoThere are lots of 8-bit mostly-ASCII character sets that assign printable glyphs to some or all of the codepoints that ASCII assigns to control characters. TeX defined one, and the IBM PC's "code page 437" defined another.