5 ms·
The most retard structure I've seen in a CSV file relates to the "What if the character separating fields is not a comma?". We get "CSV" files from Klarna, an
by mrweasel 12y ago
The most retard structure I've seen in a CSV file relates to the "What if the character separating fields is not a comma?".
We get "CSV" files from Klarna, an invoicing company, with the payments they've processed for us. Because we're Danish and they are Swedish, it's not really weird that they would use comma as the decimal separator. So to compensate for having used the comma, they for some reason picks ", " ( that's comma + space ) as the field separator. Most good csv parsers can handle the field separator to be any character you like, as long is it's just ONE character. By picking a two character separator they've just dictated that I write my own or resort to just splitting a line on ", ".
- callesgg 12y agoIn General Swedish csv files are separated with ; I have an function that I usually use in projects that counts , and ; on each line to determine which one is most likly beeing used in the file. The most annoying thing I have found in csv files is the escape sign I would like it to be \" but very often I see """ as the escape for "
- mrweasel 12y agoI believe that the Swedish Klarna files uses ; but the Danish ones uses the comma + space. That only adds to the stupidity though, why not have just one format?
- elinchrome 12y agoJust be happy to live in the best country.
- chrismcb 12y agoWell the standard says to double the double quote to escape it. If you are seeing \" or even """ to escape a quote, then someone isn't following the standard.
- tikumo 12y agoit can be irritating, but you can just as easy parse ", " to "|" or something, by simple string replacing, pre parsing..
- sunir 12y agoThink it through. What if there is free text in the field? "How are you, Sally?"
- lignuist 12y agoYou can replace all commas with a placeholder (e.g. "#COMMA#"), replace the delimiter with a comma, parse the document and then replace all placeholders in the data with ",".
- Someone 12y agoThat does not work, unless that first replacement magically ignores the commas that are part of field separators. If you know how to write the code that does that, your problem is solved.
- lignuist 12y agoI was referencing to "What if the character separating fields is not a comma?". And there it clearly works. I used this technique a few times with success. If you find a CSV file that has mixed field separator types, then you probably found a broken CSV file.
- zAy0LfpBZLC8mAC 12y agoNo, it doesn't. What if there is #COMMA# in one of the fields?
- lignuist 12y agoYou just choose a placeholder that does not appear in the data. You could even implement it in a way that a placeholder is automatically selected upfront that does not appear in the data. When it comes to parsing, the thing is that you usually have to make some assumptions about the document structure.