4 ms·

The biggest issue I have with csv files, is when the people sending them don't get the $%^&@ quotes right. The second biggest issue, is when they say "sorry, we

The biggest issue I have with csv files, is when the people sending them don't get the $%^&@ quotes right. The second biggest issue, is when they say "sorry, we're not able to do that" after I tell them exactly how to fix what they're sending.

akinder 13y ago
What is the proper way to encode the quotes? I'm playing around with writing a Livejournal data dump reader and will have to do the quote dance pretty soon. In LJ files they are encoded like so: col1,col2,col3,"Longer data and somethin ""with"" quotes",col5
- tbrownaw 13y ago
  Yes, that's how they're supposed to be. Any field with quotes, or commas, or newlines in the data needs to have quotes around it. Any quotes in the data need to be escaped by preceding them with another quote.
  - justinator 13y ago
    From what I understand, you're basically right, but you're basically right only in a sense that that's what most people do. There's no "CSV Data Format" spec. It's all just what most people agree on, most of the time. Unless someone has another idea.
    - steveklabnik 13y ago
      text/csv is defined in RFC 4180: https://tools.ietf.org/html/rfc4180 https://tools.ietf.org/html/rfc4180
      - nemetroid 13y ago
        Strictly, the definition in RFC 4180 mandates ASCII which makes it unusable for many purposes. I guess there's nothing practical stopping you from using another encoding though.
        steveklabnik 13y ago
        > Common usage of CSV is US-ASCII, but other character sets defined > by IANA for the "text" tree may be used in conjunction with the > "charset" parameter. http://www.rfc-editor.org/rfc/rfc6657.txt http://www.rfc-editor.org/rfc/rfc6657.txt https://www.iana.org/assignments/character-sets/character-sets.xhtml https://www.iana.org/assignments/character-sets/character-se...
alayne 13y ago
I prefer DSV files for this reason http://en.wikipedia.org/wiki/Delimiter-separated_values http://en.wikipedia.org/wiki/Delimiter-separated_values I have a simple scheme where comma is the delimiter and , in content is escaped as \,. There are no quotes around values.
sehrope 13y ago
Substringed JSON arrays work great for this as well. Basically you just convert each row to an array, convert the array to JSON, and the strip the leading and trailing brackets ("[" and "]"). Newlines will be properly represented as \n as well as other funky characters.

akinder 13y ago

What is the proper way to encode the quotes? I'm playing around with writing a Livejournal data dump reader and will have to do the quote dance pretty soon. In LJ files they are encoded like so: col1,col2,col3,"Longer data and somethin ""with"" quotes",col5

tbrownaw 13y ago
Yes, that's how they're supposed to be. Any field with quotes, or commas, or newlines in the data needs to have quotes around it. Any quotes in the data need to be escaped by preceding them with another quote.
- justinator 13y ago
  From what I understand, you're basically right, but you're basically right only in a sense that that's what most people do. There's no "CSV Data Format" spec. It's all just what most people agree on, most of the time. Unless someone has another idea.
  - steveklabnik 13y ago
    text/csv is defined in RFC 4180: https://tools.ietf.org/html/rfc4180 https://tools.ietf.org/html/rfc4180
    - nemetroid 13y ago
      Strictly, the definition in RFC 4180 mandates ASCII which makes it unusable for many purposes. I guess there's nothing practical stopping you from using another encoding though.
      - steveklabnik 13y ago
        > Common usage of CSV is US-ASCII, but other character sets defined > by IANA for the "text" tree may be used in conjunction with the > "charset" parameter. http://www.rfc-editor.org/rfc/rfc6657.txt http://www.rfc-editor.org/rfc/rfc6657.txt https://www.iana.org/assignments/character-sets/character-sets.xhtml https://www.iana.org/assignments/character-sets/character-se...

alayne 13y ago

I prefer DSV files for this reason http://en.wikipedia.org/wiki/Delimiter-separated_values http://en.wikipedia.org/wiki/Delimiter-separated_values I have a simple scheme where comma is the delimiter and , in content is escaped as \,. There are no quotes around values.

sehrope 13y ago

Substringed JSON arrays work great for this as well. Basically you just convert each row to an array, convert the array to JSON, and the strip the leading and trailing brackets ("[" and "]"). Newlines will be properly represented as \n as well as other funky characters.