3 ms·
Just to clarify something from the article and the github repo code (sorry if this is readily explained in the repo, C code using AVX stuff is not something I h
by VoidWhisperer 1y ago
Just to clarify something from the article and the github repo code (sorry if this is readily explained in the repo, C code using AVX stuff is not something I have any experience with): does the SIMD version still handle things like escaped quotes inbetween quotes denoting a string in a field in the csv file? It isn't mentioned in the article
- jiggawatts 1y agoThe readme in the source code mentions RFC 4180 compliance. This snippet of the code shows that it is correctly handled: // from init_tables() } else { // When in a quoted string, a quote char transitions to the escape state. st[S_QUOTED][q] = S_QUOTE_ESC; // By default, any character seen after the escape state's quote // will transition back to the unquoted state (ending the field). memset(st[S_QUOTE_ESC], S_UNQUOTED, 256); // CRITICAL OVERRIDE: If the character is another quote, it was an // escaped quote, so we go back to the quoted state. st[S_QUOTE_ESC][q] = S_QUOTED; }