4 ms·
I ran into this problem while building my new data management system. I create relational tables by parsing CSV, TSV, or other 'character separated' files to ge
by didgetmaster 4y ago
I ran into this problem while building my new data management system. I create relational tables by parsing CSV, TSV, or other 'character separated' files to get the row data and then use separate threads to insert each row. Once I had successfully parsed out a row, I could multi-thread the insertion code but only one parser thread could work with the actual file since embedded line breaks could be enclosed within quotes. It only takes one line to break any kind of parallel processing on the file.