3 ms·

> The right answer will depend on the column separator. Proposing the UNIX cut if the file is CSV is not such a good answer, but for tab-separated files, it is

> The right answer will depend on the column separator. Proposing the UNIX cut if the file is CSV is not such a good answer, but for tab-separated files, it is just fine. If the file is CSV and they tell me about cut, my next question would be if that is a good universal solution for CSV files in general. TSV and CSV have the same limitations. A tab-separated file could still have tabs inside a field depending on the quoting convention. Either separator could be used with cut. I can't believe you are so confident in your partially truthful answers.

AtlasBarfed 7y ago
Bingo!
buzzkillington 7y ago
The only true Unix column format is ascii delimited text. Oddly no one has heard of that, the only reason why I found out about it is because I had to read in punched tapes with 7 character ascii from an experiment done in the 80s during my undergrad. https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text
- amznthrowaway5 7y ago
  Fascinating, thanks. In all of the large projects I've worked on, the CSV format variants and their inconsistencies have caused disasters. Who knew that Pandas and Spark handle CSV settings differently by default and spark has a hard time with newlines in its CSV output. CSVs inconsistencies result in a lot of data corruption, doubly so in large teams. Replacing CSV with flat JSON or parquet depending on the use case has been a good move for avoiding these issues. The risks of CSV are usually just too high.
- BlueTemplar 7y ago
  Quad-spaced facepalm.... How come these were dropped out of usage ?!?
efficax 7y ago
Seems like you're completely missing the point of the interview question which is to see how someone would approach a problem, investigate its requirements, propose a solution, examine its drawbacks and how they would take feedback on that solution and its possible advantages and disadvantages.
- amznthrowaway5 7y ago
  I did not miss that. I was pointing out that glofish said he asked questions like these repeatedly to candidates, but he doesn't even understand the basics of the subject matter. That would not make for a good interview.

AtlasBarfed 7y ago

Bingo!

buzzkillington 7y ago

The only true Unix column format is ascii delimited text. Oddly no one has heard of that, the only reason why I found out about it is because I had to read in punched tapes with 7 character ascii from an experiment done in the 80s during my undergrad. https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text

amznthrowaway5 7y ago
Fascinating, thanks. In all of the large projects I've worked on, the CSV format variants and their inconsistencies have caused disasters. Who knew that Pandas and Spark handle CSV settings differently by default and spark has a hard time with newlines in its CSV output. CSVs inconsistencies result in a lot of data corruption, doubly so in large teams. Replacing CSV with flat JSON or parquet depending on the use case has been a good move for avoiding these issues. The risks of CSV are usually just too high.
BlueTemplar 7y ago
Quad-spaced facepalm.... How come these were dropped out of usage ?!?

efficax 7y ago

Seems like you're completely missing the point of the interview question which is to see how someone would approach a problem, investigate its requirements, propose a solution, examine its drawbacks and how they would take feedback on that solution and its possible advantages and disadvantages.

amznthrowaway5 7y ago
I did not miss that. I was pointing out that glofish said he asked questions like these repeatedly to candidates, but he doesn't even understand the basics of the subject matter. That would not make for a good interview.