3 ms·
Does cut work with quoted fields correctly? My understanding was that it was just a dumb line tokenizer and CSV is a little more complex than that, e.g., CSV ro
by ygra 10y ago
Does cut work with quoted fields correctly? My understanding was that it was just a dumb line tokenizer and CSV is a little more complex than that, e.g., CSV rows can span more than one line when fields contain line breaks.
- rspeer 10y agoWhich means an effective thing to do when you control the data pipeline is to ban tabs and line-breaks from your values, and then use `cut` on tab-separated files.
- Twirrim 10y agoOr just not use CSV as a data transfer format :D
- rspeer 10y agoI think you're missing the point. Unix tools like `cut` are really effective ways to deal with plain text data, who cares if it's CSV.
- ygra 10y agoSure, if you can redefine your data format, you can take shortcuts. But otherwise comparing cut to a CSV parser is just a different way of saying »if it doesn't have to be correct, I can make it as fast as I want«.
- gkst 10y agoNo, it doesn't and therefore is no viable option for many real-world datasets.