3 ms·
> Do you really still want to roll your own code to handle CSV? No one really does, but we get backed into it due to these various issues. The general issue i
by jmull 6y ago
> Do you really still want to roll your own code to handle CSV?
No one really does, but we get backed into it due to these various issues.
The general issue is that there is no real standard for csv, yet it's often used as the touch-point for integration of disparate systems!
The integration goes something like this:
1. A and B decide to integrate! A will accept data from B.
2. A and B want to use a good interchange format, but after weeks of intense negotiations B can only get her IT department to agree to deliver csv (and feels lucky to get that much).
3. B uses excel to provide sample and test files, which are used by A for development or even in early stages of the live integration. Things seem to be going smoothly.
4. At some the export process at B changes, a new csv generator is used and now things break on A's side. Or something on the data input side changes at B so that new forms of data are now present that their csv generator does handle the way A expects, etc.
- 3pt14159 6y agoEh—I had to do it recently. There was a file format that I was dealing with that was basically a combination of CSV for data and Yaml for headers (and some other edge case stuff) and the Python CSV reader couldn't handle some edge case that we needed handling, mostly around reporting back to the file provider where they messed up one thing or another in the file. It didn't really take me that long. Lots of easy to write tests and the thing is processing a ton of data now for a well known stock exchange. Far easier than, say, the machine learning stuff I've written.
- cryptonector 6y agoThe breakage can be subtle and unnoticed for a while, and it can be in the future. It sucks.