There's this system, which has become extremely popular for processing data (Apache Spark). And its default configuration cannot parse CSVs properly and can lea

There's this system, which has become extremely popular for processing data (Apache Spark). And its default configuration cannot parse CSVs properly and can lead to pretty severe data loss. I wrote about this a while ago and it's still an issue. (Yes, we did lose data his way, it's not a theoretical exercise.) https://kokes.github.io/blog/2019/07/09/losing-data-apache-spark.html#csvs-dont-have-a-standard-per-se-but-lets-be-reasonable https://kokes.github.io/blog/2019/07/09/losing-data-apache-s...