Bioinformatics tools are often used in batch processing pipelines, where the expected running time is hours. The ratio of users to developers is usually low eno

Bioinformatics tools are often used in batch processing pipelines, where the expected running time is hours. The ratio of users to developers is usually low enough that trade-offs between developer time and running time remain meaningful. If you have a 50-gigabyte TSV file, you can memory-map, index, and validate it in a few minutes. Overhead like that is usually negligible. And if you encounter something weird like quoted strings, you can just print an error message and abort. It's your file format, so you don't have to make your life difficult by supporting unnecessary features. That approach still kind of works with hundreds of gigabytes of data. Once you have terabytes, text files become too cumbersome. (Here I'm unfortunately speaking from experience.)