> It's not that hard to pre process prior to feeding into whatever the latest processing architecture of the decade is .. but it's hell week when you're tasked

> It's not that hard to pre process prior to feeding into whatever the latest processing architecture of the decade is .. but it's hell week when you're tasked with decoding some decades old non standard "only used for 18 months" "best practice at the time" obscure compression format data. That's quite disingenuous, there are very well documented data formats that have been around for decades (netCDF, HDF) those are not "obscure" formats and are infinitesimally better than text. > Field seperated line orientated ASCII data doesn't require extensive out of band notes to understand or decipher years after you've passed on. Field separated ACII data is terrible because it does not contain relevant metadata (except for column names), you quite possibly loose precision (also what even was the precision?), are slow to read, need to be additionally compressed ...