It isn't trivial actually. Detection would be a multipass process, the first pass would be a measurement of entropy, which would also flag compressed data (whi

It isn't trivial actually. Detection would be a multipass process, the first pass would be a measurement of entropy, which would also flag compressed data (which would be very common). The second pass being magic number detection (file headers, can't be trusted). The third pass would likely be some sort of conformance test to every know compression format. The result of all this io intensive processing would not be a definitive answer. If you want to get a taste of how difficult it would be to implement some method of detection, with a useful detection rate at scale, try to write some software that can detect localization encoding of strings on a binary stream. There is a reason why automatic translation of unknown localization doesn't really exist, there have been plenty of attempts, but nothing that results in a definitive answer. I once wrote some software that would parse very poorly written csv files, it was one of the most frustrating projects I've ever done - and taught me the value of standards conformance. Basically, if the data source is non-cooperative, you aren't going to be able to reliably use the data.