Consider, it's perfectly possible for a file to fit two or more file formats - polyglot files are a hobby for some people. And there are also a billion formats

Consider, it's perfectly possible for a file to fit two or more file formats - polyglot files are a hobby for some people. And there are also a billion formats that are not uniquely determined by magic bytes. You don't have to go further than text files.