If you want something that scales horizontally (mostly), dcmp from https://github.com/hpc/mpifileutils https://github.com/hpc/mpifileutils is an option. It can

If you want something that scales horizontally (mostly), dcmp from https://github.com/hpc/mpifileutils https://github.com/hpc/mpifileutils is an option. It can chunk up files and do the comparison in parallel on multiple servers.