There are programs, used for testing and “fuzzing” compilers, that generate random programs based on a seed. When a node runs a random program, it doesn’t know

There are programs, used for testing and “fuzzing” compilers, that generate random programs based on a seed. When a node runs a random program, it doesn’t know if the output is correct. But it could report the seed and result to a central database. Then, if you had a several nodes, and two nodes ran the same seed and got different output, that would mean something was wrong and needed investigation. There are also programs for reducing such programs down to a minimum test case. So once a discrepancy is found, it can be reduced to some small program that recreates it. I once worked on a compiler backend and a CI job generated random C programs and compared x86 output. against the novel cpu simulator. Any discrepancies found were auto reduced by these tools and then a ticket was automatically created. Lots of our bugs were found and fixed this way. (My memory is we used C-reduce for the reductions. I can’t remember the tool we used for generating the test programs, but there are several.)