Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture whe

Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture where we measure success of a new varietal on output and resource use and have a very dim understanding of the inner workings that might have led to those observables. Or to take it further, you hire a consultant to produce some work in a given amount of time. You accept the work over time, you don’t feel the need to ask the consultant to turn over all their notes or to plug their brain into a mind reader machine to vet the source. They could indeed be lying and take whatever they learned about your company elsewhere, but we have no protection over that possibility lacking a mind reading instrument today. And society also doesn’t collapse today given that risk.