That statement slaps of some existential philosophy that will probably need to be addressed at some point... For this context though I agree, we as a society s

That statement slaps of some existential philosophy that will probably need to be addressed at some point... For this context though I agree, we as a society should have the right to inspect any algorithm that makes safety or life altering decisions (looking at you "AI" judges) about your person or in public spaces you might reasonably occupy. I do think data used by an algorithm, or even potentially data that was used to train an algorithm should have a reasonable expectation of privacy. The trained weights do become part of the algorithm in my humble opinion, and that does potentially lead to leakage about the source data, but that doesn't justify not allowing inspection of the algorithm.