I agree, AI output should effectively be treated similarly to untrusted user input. There are some situations where it's ok to use directly, because the consequ

I agree, AI output should effectively be treated similarly to untrusted user input. There are some situations where it's ok to use directly, because the consequences of getting it wrong are minimal. There's other cases where it makes sense with human review. For anything with real consequences, there ought to be the equivalent of an FDA drug trial showing that the trained model works before allowing its use without human review.