> I'm trying to point out there's a difference between not understanding why a model outputs a specific token, and not understand what the computer is doing und

> I'm trying to point out there's a difference between not understanding why a model outputs a specific token, and not understand what the computer is doing under the hood. Whenever there is trust involved, there is no difference to your point. How you're running the model requires trusting that the model, parser, etc isn't compromised and especially if it can be trusted to behave correctly after training - thus transparent explanations rather than hallucinations and its easy to trick and compromise them to do something else. Given that we already don't trust the outputs of these AI models, the above security issue make this even worse and untrustworthy. The plain old regular average joe users do not care about the neural network format, etc and will run and open anything without checking regardless even if it is a text file disguised as a program. Thus, it is entirely no different and we are back to square -1 (with the unexplainable properties of these models that people will try it out and trust its outputs)