It's a silly question to ask as the models can't reason about anything and know nothing about the world. They just return statistically relevant output to inpu

It's a silly question to ask as the models can't reason about anything and know nothing about the world. They just return statistically relevant output to input and mash the output in such a way that there is no way to know if there ever was a corresponding human input or if it was completely mashed together from non-related inputs making it absolutely false.