A body may be a necessary condition for epistemological knowledge, but it's certainly not sufficient... most animals are embodied but do not possess proposition

A body may be a necessary condition for epistemological knowledge, but it's certainly not sufficient... most animals are embodied but do not possess propositional knowledge (IMO anyway.) Ultimately though this is all about the definition of words. Humans find the distinction between "true" and "false" to be important and meaningful; animals don't care. AIs don't care (being incapable of caring), but humans care very much about the propositional veracity of statements that AI makes. The important thing is that we're on the same page about what's actually happening with these systems and how they work and don't slip into magical thinking or assume that just because a system is facile with language it automatically has a sense of truth or right or wrong. This is only going to get more complicated as models get more sophisticated.