I find it funny that we expect AI-du-jour to qualify as equal to human brains when the first has been trained on a slice of content for a bunch of hours and is

I find it funny that we expect AI-du-jour to qualify as equal to human brains when the first has been trained on a slice of content for a bunch of hours and is then getting compared to wetware that's been trained for at least a decade. Recently stuff like ChatGPT is challenged by people pointing out the nonsense it outputs, but it has no way of knowing whether either of its training input or its output is valid or not. I mean one could hack the prompt and make it spit out that fire is cold, but you and I know for a fact that it is nonsense, because at some point we challenged that knowledge by actually putting our hand over a flame. And that's actually what kids do! As a parent you can tell your kid not to do this or that and they will still do it. I can't recall where I read last week that the most terrible thing about parenting is the realisation that they can only learn through pain... which is probably one of the most efficient feedback loops. Copilot is no different, it can spit out broken or nonsensical code in response to a prompt but developers do that all the time, especially beginners because that's part of the learning process, but also experts as well. Yet we somehow expect Copilot to spit out perfect code, and then claim "this AI is lousy!", and while it has been trained with a huge body of work it has never been able to challenge it with a feedback loop. Similarly I'm quite convinced that if I were uploaded everything there is to know about kung fu, I would be utterly unable to actually perform kung fu, nor would I be able to know whether this or that bit that I now know about kung fu is actually correct without trying it. So, I'm not even sure moving goal posts is actually the real problem but only a symptom, because the whole thing seems to me as being debated over the wrong equivalence class.