Isn’t it always hallucinating though? We just don’t care when it gets it “right”. As I understand it, it’s still just probability based on what is likely to be

Isn’t it always hallucinating though? We just don’t care when it gets it “right”. As I understand it, it’s still just probability based on what is likely to be a good set of words to answer the prompt tasking it. It doesn’t actually know anything, it’s just extremely good at making stuff up. Which is still very useful for a lot of things. Just maybe not things to which value is assigned based on how efficient and correct the answer is. Like you can have GPT make a marketing campaign for you, or you can have it design all the icons you need for your application UI, but you can’t reliably make it wrote high performance back-end code without having humans judge the results. Similarly you can’t use it to teach anyone anything, not really, because unless you’re already an expert on the subject being taught, you aren’t likely to spot when it gets things wrong. I guess you can argue that a lot of teaching is flawed like that, and you wouldn’t be wrong. Like, I was taught that the pyramids was build by slave labour, even after the archeological evidence had shown this to be likely false. But our text books were a decade old because our school didn’t really renew them very often… in such a case GPT might have been a more correct teacher, but the trick is that you won’t really know. Which is made even more complicated by the fact that it might teach different things to different students. Like, I just asked ChatGPT 3.5 who build the pyramids in 3 different prompts, in one it told me it was ordinary people. In the others it told me it was mostly skilled labour under guidance of “architects” and “engineers”. Still better than teaching us it was done by slave labour like my old book, but the book was still consistent in what was considered to be the truth at the time.