5 ms·
I was musing on my website about ChatGPT [1] and one of the things I noticed about it is that if you analogize it to your own brain, which has a language model
by jerf 4y ago
I was musing on my website about ChatGPT [1] and one of the things I noticed about it is that if you analogize it to your own brain, which has a language model in it too, we are collectively asking ChatGPT to do something we would never ask of our own language model. ChatGPT does not have the "higher level" reasoning that we're asking for, to do logic, attribution, and the other things we want.
This is not necessarily a "criticism". It is obvious that ChatGPT produces an incredibly rich and meaning-filled representation of the text, as the extension of the text it can be used for demonstrates. But we should not be trying to "tweak" the language model here and there, trying to make a language model do something it isn't really suited for. We should be working on how to hook up that higher-level functionality. It is not obvious how to do that, true. But it at least stands a chance of producing the AI in reality that people think we have now, but don't.
For that reason, I don't think attribution is coming without a significant addition to the architecture. Language models shouldn't be doing attribution; the mess you'd create of them to accomplish that would ruin them as language models. Can you imagine how insane you'd go if your internal language model tried to stuff down every instance of when you have seen and/or used the word "brain" every time you tried to use the word "brain"? That's not what language models are for. That's what the thing using the language model does. I expect AIs to parallel this. I see no reason at all, and abundant reasons to the contrary, to expect AIs to just be undifferentiated blobs of numbers. I expect them to have structures just as our brains do, and for the exact same reasons. If undifferentiated blobs of identical neurons was the way to go, our brains would work that way too; it's far simpler than what we actually have.
[1]: https://www.jerf.org/iri/post/2023/understanding_gpt_better/ https://www.jerf.org/iri/post/2023/understanding_gpt_better/
- sho_hn 4y agoYep, this resonates heavily with my own thinking. Thanks!
- Jensson 4y agoYou don't remember where you learned the things you know from? How can you trust anything you think then? Can such people get anything done at all, seems basically impossible to learn new things then since you can't know if it is better than what you thought before? So I don't think that remembering sources is a hard problem to solve, likely making the AI work well is probably easier if it remembers sources.
- jerf 4y agoI don't think remembering sources is an intrinsically difficult problem. I think it's hard for language models, and indeed, after further thought, I'm pretty sure they shouldn't remember sources: https://www.jerf.org/iri/post/2023/understanding_gpt_better/ https://www.jerf.org/iri/post/2023/understanding_gpt_better/ But the problem in general is probably not a particular stopper. A bigger problem is people thinking that we should be taking language models and using them as AIs themselves, rather than seeing them as a component of an AI, which the more I think about, the more obvious it is that this is the case. Language models aren't what we want. They're a necessary step to what we do want and I don't expect them to go anywhere. But history will look back and laugh at this misguided attempt to make them be the entire AI. Also, if you are implying you can just spew out the exact sources you learned some fact from... no you can't. You think so only because you haven't tried. Ask someone around you for five random "facts" for you to cite your source from, then try to write on a sheet of paper where you heard this fact from for the first time. Then hand the sheet of paper back to your friend and have them verify your claims... so, note, immediately you have the problem that an unverifiable claim is a fail, so "in elementary school" doesn't cut it any more than you can cite "at the library" as a source on your high school essay. I'm not looking for words that "identify" a several year span of time. You need to give me book, page, and line, or timestamps on a video, or something like that. No, you can't. Nobody can just spit out a bibliographic citation from memory for when they learned that Pluto isn't a planet. I can give you "in the news" for that, sure, but what news source was first? What is the title of that article? Who wrote it? What was the exact date? Of course not.
- telotortium 4y agoSteve Hsu has recently claimed[1] to have started a startup in stealth to solve the hallucination problem over a corpus as large as 10,000 pages of dense, college-level textbooks, to the point where it can answer the end-of-chapter questions in the textbook with almost 100% accuracy (albeit not math questions) - I hope without using the answer key. Not sure if their approach is more robust than Supabase[2] or similar approaches, and no indication of whether it might scale up to something on the order of a search engine corpus, but it's something. [1] https://www.youtube.com/watch?v=peHkL_MaxTU&t=1558s https://www.youtube.com/watch?v=peHkL_MaxTU&t=1558s [2] https://news.ycombinator.com/item?id=34695306 https://news.ycombinator.com/item?id=34695306