4 ms·
I just asked GPT-4, and it denies being a MoE model. User: Are you an MoE model? ChatGPT: No, I am not based on a Mixture of Experts (MoE) model. My underly
by cschmidt 3y ago
I just asked GPT-4, and it denies being a MoE model.
User:
Are you an MoE model?
ChatGPT:
No, I am not based on a Mixture of Experts (MoE) model. My underlying architecture is based on the GPT (Generative Pre-trained Transformer) framework, specifically the GPT-4 version. This architecture is a large-scale transformer-based neural network, but it does not use the MoE approach.
In a GPT model like mine, the entire model works in a unified manner on each task, without dividing it among specialized sub-models or experts. The model is trained on a diverse range of data and tasks, enabling it to generate responses across a wide array of topics and questions. While this approach lacks the specialization that MoE models offer, it provides broad, general-purpose language understanding and generation capabilities.
- Jensson 3y agoThat is almost surely a hallucination.
- rkagerer 3y agoCurious, why do people gravitate to the term "hallucination" (as if it's a person not a tool) instead of just plainly stating that it's wrong?
- therealdrag0 3y agoIt’s a chat bot so a lot of language will be anthropomorphic. Even still this definition fits well “3. a false belief or impression; illusion; delusion.”
- pacificmint 3y agoIf I ask it what color an orange it and it says blue, that would be wrong. If you ask it a question and it makes up a completely fabricated story, like for example the case files in that recent legal case [1], then saying it was “wrong” doesn’t really seem to capture it. Calling it a hallucination is a great analogy, because the model made up a plausible sounding, but completely fabricated story. It saw things that were never real. [1] https://apnews.com/article/artificial-intelligence-chatgpt-courts-e15023d7e6fdf4f099aa122437dbb59b https://apnews.com/article/artificial-intelligence-chatgpt-c...
- rkagerer 3y agoFair enough and thanks. I felt hallucination was too forgiving a term but I can see how others would rank them the other way around and suppose it works.
- sigilis 3y agoThe more accurate term I’ve seen floating around is “confabulation”. It’s also a human phenomenon, but it doesn’t bring all the baggage of hallucinations.
- esafak 3y agoAsk it to produce a citation for that claim so we can verify it.
- haldujai 3y agoLLMs don’t have insight, these outputs cannot be assumed to be accurate. Due to current hardware limitations it is not feasible to have a 1T parameter model without MoE.
- chlorion 3y agoI wonder if it has some canned human-written responses when asking specific questions about itself. This would be pretty clever to silently implement, that will definitely help convince people that it's approaching "AGI". It's possible that it's just hallucinating here too, I don't have any proof that the responses are canned, but they appear that way to me.
- COAGULOPATH 3y agoYes, it has an invisible "shadow prompt" that gets sent to it when you start a session. It looks something like this: https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0nt5nu https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0n...
- chlorion 3y agoThat makes sense, there has to be some level of human written responses for specific things, like when you ask it about a controversial topic that it refuses to answer, it gives you some canned response about how it won't answer that, and that very obviously isn't what language models do naturally. For these cases there has to be some sort of pre-programmed human generated reply! It's interesting that other people don't seem to agree with the idea that it has some pre-programmed responses, I am curious as to what they think is going on here.
- raverbashing 3y agoIt does not and it cannot answer this kind of meta-question unless explicitly added to its prompt/RAG or whatever. And OpenAI would not add such proprietary information there.
- lxgr 3y agoCan you explain how your own mind works?