3 ms·
LLMs don’t have insight, these outputs cannot be assumed to be accurate. Due to current hardware limitations it is not feasible to have a 1T parameter model wi
by haldujai 3y ago
LLMs don’t have insight, these outputs cannot be assumed to be accurate.
Due to current hardware limitations it is not feasible to have a 1T parameter model without MoE.
- chlorion 3y agoI wonder if it has some canned human-written responses when asking specific questions about itself. This would be pretty clever to silently implement, that will definitely help convince people that it's approaching "AGI". It's possible that it's just hallucinating here too, I don't have any proof that the responses are canned, but they appear that way to me.
- COAGULOPATH 3y agoYes, it has an invisible "shadow prompt" that gets sent to it when you start a session. It looks something like this: https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0nt5nu https://www.reddit.com/r/ChatGPT/comments/zo9of4/comment/j0n...
- chlorion 3y agoThat makes sense, there has to be some level of human written responses for specific things, like when you ask it about a controversial topic that it refuses to answer, it gives you some canned response about how it won't answer that, and that very obviously isn't what language models do naturally. For these cases there has to be some sort of pre-programmed human generated reply! It's interesting that other people don't seem to agree with the idea that it has some pre-programmed responses, I am curious as to what they think is going on here.