7 ms·
What Is ChatGPT Doing and Why Does It Work? (2023)
- deleted 2y ago[deleted]
- dtagames 2y agoThis is a terrific deep-dive from Stephen Wolfram that explains not only how ChatGPT and other LLM applications work "under the hood," but also how they can give rise to logic and grammar -- but notably not math or computation.
- pungentsound 2y ago[dead]
- deleted 2y ago[deleted]
- sidcool 2y agoPlease add 2023 tag to the title.
- mg 2y agoThe better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is logically strange here. Newer models still sometimes do, but more often than not catch it. GPT-4o for example: There is a slight inconsistency in the narrative. The text states that Joe drank tea with Sue in the kitchen after driving her to university, which implies that Sue is at home, contradicting the earlier statement that she was driven to university. Surely nothing in the prompt directly triggered the word "inconsistency". Did the model form some kind of "world model" in its inner layers in which it knows about a person "Sue" who is at a location called "university" after the first sentence?
- abalaji 2y agoMy guess is that this is a post training (rlhf) artifact on world model prompts. There were likely many “logical inconsistency” prompts which humans coerced to the above response.
- brcmthrowaway 2y agoIs RLHF same as fine tuning?
- flebron 2y agoNo. In the common use of the word fine-tuning, one is in the supervised learning scenario. One has an input prompt, and an output sentence. One teaches the model to say that output in response to that prompt. In the reinforcement learning scenario, one has a prompt, and a way of rewarding the model for different outputs. One can have, for instance, a reward model, that assigns a reward for a given model output. One could also have a pairwise reward model, where the learner is sampled with that prompt twice (with different RNGs), and the reward model gives a reward based on the better of the two samples. You could also have humans give these pointwise or pairwise rewards. In essence, one is not telling the model "This. This is what you should output next time." but rather "I liked this reply. Have a cookie." The behaviors that you can learn in RL are more subtle, but you get a lot less information per step. That's because, in a causal language modeling objective, when I tell you "For the prompt X, you should output exactly Y[0...m)", you get a gradient for P(Y[0] | X), another one for P(Y[1] | X Y[0..1)), another for P(Y[2] | X Y[0..2)), another for P(Y[3] | X Y[0..3)), and so on. It's a lot more of a step-by-step guidance, than it is a sentence-wise reward that you get in the RL framework. In RL, I'd give you a cookie for P(Y | X). What part of Y made me give you that cookie? Was there even such a part? Was it perhaps some internal representation that made everything in Y better? That's for the model to learn.
- jacobr1 2y agoOne wrinkle, is that it is now common to fine-tune on previously derived RL datasets, with the tested inputs and preferred sample outputs as the training data.
- Anon84 2y agoExcellent deep dive (and surprisingly not self-referential). I read the dead tree version (https://amzn.to/4cs5EDA https://amzn.to/4cs5EDA) when it first came out.
- dtagames 2y agoYou're right! I'm used to his articles being full of self promotion and Wolfram-talk, but this one is not.
- Anon84 2y agoI was pleasantly surprised, to say the least.
- y-c-o-m-b 2y agoI saw Stephen Wolfram in the URL and hesitated to click at first for that exact reason. This turned out to be a fantastic write-up though, so I'm glad I gave it a chance.
- justincredible 2y ago[dead]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- ChrisArchitect 2y ago(2023) Discussion then: https://news.ycombinator.com/item?id=34796611 https://news.ycombinator.com/item?id=34796611
- ChrisArchitect 2y agoAssociated video: https://www.youtube.com/watch?v=flXrLGPY3SU https://www.youtube.com/watch?v=flXrLGPY3SU
- mitchbob 2y agoEarlier discussion (499 comments): https://news.ycombinator.com/item?id=34796611 https://news.ycombinator.com/item?id=34796611