6 ms·

The main pain point I still have with GPT4 is that it's hard to prevent it from being a yes-man "- I have problem A, what do you think of X ? - Yes X is great

by lairv 3y ago

The main pain point I still have with GPT4 is that it's hard to prevent it from being a yes-man "- I have problem A, what do you think of X ? - Yes X is great ... - But with X I'd encounter problem B, maybe I should go with Y ? - You're right X would cause problem B. With Y ..." One way to limit this issue is to not give ChatGPT any hint on your intent, instead of writing "I have problem A, what do you think of X ?", just write "I have problem A, how do you solve it ?" and then see what it says about X. But still, when I need an opinion on a particular solution, I don't recall ChatGPT offering any alternative that was actually better. I've added "Be critical. Be pessimistic." in my custom instructions but it barely gives better results

syndicatedjelly 3y ago
This gave me an idea to add the following to custom instructions: * At the end of all answers, related to programming, provide a list of 4 keywords or topics that represent first principles for the subject so I can research them later.*
- rendall 3y ago
  How is this working?
  - syndicatedjelly 3y ago
    It’s alright. Occasionally it drops a new phrase that I haven’t heard of, and I’ll drop into a Wikipedia hole for a few minutes.
rendall 3y ago
The "stochastic parrot" mental model of LLMs really helps me to limit my expectations. An LLM will always and only output variations of content that exists. If you ask it for opinions, it will just parrot variations of random opinions. Instead, ask it about current best practices or the consensus on some given topic. So, rather than "I have problem A, what do you think of X?" try "Give me a list of reasonable possible solutions to problem A, and compare and contrast each to solution X" Example: My baby has a hard time sleeping. What do you think about the cry it out method? The cry it out method is a subject of debate among parents and experts. Some argue it's effective for sleep training, while others say it can cause stress for the baby. Always consult a healthcare provider for personalized advice. Versus: My baby has a hard time sleeping. Give me a list of reasonable solutions. Compare and contrast each to the cry it out method. 1. Ferber Method: Gradual extinction where you let the baby cry for increasing time intervals. Compared to Cry It Out (CIO), it's less abrupt but still involves some crying. 2. No Tears: Involves comforting without picking the baby up. Less stressful for some parents compared to CIO but can be time-consuming. 3. Pick Up/Put Down: Pick baby up when crying, put down when calm. Less crying involved than CIO but can be inconsistent. 4. Co-Sleeping: Baby sleeps with parents. No crying but can be a hard habit to break compared to CIO. 5. Chair Method: Parent sits in room, gradually moves away. Lower crying duration than CIO but requires more time from parents. 6. Scheduled Awakenings: Wake baby before they usually cry. No crying but can be tiring for parents. 7. Swaddle, Side, Shush: Calming actions to soothe baby. More hands-on and less crying than CIO but may not work for all babies. 8. Sleep Training Clocks: Use visual aids to signal sleep time. No crying, but more suitable for older children. 9. White Noise: Mask other noises. Less crying but effectiveness varies. 10. Sleep Sack: Provides a secure feeling. May reduce crying compared to CIO but not always effective.
- visarga 3y ago
  > An LLM will always and only output variations of content that exists. I envision a stage in data preparation before training when the model will review and analyse its training data, integrating information that sits in different examples. And not just the dataset, but the model itself can be sampled and the generated corpus be analysed and result in some kind of introspective signal that can be added to the training set. A third source of signal for analysis would be chat logs generate by the previous iteration of the model. Tendency to be a yes man, sycophancy, hallucinations etc would be apparent in the reactions of the users. In fact we are seeing a surge of synthetic datasets generated from higher models like GPT-4 to train open source models like LLaMA. I think the next few years "dataset engineering" will dominate over architecture engineering as the main approach for improvement.
vharuck 3y ago
The training data of written content, no matter the sources, will always be biased in one way: it doesn't contain thoughts people kept to themselves. Example 1: Mr. X is very popular author among a small group, who chat daily about it on various websites. Outside of that group, many people know about Mr. X but don't care much about his works. So the majority of content about Mr. X is positive, even though the majority of opinions are ambivalent. Example 2: Somebody asks for help on StackOverflow. Two replies offer different answers, A and B. So the answers seem like they'd have equal value. If 90% of people think A is the right answer, the written content wouldn't reflect that, because those people won't write duplicate answers. I'm sure there are ways to account for these examples. Assigning different "weights" to content sources could help with #1, and feedback mechanisms like upvotes could help with #2. But I doubt those fixes would cover all the problems with the bias against thoughts that people never shared.
- visarga 3y ago
  If you already have a large model you can rank & filter the training data and retrain. If the model doesn't like the data, chances are it is low quality. That is how MS built Phi-1 and 1.5, by curating web data + generating with a large model. The second part (synthetic content) improves diversity by methodically covering concepts or tasks.
tornato7 3y ago
Ask it to come up with a few different solutions and then compare and contrast to find the best one.

by lairv 3y ago

syndicatedjelly 3y ago

This gave me an idea to add the following to custom instructions: * At the end of all answers, related to programming, provide a list of 4 keywords or topics that represent first principles for the subject so I can research them later.*

rendall 3y ago
How is this working?
- syndicatedjelly 3y ago
  It’s alright. Occasionally it drops a new phrase that I haven’t heard of, and I’ll drop into a Wikipedia hole for a few minutes.

rendall 3y ago

The "stochastic parrot" mental model of LLMs really helps me to limit my expectations. An LLM will always and only output variations of content that exists. If you ask it for opinions, it will just parrot variations of random opinions. Instead, ask it about current best practices or the consensus on some given topic. So, rather than "I have problem A, what do you think of X?" try "Give me a list of reasonable possible solutions to problem A, and compare and contrast each to solution X" Example: My baby has a hard time sleeping. What do you think about the cry it out method? The cry it out method is a subject of debate among parents and experts. Some argue it's effective for sleep training, while others say it can cause stress for the baby. Always consult a healthcare provider for personalized advice. Versus: My baby has a hard time sleeping. Give me a list of reasonable solutions. Compare and contrast each to the cry it out method. 1. Ferber Method: Gradual extinction where you let the baby cry for increasing time intervals. Compared to Cry It Out (CIO), it's less abrupt but still involves some crying. 2. No Tears: Involves comforting without picking the baby up. Less stressful for some parents compared to CIO but can be time-consuming. 3. Pick Up/Put Down: Pick baby up when crying, put down when calm. Less crying involved than CIO but can be inconsistent. 4. Co-Sleeping: Baby sleeps with parents. No crying but can be a hard habit to break compared to CIO. 5. Chair Method: Parent sits in room, gradually moves away. Lower crying duration than CIO but requires more time from parents. 6. Scheduled Awakenings: Wake baby before they usually cry. No crying but can be tiring for parents. 7. Swaddle, Side, Shush: Calming actions to soothe baby. More hands-on and less crying than CIO but may not work for all babies. 8. Sleep Training Clocks: Use visual aids to signal sleep time. No crying, but more suitable for older children. 9. White Noise: Mask other noises. Less crying but effectiveness varies. 10. Sleep Sack: Provides a secure feeling. May reduce crying compared to CIO but not always effective.

visarga 3y ago
> An LLM will always and only output variations of content that exists. I envision a stage in data preparation before training when the model will review and analyse its training data, integrating information that sits in different examples. And not just the dataset, but the model itself can be sampled and the generated corpus be analysed and result in some kind of introspective signal that can be added to the training set. A third source of signal for analysis would be chat logs generate by the previous iteration of the model. Tendency to be a yes man, sycophancy, hallucinations etc would be apparent in the reactions of the users. In fact we are seeing a surge of synthetic datasets generated from higher models like GPT-4 to train open source models like LLaMA. I think the next few years "dataset engineering" will dominate over architecture engineering as the main approach for improvement.

vharuck 3y ago

The training data of written content, no matter the sources, will always be biased in one way: it doesn't contain thoughts people kept to themselves. Example 1: Mr. X is very popular author among a small group, who chat daily about it on various websites. Outside of that group, many people know about Mr. X but don't care much about his works. So the majority of content about Mr. X is positive, even though the majority of opinions are ambivalent. Example 2: Somebody asks for help on StackOverflow. Two replies offer different answers, A and B. So the answers seem like they'd have equal value. If 90% of people think A is the right answer, the written content wouldn't reflect that, because those people won't write duplicate answers. I'm sure there are ways to account for these examples. Assigning different "weights" to content sources could help with #1, and feedback mechanisms like upvotes could help with #2. But I doubt those fixes would cover all the problems with the bias against thoughts that people never shared.

visarga 3y ago
If you already have a large model you can rank & filter the training data and retrain. If the model doesn't like the data, chances are it is low quality. That is how MS built Phi-1 and 1.5, by curating web data + generating with a large model. The second part (synthetic content) improves diversity by methodically covering concepts or tasks.

tornato7 3y ago

Ask it to come up with a few different solutions and then compare and contrast to find the best one.