10 ms·
Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get m
by kromem 3y ago
Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat."
It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
- anon373839 3y agoIs there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
- kromem 3y agoThere's a lot of evidence that users use those techniques for "prompt engineering." So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
- anon373839 3y agoSorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.