4 ms·
I took Simon's post and pasted it into two different ChatGPT 4o windows. In one, I put "what's wrong with this post, especially with the way it's written?". In
by kordlessagain 2y ago
I took Simon's post and pasted it into two different ChatGPT 4o windows. In one, I put "what's wrong with this post, especially with the way it's written?". In the other, I put "what's really great about this post and what does it do well?".
Both sessions worked toward those implied goals. The one where I asked what was wrong indicated long-winded sentences, redundancy, lack of clear structure, and focus issues. Sorry, Simon!
The one where I asked what was good about the post indicated clarity around misconceptions, good explanations of training, consistent theme, clear sections, logical progression, and focused arguments. I tend to agree with this one more than the other, but clearly the two are conflicted.
So, what's my point?
My point is that AI/LLMs interactions are driven by the human's intent during the interaction. It is literally our own history and training that brings the stateless function responses to where they arrive. Yes, facts and whatnot matter in the training data, and the history, if there is one, but the output is very much dependent on the user.
If we think about this more, it's clear that training data does matter, but likely doesn't matter as much as we think it does. It's probably just as equally important to consider the historic data, and the data coming in from RAG/search processes, as making as big of an impact on the output.
- roywiggins 2y agoIt's at least sometimes a "Clever Hans" effect, or an "ELIZA" effect. These models have been RLHFed to just go along and agree with whatever the user asks for, even if the user doesn't realize what's going on. This is a bit of a cheap trick- of course users will find something more intelligent if it agrees with them. When models start arguing back you get stuff like early Bing Chat going totally deranged.