3 ms·

Using LLM's in any unfamiliar context is dangerous. When asking them anything, always follow up with: "Are you sure?". Far too often it'll cheerily apologise a

Using LLM's in any unfamiliar context is dangerous. When asking them anything, always follow up with: "Are you sure?". Far too often it'll cheerily apologise and correct their own answer.

jon-wood 1y ago
Often it'll even apologise and correct it's own answer despite the original answer being correct, because you just primed it's model to believe it was wrong.
jononor 1y ago
Not necessarily correct the answer! Sometimes it just changes it, insists that it is correct - but it might still be wrong (perhaps now in a new way)...
klabb3 1y ago
First, these tips and tricks that ”works for me” aren’t universal, it may or may not work, there’s literally no way to tell other than to run large scale empirical experiments, otherwise it’s just mythbuilding. Secondly, it can be even worse. I’ve been ”gaslighted” when pressing on answers I knew were incorrect (in this case, cryptography). It comes up with extremely plausibly sounding arguments, specifically addressing my counterpoints, and even chain of reasoning, yet still isn’t correct. You’d have to be a domain expert to tell it’s wrong, at which point it makes no sense to use LLMs in the first place.
- xnorswap 1y ago
  Oh for sure, asking "Are you sure?" isn't a trick to improve accuracy, it's a trick to show people the danger of asking it in the first place. It just leaves you with two contradictory statements, much like the man with two watches who never knows the correct time.
  - klabb3 1y ago
    Ah, I think I misunderstood the sentiment. Thanks for clarifying
aaronbaugher 1y ago
I asked Grok for advice on a personal decision I was making. It suggested I do A. I said I was leaning toward not-A, and explained why. Then it said I should do not-A. I asked why, if not-A was the best choice, it hadn't said that in the first place. Was it unable to think of not-A before I mentioned it? It said no, it had considered not-A from the start, but thought A was the best choice for me until I gave it more context. That reminded me how important it is to give it the full parameters and context of my question, including things you could assume another human being would just get. It also has a sort of puppy-dog's eagerness to please that I've had to tell it not to let get in the way of objective analysis. Sometimes the "It's awesome that you asked about that" stuff verges on a Hitchhiker's Guide joke. Maybe that's what they were going for.
- daveguy 1y ago
  It should have told you it has no concept of A or not-A, doesn't perform anything close to thinking, and was just picking the most likely words to follow the prompt and context window both times. The people who program it weren't "going for" anything as they couldn't "go for" any specific response. And the model has no concept of self, jokes, or even perception by an outside entity. But it will pick phrases that mimic its training set and those might happen to be about any one of those topics.
zehaeva 1y ago
I wonder how you are to tell when it does give you the correct answer, and one asks "Are you sure?" and it cheerily apologizes and give you a new incorrect answer.
zahlman 1y ago
... But does it even actually try to evaluate the veracity of what it just output? Or is it merely modelling the idea that being asked "Are you sure?" is a reason for self-doubt?
- xnorswap 1y ago
  Do LLMs "evaluate the veracity" of anything? That's not really how they work.

Using LLM's in any unfamiliar context is dangerous. When asking them anything, always follow up with: "Are you sure?". Far too often it'll cheerily apologise and correct their own answer.

jon-wood 1y ago

Often it'll even apologise and correct it's own answer despite the original answer being correct, because you just primed it's model to believe it was wrong.

jononor 1y ago

Not necessarily correct the answer! Sometimes it just changes it, insists that it is correct - but it might still be wrong (perhaps now in a new way)...

klabb3 1y ago

First, these tips and tricks that ”works for me” aren’t universal, it may or may not work, there’s literally no way to tell other than to run large scale empirical experiments, otherwise it’s just mythbuilding. Secondly, it can be even worse. I’ve been ”gaslighted” when pressing on answers I knew were incorrect (in this case, cryptography). It comes up with extremely plausibly sounding arguments, specifically addressing my counterpoints, and even chain of reasoning, yet still isn’t correct. You’d have to be a domain expert to tell it’s wrong, at which point it makes no sense to use LLMs in the first place.

xnorswap 1y ago
Oh for sure, asking "Are you sure?" isn't a trick to improve accuracy, it's a trick to show people the danger of asking it in the first place. It just leaves you with two contradictory statements, much like the man with two watches who never knows the correct time.
- klabb3 1y ago
  Ah, I think I misunderstood the sentiment. Thanks for clarifying

aaronbaugher 1y ago

I asked Grok for advice on a personal decision I was making. It suggested I do A. I said I was leaning toward not-A, and explained why. Then it said I should do not-A. I asked why, if not-A was the best choice, it hadn't said that in the first place. Was it unable to think of not-A before I mentioned it? It said no, it had considered not-A from the start, but thought A was the best choice for me until I gave it more context. That reminded me how important it is to give it the full parameters and context of my question, including things you could assume another human being would just get. It also has a sort of puppy-dog's eagerness to please that I've had to tell it not to let get in the way of objective analysis. Sometimes the "It's awesome that you asked about that" stuff verges on a Hitchhiker's Guide joke. Maybe that's what they were going for.

daveguy 1y ago
It should have told you it has no concept of A or not-A, doesn't perform anything close to thinking, and was just picking the most likely words to follow the prompt and context window both times. The people who program it weren't "going for" anything as they couldn't "go for" any specific response. And the model has no concept of self, jokes, or even perception by an outside entity. But it will pick phrases that mimic its training set and those might happen to be about any one of those topics.

zehaeva 1y ago

I wonder how you are to tell when it does give you the correct answer, and one asks "Are you sure?" and it cheerily apologizes and give you a new incorrect answer.

zahlman 1y ago

... But does it even actually try to evaluate the veracity of what it just output? Or is it merely modelling the idea that being asked "Are you sure?" is a reason for self-doubt?

xnorswap 1y ago
Do LLMs "evaluate the veracity" of anything? That's not really how they work.