3 ms·

I wish chatgpt would stop saying things like "here's a no nonsense answer" like maybe just don't include nonsense in the answer?

mabedan 11mo ago
Right? That drives me crazy. It only does that for me in the voice mode. And in cases I ask it to elaborate, it ignores my request and repeats the system instructions from my preferences “ok, I’ll keep it concise” and gives a 5 word answer
- mips_avatar 11mo ago
  It's some kind of shortcut these models are getting in alignment because the base models don't do that stuff
utopiah 11mo ago
Well... that's the whole point, it can not make sense. It's stringing up words based on it's dataset. There is 0 sense making, 0 interpretation, 0 understanding. Words. Strung together, including then it says "no nonsense" because somewhere in its datasets often enough that's the series of words that best match the "stop saying BS!" kind of prompt.
- anuramat 11mo ago
  do you ever get tired of pointing out that a large language model is a language model? UPD I do that as well when explaining to my relatives why I don't care what ChatGPT thinks about $X, but also they're not on HN
  - utopiah 11mo ago
    Worry not, pointing out improper use of language that benefits the biggest corporations on Earth that are destroying the planet is kind of hobby of mine.
    - anuramat 11mo ago
      stylistic preferences are pretty much the ONLY thing you could discuss (in the context of LLMs) that actually has anything to do with (natural) language in the first place; how is having preferences an "improper use of langauge"?
      - utopiah 11mo ago
        I'm not sure I follow. My point is that pretty much everybody who doesn't have a degree in CS or IT assumes due to BigAI corporations that LLMs or GenAI tools think. This is reflected by the words they use. Such people do not say "the model parse my query and process it via it neural network based architecture to give a statistically plausible answer given the context" but rather they say "I had a chat with Claude and he said something useful" thus implying agency and a lot more.
        anuramat 11mo ago
        two questions: 1. do you ever point out that you can't actually mine bitcoin with a pickaxe? 2. what made you think that the parent comment somehow implied that it "actually thinks"?
        utopiah 11mo ago
        Excellent questions, 1. I did actually mine Bitcoins back in the days (back when it was still a cryptoanarchist dream not coopted by the finance industry, scammers and destroying the planet... so a while ago) so I had to explain that too unfortunately. It does highlight a trend that, again, non technical expert take marketing terms at face value. 2. they said "maybe just don't include nonsense in the answer?" which does imply that they believe hallucinations are a side effect that can be solved.
        anuramat 11mo ago
        1. my point is that "thinking" is easier to say than "composition of parameterized nonlinear functions trained by stochastic gradient descent with reinforcement learning on top". misnomer or not, it's not even ambiguous here (unless we're talking CoT vs arbitrary single token) 2. OR they meant that it's violating Gricean maxims; why are you assuming everyone is stupid?
amelius 11mo ago
Maybe you used "Don't give me nonsense" in your custom system prompt?
- Sharlin 11mo ago
  An LLM should never refer to the user's "style" prompt like that. It should function as the model's personality, not something the user asked it to do or be like.
  - mycall 11mo ago
    System prompt is for multi-client/agent applications, so if you wish to fix something for everyone, that is the right place to put it.
- AlwaysRock 11mo ago
  That does nothing. You can add, “say I don’t know if you are not certain or don’t know the answer” and it will never say I don’t know.
  - embedding-shape 11mo ago
    That's because "certain" and "know the answer" has wildly different definitions depending on the person, you need to be more specific about what you actually mean with that. Anything that can be ambiguous, will be treated ambiguously. Anything that you've mentioned in the past (like `no nonsense`) that still exists in context, will have a higher possibility of being generated than other tokens.
svantana 11mo ago
It's analogous to how politicians nowadays are constantly saying "let me be clear", it drives me nuts.
- embedding-shape 11mo ago
  Another annoyance: "In my honest opinion...". Does that mean that you other times are sharing dishonest opinions? Why would you need to declare that this time you're honest?
  - infamouscow 11mo ago
    This has been a pet peeve of mine for years. I call people out when they say this for the abuse of language and for being a time vampire.
  - cindyllm 11mo ago
    [dead]
albert_e 11mo ago
Recently microsoft copilot's (only one that's allowed within our corporate network) replies all have the first section prefixed as "Direct answer:" And after the short direct answer it puts the usual five section blog post style answer with emoji headings
vessenes 11mo ago
Yes, I had total PTSD reading that in the announcement. Whether it's just evolving a tone so that we don't get fatigue or actually improving, I'm happy we're moving on. My audio (still 4o I believe) interactions are maddening - somehow it's remembered I want a quick answer, so EVERY.SINGLE.ANSWER starts with "Okay, let's keep this snappy and info dense." Srsly. Wiping instructions / memory reset seems to have no effect, it comes back almost immediately.
gilfoy 11mo ago
It might actually help output answer with less nonsense. As an example in some workflow I ask chatgpt to figure out if the user is referring to a specific location and output a country in json like { country } It has some error rate at this task. Asking it for a rationale improves this error rate to almost none. { rationale, country }. However reordering the keys like { country, rationale } does not. You get the wrong country and a rationale that justifies the correct one that was not given.
- extr 11mo ago
  This is/was a great trick for improving accuracy of small model + structured output. Kind of an old-fashoined Chain of Thought type of thing. Eg: I used this before with structured outputs in Gemini Flash 2.0 to significantly improve the quality of answers. Not sure if 2.5 Flash requires it, but for 2.0 Flash you could use the propertyOrdering field to force a specific ordering of JSONSchema response items, and force it to output things like "plan", "rationale", "reasoning", etc as the first item, then simply discard it.
- mips_avatar 11mo ago
  But if it’s going to give itself a little pep talk can’t it just do that in a thinking token?

mabedan 11mo ago

Right? That drives me crazy. It only does that for me in the voice mode. And in cases I ask it to elaborate, it ignores my request and repeats the system instructions from my preferences “ok, I’ll keep it concise” and gives a 5 word answer

mips_avatar 11mo ago
It's some kind of shortcut these models are getting in alignment because the base models don't do that stuff

utopiah 11mo ago

Well... that's the whole point, it can not make sense. It's stringing up words based on it's dataset. There is 0 sense making, 0 interpretation, 0 understanding. Words. Strung together, including then it says "no nonsense" because somewhere in its datasets often enough that's the series of words that best match the "stop saying BS!" kind of prompt.

anuramat 11mo ago
do you ever get tired of pointing out that a large language model is a language model? UPD I do that as well when explaining to my relatives why I don't care what ChatGPT thinks about $X, but also they're not on HN
- utopiah 11mo ago
  Worry not, pointing out improper use of language that benefits the biggest corporations on Earth that are destroying the planet is kind of hobby of mine.
  - anuramat 11mo ago
    stylistic preferences are pretty much the ONLY thing you could discuss (in the context of LLMs) that actually has anything to do with (natural) language in the first place; how is having preferences an "improper use of langauge"?
    - utopiah 11mo ago
      I'm not sure I follow. My point is that pretty much everybody who doesn't have a degree in CS or IT assumes due to BigAI corporations that LLMs or GenAI tools think. This is reflected by the words they use. Such people do not say "the model parse my query and process it via it neural network based architecture to give a statistically plausible answer given the context" but rather they say "I had a chat with Claude and he said something useful" thus implying agency and a lot more.
      - anuramat 11mo ago
        two questions: 1. do you ever point out that you can't actually mine bitcoin with a pickaxe? 2. what made you think that the parent comment somehow implied that it "actually thinks"?
        utopiah 11mo ago
        Excellent questions, 1. I did actually mine Bitcoins back in the days (back when it was still a cryptoanarchist dream not coopted by the finance industry, scammers and destroying the planet... so a while ago) so I had to explain that too unfortunately. It does highlight a trend that, again, non technical expert take marketing terms at face value. 2. they said "maybe just don't include nonsense in the answer?" which does imply that they believe hallucinations are a side effect that can be solved.
        anuramat 11mo ago
        1. my point is that "thinking" is easier to say than "composition of parameterized nonlinear functions trained by stochastic gradient descent with reinforcement learning on top". misnomer or not, it's not even ambiguous here (unless we're talking CoT vs arbitrary single token) 2. OR they meant that it's violating Gricean maxims; why are you assuming everyone is stupid?

amelius 11mo ago

Maybe you used "Don't give me nonsense" in your custom system prompt?

Sharlin 11mo ago
An LLM should never refer to the user's "style" prompt like that. It should function as the model's personality, not something the user asked it to do or be like.
- mycall 11mo ago
  System prompt is for multi-client/agent applications, so if you wish to fix something for everyone, that is the right place to put it.
AlwaysRock 11mo ago
That does nothing. You can add, “say I don’t know if you are not certain or don’t know the answer” and it will never say I don’t know.
- embedding-shape 11mo ago
  That's because "certain" and "know the answer" has wildly different definitions depending on the person, you need to be more specific about what you actually mean with that. Anything that can be ambiguous, will be treated ambiguously. Anything that you've mentioned in the past (like `no nonsense`) that still exists in context, will have a higher possibility of being generated than other tokens.

svantana 11mo ago

It's analogous to how politicians nowadays are constantly saying "let me be clear", it drives me nuts.

embedding-shape 11mo ago
Another annoyance: "In my honest opinion...". Does that mean that you other times are sharing dishonest opinions? Why would you need to declare that this time you're honest?
- infamouscow 11mo ago
  This has been a pet peeve of mine for years. I call people out when they say this for the abuse of language and for being a time vampire.
- cindyllm 11mo ago
  [dead]

albert_e 11mo ago

Recently microsoft copilot's (only one that's allowed within our corporate network) replies all have the first section prefixed as "Direct answer:" And after the short direct answer it puts the usual five section blog post style answer with emoji headings

vessenes 11mo ago

Yes, I had total PTSD reading that in the announcement. Whether it's just evolving a tone so that we don't get fatigue or actually improving, I'm happy we're moving on. My audio (still 4o I believe) interactions are maddening - somehow it's remembered I want a quick answer, so EVERY.SINGLE.ANSWER starts with "Okay, let's keep this snappy and info dense." Srsly. Wiping instructions / memory reset seems to have no effect, it comes back almost immediately.

gilfoy 11mo ago

It might actually help output answer with less nonsense. As an example in some workflow I ask chatgpt to figure out if the user is referring to a specific location and output a country in json like { country } It has some error rate at this task. Asking it for a rationale improves this error rate to almost none. { rationale, country }. However reordering the keys like { country, rationale } does not. You get the wrong country and a rationale that justifies the correct one that was not given.

extr 11mo ago
This is/was a great trick for improving accuracy of small model + structured output. Kind of an old-fashoined Chain of Thought type of thing. Eg: I used this before with structured outputs in Gemini Flash 2.0 to significantly improve the quality of answers. Not sure if 2.5 Flash requires it, but for 2.0 Flash you could use the propertyOrdering field to force a specific ordering of JSONSchema response items, and force it to output things like "plan", "rationale", "reasoning", etc as the first item, then simply discard it.
mips_avatar 11mo ago
But if it’s going to give itself a little pep talk can’t it just do that in a thinking token?