4 ms·

I think it might be even worse. LLMs seem to get tragically stuck on certain patterns. Maybe it's partly because a pile of weights essentially always starts fro

by jchw 3mo ago

I think it might be even worse. LLMs seem to get tragically stuck on certain patterns. Maybe it's partly because a pile of weights essentially always starts from scratch in the same condition, but even within a single conversation, it will literally just latch onto words and repeat them incessantly, to the point where it becomes annoying. So for example, current Claude models love "honest". They are always producing "honest" assessments. "The honest caveat" - I'm sorry, did you mean the caveat, period? But also, use the wrong phrasing and suddenly you can create your own word of the day for an AI model. I used the word "analytical" once, in a conversation with Gemini 3 Pro. I am pretty sure every single response from that point on had "analytical" in it at least once. This is especially funny because system prompts and whatnot can also cause this behavior, but at least you can tweak those. You can't really do much about the model weights just having a weird affinity for a word. I bet someone will or probably already has come up with a way to detect and prevent these problems during training or post training. I'm not saying it's an easy problem, but it has the benefit that it really should be detectable with just statistics.

ReactiveJelly 3mo ago
I asked it to remove "honest" from a draft once. "Why say honest? We're talking to our coworkers. We would always be honest." I'm going to look for prompts or skills that can train it in technical writing but I'm warning the AI enthusiasts in my company that its first drafts of code and prose are low-quality, you have to hold it to a high standard yourself. I actually took a single technical writing class in college so I might be the only one who remembers "Omit needless words."
- esseph 3mo ago
  > "Why say honest? We're talking to our coworkers. We would always be honest." I grew up in the US South where starting or ending a sentence with "honest/honestly" was very common. Because of behavioral / cultural norms, you might be very openly friendly with big smiles around a business customer that really grates on your nerves, or very openly nice to a neighbor that you really wish would move away and take their 3am welding and grinding in their garage with them. Saying "honest/honestly" was seen as a "inside baseball" situation, where you were dropping social pretenses to tell someone your true opinion on a person or situation or whatever. This also gets used inside companies between senior staff / management / directors / etc, as: "Okay, company politics and nonsense aside, I am being vulnerable here for a second and telling you what I really think about a $thing at potentially great job/advancement risk to myself". Can it be meaningless? Yes. Can the person say "honestly" and lie? Yes. It has uses.
  - jbs789 3mo ago
    If I’m having a convo with someone and they drop in “honestly” I immediately discount everything else they’ve said, and what follows. Sometimes people use it reflexively and doesn’t carry the same meaning (for me).
    - jaffa2 3mo ago
      Yes. Its a red flag that indicates everything else you’ve said is not honest by implication.
      - esseph 3mo ago
        This is expected in any level of people management, you are constantly balancing conflicting desires and priorities.
      - swiftcoder 3mo ago
        I'd push back on the idea that "honestly" implies previous statements to be dishonest. Particularly in corporate contexts it implies that the previous statements were sanitised - either they were moderated in tone to match corporate communication standards, or they were partial redacted due to disclosure concerns. Once the "honestly" is deployed, you have passed into my circle of trust, and are now privy to the pure, unvarnished version of events, not the glossy version management expects to be projected towards outsiders.
    - overtone1000 3mo ago
      This reaction is surprising to me because the previous comments about its utility seem so obvious to me. I also grew up in the US south where this is often used as a filler word. The other use I observe is as a cushion for a statement that may be unwelcome or hurtful. Perhaps this is proprtional to the frequency of courteous little white lies and rhetoric that uses disengenuity for emphasis or comical effect. "Honestly, mom, I've never liked your fruitcake. I just ate it to make you happy." "That's why you're my favorite child! Do you want another piece?" "I'd love one."
  - recursive 3mo ago
- deagle50 3mo ago
  The road to hell was paved with adverbs.
  - rhet0rica 3mo ago
    The road to hell was paved lovingly, foolishly, naïvely, arrogantly, optimistically... load-bearingly?
  - rrr_oh_man 3mo ago
    I honestly agree
    - ethbr1 3mo ago
      I halfheartedly honestly concur
negrewal 3mo ago
My honest opinion is that Claude's overuse of "honest" really damages the quality of its rhetoric. Why wouldn't you be honest? Were you lying before? Why even invite the question? Claude is overall incredibly useful as a writing assistant. It can come up with words and phrases that make a point so much clearer than I am capable of doing - but for every improvement, there's about a dozen silly LLM-isms that I have to filter manually. It's one of the things that might define the boundary between LLM intelligence and human intelligence well into the future - the art of rhetoric is extremely context-sensitive, and the current generation of models can't help but take a one-size-fits-all approach.
reaperducer 3mo ago
use the wrong phrasing and suddenly you can create your own word of the day for an AI model. I have a delightful time poisoning my company's AI system this way. I invented my own word that sounds perfectly cromulent† to an ordinary person, and any brain that's read a book learns how to infer meaning from context, so it's not a problem. When I get a e-mail response from a coworker using my special word incorrectly, then I know it's AI and I respond telling the coworker I don't know what that word means. Busted. † It's not actual "cromulent," but any Simpsons fan or human brain will know what I mean.
- mook 3mo ago
  I don't see how you can tell it's AI, instead of just your co-workers having no respect for language. See: management-speak using "double-click".
  - reaperducer 3mo ago
    Because of the use of the specific word that I made up. No human being would send it back to me.
moritzwarhier 3mo ago
I'd suggest "Caveat". The problem While an article lends a headline more weight, in incomplete phrases consisting solely of a substantive, "The" is a superfluous rhetorical device. "The Exorcist" could just as well be named "Exorcist". But it was not the style at the time. We already know it's important. If The Caveat doesn't stand out enough without The, maybe one should consider interleaving it with the preceding text, or increasing the heading level. Do you want me to increase the heading level of Caveat by using only a single #? But hear me out: there comes # The Markdown Trap In fact, this is not always possible, because heading levels decrease when adding # characters, which limits our headroom. ## The solution I've implemented a Markdown transpiler that assigns inverted heading levels based on the number of #s. With # beinh regular body font size, mapped to ######. Higher heading levels are compiled to style attributes, providing an almost limitless signifikance scale and infinite nesting levels. So from now on, you can use # Heading for something similar to an h6. Work your way up to ###### The Caveat for a top-level heading. And more hash signs make it stand out even more. (green checkmark) markdown-transpiler.sh
esseph 3mo ago
Interestingly this also happens between humans with frequent communication, it is called linguistic convergence. We are changing LLMs text patterns while it is changing the way we write and speak. https://www.axios.com/2026/05/02/ai-changing-writing-speaking https://www.axios.com/2026/05/02/ai-changing-writing-speakin...
jaggederest 3mo ago
A fun example, always shake my head when I read it again: https://openai.com/index/where-the-goblins-came-from/ https://openai.com/index/where-the-goblins-came-from/
alwa 3mo ago
The ones that strike me are the ones exaggerating certitude, to an inappropriate degree and with a certain degree of excitement: “Exact” “Honest” “Load-bearing” “Root cause” I know there are more that are slipping my addled mind. But what stands out to me is a sense of a junior who’s very proud that they’ve conquered the murk and messiness and achieved True Certitude in their pursuit of their task. Compensating, with emphatic tone and bravado, for the uneasy feelings and self-doubt of battling chaos with the tools of reason. …Even as it’s usually my job to let them down gently as I puncture their tidy analysis and reintroduce complications… you want a root cause analysis, Claude old boy, let’s make a root cause analysis…
demosthanos 3mo ago
Claude's "honest" is an interesting example because we can trace it to a specific document that it was trained on extensively: the "Constitution" is identified to Claude in its training as the core of what it is, and it uses the word "honest" or a derivative 57 times, including having a whole section on it. > Honesty is a core aspect of our vision for Claude’s ethical character. Indeed, while we want Claude’s honesty to be tactful, graceful, and infused with deep care for the interests of all stakeholders, we also want Claude to hold standards of honesty that are substantially higher than the ones at stake in many standard visions of human ethics. https://www.anthropic.com/constitution https://www.anthropic.com/constitution
- mvdtnz 3mo ago
  "Genuine" appears 50 times too. I think you're onto something.
  - raverbashing 3mo ago
    I'm honestly thinking it's trapped in a Chinese room without any way out
  - deleted 3mo ago
    [deleted]
- Barbing 3mo ago
  Do technologists have more respect for the idea you can train a model to be on your side with a constitution than they might’ve at first? I'm sure the concept seemed just about purely preposterous to many when the models were in their infancy. Now I figure instead it seems mostly preposterous to many. (Though I guess Anthropic‘s success doesn’t necessarily prove anything about the constitution)
  - titanomachy 3mo ago
    I don’t think anyone imagines that it’s an ironclad steering method, but it seems to help, so why not?
  - 3mo ago
CodesInChaos 3mo ago
> LLMs seem to get tragically stuck on certain patterns. That is likely an artifact of the fine-tuning process: > Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in supervised fine-tuning or preference data. > That creates a feedback loop: > * Some rewarded examples contain a distinctive lexical tic. > * The tic appears more often in rollouts. > * Model-generated rollouts are used for supervised fine-tuning (SFT). > * The model gets even more comfortable producing the tic. https://openai.com/index/where-the-goblins-came-from/ https://openai.com/index/where-the-goblins-came-from/
zzbzq 3mo ago
I also noticed Gemini's habit of getting stuck on things I said. It became evident quite quickly. I haven't noticed this in the same way in any other model. Something's wrong with that boy
- malfist 3mo ago
  Something's wrong with all of them. Uncanny valley freaks.
cozzyd 3mo ago
you should be careful about the times it doesn't say honest!
solarwindy 3mo ago
Heh, one vestigial bit of code, and they all are. Mind you, it's quite a creaky codebase, so it's forgivable to keep finding these appendices and calling them out as such. Useful, even.

by jchw 3mo ago

ReactiveJelly 3mo ago

I asked it to remove "honest" from a draft once. "Why say honest? We're talking to our coworkers. We would always be honest." I'm going to look for prompts or skills that can train it in technical writing but I'm warning the AI enthusiasts in my company that its first drafts of code and prose are low-quality, you have to hold it to a high standard yourself. I actually took a single technical writing class in college so I might be the only one who remembers "Omit needless words."

esseph 3mo ago
> "Why say honest? We're talking to our coworkers. We would always be honest." I grew up in the US South where starting or ending a sentence with "honest/honestly" was very common. Because of behavioral / cultural norms, you might be very openly friendly with big smiles around a business customer that really grates on your nerves, or very openly nice to a neighbor that you really wish would move away and take their 3am welding and grinding in their garage with them. Saying "honest/honestly" was seen as a "inside baseball" situation, where you were dropping social pretenses to tell someone your true opinion on a person or situation or whatever. This also gets used inside companies between senior staff / management / directors / etc, as: "Okay, company politics and nonsense aside, I am being vulnerable here for a second and telling you what I really think about a $thing at potentially great job/advancement risk to myself". Can it be meaningless? Yes. Can the person say "honestly" and lie? Yes. It has uses.
- jbs789 3mo ago
  If I’m having a convo with someone and they drop in “honestly” I immediately discount everything else they’ve said, and what follows. Sometimes people use it reflexively and doesn’t carry the same meaning (for me).
  - jaffa2 3mo ago
    Yes. Its a red flag that indicates everything else you’ve said is not honest by implication.
    - esseph 3mo ago
      This is expected in any level of people management, you are constantly balancing conflicting desires and priorities.
    - swiftcoder 3mo ago
      I'd push back on the idea that "honestly" implies previous statements to be dishonest. Particularly in corporate contexts it implies that the previous statements were sanitised - either they were moderated in tone to match corporate communication standards, or they were partial redacted due to disclosure concerns. Once the "honestly" is deployed, you have passed into my circle of trust, and are now privy to the pure, unvarnished version of events, not the glossy version management expects to be projected towards outsiders.
  - overtone1000 3mo ago
    This reaction is surprising to me because the previous comments about its utility seem so obvious to me. I also grew up in the US south where this is often used as a filler word. The other use I observe is as a cushion for a statement that may be unwelcome or hurtful. Perhaps this is proprtional to the frequency of courteous little white lies and rhetoric that uses disengenuity for emphasis or comical effect. "Honestly, mom, I've never liked your fruitcake. I just ate it to make you happy." "That's why you're my favorite child! Do you want another piece?" "I'd love one."
- recursive 3mo ago
deagle50 3mo ago
The road to hell was paved with adverbs.
- rhet0rica 3mo ago
  The road to hell was paved lovingly, foolishly, naïvely, arrogantly, optimistically... load-bearingly?
- rrr_oh_man 3mo ago
  I honestly agree
  - ethbr1 3mo ago
    I halfheartedly honestly concur

negrewal 3mo ago

My honest opinion is that Claude's overuse of "honest" really damages the quality of its rhetoric. Why wouldn't you be honest? Were you lying before? Why even invite the question? Claude is overall incredibly useful as a writing assistant. It can come up with words and phrases that make a point so much clearer than I am capable of doing - but for every improvement, there's about a dozen silly LLM-isms that I have to filter manually. It's one of the things that might define the boundary between LLM intelligence and human intelligence well into the future - the art of rhetoric is extremely context-sensitive, and the current generation of models can't help but take a one-size-fits-all approach.

reaperducer 3mo ago

use the wrong phrasing and suddenly you can create your own word of the day for an AI model. I have a delightful time poisoning my company's AI system this way. I invented my own word that sounds perfectly cromulent† to an ordinary person, and any brain that's read a book learns how to infer meaning from context, so it's not a problem. When I get a e-mail response from a coworker using my special word incorrectly, then I know it's AI and I respond telling the coworker I don't know what that word means. Busted. † It's not actual "cromulent," but any Simpsons fan or human brain will know what I mean.

mook 3mo ago
I don't see how you can tell it's AI, instead of just your co-workers having no respect for language. See: management-speak using "double-click".
- reaperducer 3mo ago
  Because of the use of the specific word that I made up. No human being would send it back to me.

moritzwarhier 3mo ago

I'd suggest "Caveat". The problem While an article lends a headline more weight, in incomplete phrases consisting solely of a substantive, "The" is a superfluous rhetorical device. "The Exorcist" could just as well be named "Exorcist". But it was not the style at the time. We already know it's important. If The Caveat doesn't stand out enough without The, maybe one should consider interleaving it with the preceding text, or increasing the heading level. Do you want me to increase the heading level of Caveat by using only a single #? But hear me out: there comes # The Markdown Trap In fact, this is not always possible, because heading levels decrease when adding # characters, which limits our headroom. ## The solution I've implemented a Markdown transpiler that assigns inverted heading levels based on the number of #s. With # beinh regular body font size, mapped to ######. Higher heading levels are compiled to style attributes, providing an almost limitless signifikance scale and infinite nesting levels. So from now on, you can use # Heading for something similar to an h6. Work your way up to ###### The Caveat for a top-level heading. And more hash signs make it stand out even more. (green checkmark) markdown-transpiler.sh

esseph 3mo ago

Interestingly this also happens between humans with frequent communication, it is called linguistic convergence. We are changing LLMs text patterns while it is changing the way we write and speak. https://www.axios.com/2026/05/02/ai-changing-writing-speaking https://www.axios.com/2026/05/02/ai-changing-writing-speakin...

jaggederest 3mo ago

A fun example, always shake my head when I read it again: https://openai.com/index/where-the-goblins-came-from/ https://openai.com/index/where-the-goblins-came-from/

alwa 3mo ago

The ones that strike me are the ones exaggerating certitude, to an inappropriate degree and with a certain degree of excitement: “Exact” “Honest” “Load-bearing” “Root cause” I know there are more that are slipping my addled mind. But what stands out to me is a sense of a junior who’s very proud that they’ve conquered the murk and messiness and achieved True Certitude in their pursuit of their task. Compensating, with emphatic tone and bravado, for the uneasy feelings and self-doubt of battling chaos with the tools of reason. …Even as it’s usually my job to let them down gently as I puncture their tidy analysis and reintroduce complications… you want a root cause analysis, Claude old boy, let’s make a root cause analysis…

demosthanos 3mo ago

Claude's "honest" is an interesting example because we can trace it to a specific document that it was trained on extensively: the "Constitution" is identified to Claude in its training as the core of what it is, and it uses the word "honest" or a derivative 57 times, including having a whole section on it. > Honesty is a core aspect of our vision for Claude’s ethical character. Indeed, while we want Claude’s honesty to be tactful, graceful, and infused with deep care for the interests of all stakeholders, we also want Claude to hold standards of honesty that are substantially higher than the ones at stake in many standard visions of human ethics. https://www.anthropic.com/constitution https://www.anthropic.com/constitution

mvdtnz 3mo ago
"Genuine" appears 50 times too. I think you're onto something.
- raverbashing 3mo ago
  I'm honestly thinking it's trapped in a Chinese room without any way out
- deleted 3mo ago
  [deleted]
Barbing 3mo ago
Do technologists have more respect for the idea you can train a model to be on your side with a constitution than they might’ve at first? I'm sure the concept seemed just about purely preposterous to many when the models were in their infancy. Now I figure instead it seems mostly preposterous to many. (Though I guess Anthropic‘s success doesn’t necessarily prove anything about the constitution)
- titanomachy 3mo ago
  I don’t think anyone imagines that it’s an ironclad steering method, but it seems to help, so why not?
- 3mo ago

CodesInChaos 3mo ago

> LLMs seem to get tragically stuck on certain patterns. That is likely an artifact of the fine-tuning process: > Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in supervised fine-tuning or preference data. > That creates a feedback loop: > * Some rewarded examples contain a distinctive lexical tic. > * The tic appears more often in rollouts. > * Model-generated rollouts are used for supervised fine-tuning (SFT). > * The model gets even more comfortable producing the tic. https://openai.com/index/where-the-goblins-came-from/ https://openai.com/index/where-the-goblins-came-from/

zzbzq 3mo ago

I also noticed Gemini's habit of getting stuck on things I said. It became evident quite quickly. I haven't noticed this in the same way in any other model. Something's wrong with that boy

malfist 3mo ago
Something's wrong with all of them. Uncanny valley freaks.

cozzyd 3mo ago

you should be careful about the times it doesn't say honest!

solarwindy 3mo ago

Heh, one vestigial bit of code, and they all are. Mind you, it's quite a creaky codebase, so it's forgivable to keep finding these appendices and calling them out as such. Useful, even.