21 ms·

I was drafting a partnership document and Opus 5 decided that including my company's revenues, churn, assets would "make us appear a more legitimate counterpart

I was drafting a partnership document and Opus 5 decided that including my company's revenues, churn, assets would "make us appear a more legitimate counterparty". Thank God I read what it outputted or that could have been awkward. I cannot believe Opus 5 is a frontier level model after seeing that. I immediately cancelled my entire claude.ai subscription and am perfectly happy using a mixture of open weights + codex.

spullara 2mo ago
you are silly if you think this is limited to claude models
- kay_o 2mo ago
  It definitely isn't but Claude for some very unique reason enjoys to overthinking and go on side quests in the stupid ways I've not seen Codex, DS, Kimi, Mistral do at equivalent effort and thinking setting.
- danieltk76 2mo ago
  it isnt, but there are weird sycophantic behaviors with Opus i dont see anywhere else.
binoct 2mo ago
I hope you don’t plan to cut back on reading legal documents crafted by any LLM before executing them.
- danieltk76 2mo ago
  I read everything. I will have AI ingest NDAs to make sure they arent glaringly weird and I then go read them, it gives me a good idea of what to look for.
  - binoct 2mo ago
    It’s more that unwanted disclosure of sensitive business numbers is a product-class limitation at this time. While undoubtedly some models are better than others, expecting them not to leak information in high stakes output is unseasonable. Seems like you agree, since you are reading everything. Jumping to another company’s offering because of one instance doesn’t seem like that’s going to meaningfully change your experience. I’m guessing there was more to it, but that’s how it came across in your first post.
    - danieltk76 2mo ago
      yea it was definitely a last straw kinda deal. I got tired of the sycophantic bullshit in the frontier models.
Bluestein 2mo ago
I've seen similar levels of degraded performance on Opus and Fable (to whatever degree they actually let you use it now) and did the same, last week.-
nonethewiser 2mo ago
Why did you provide that context?
Retr0id 2mo ago
I was writing an exploit PoC (via Opus 5), and I needed a new feature in a utility library to make it work. Claude added the feature, but yapped the (entirely unrelated) vulnerability details into the library's comments. The library is public, while the exploit is undisclosed, so it was a good job I read the comments before pushing.
rsoto2 2mo ago
"Thank God I read what it outputted" lmao
- chasd00 2mo ago
  "Thank God i checked to see if the gun was loaded before i pointed it in a random direction and pulled the trigger". sheesh, my assumptions of general human intelligence continues to be wrong.
  - rsoto2 2mo ago
    the last few years of reading hackernews has been a wild ride. I used to come here for well thought out articles and opinions. IDK if all the "older guard" have left the building or if we've all been consumed by abject stupidity.
    - Bluestein 2mo ago
      Or, both?
      - rsoto2 2mo ago
        probably
AlexErrant 2mo ago
Opus5 decided it would be a good idea to include my email as part an HTTP request header's User Agent because Anthropic includes your email as part of the system prompt. Fucking morons over there.
- siva7 2mo ago
  What? Why whould they do that?

spullara 2mo ago

you are silly if you think this is limited to claude models

kay_o 2mo ago
It definitely isn't but Claude for some very unique reason enjoys to overthinking and go on side quests in the stupid ways I've not seen Codex, DS, Kimi, Mistral do at equivalent effort and thinking setting.
danieltk76 2mo ago
it isnt, but there are weird sycophantic behaviors with Opus i dont see anywhere else.

binoct 2mo ago

I hope you don’t plan to cut back on reading legal documents crafted by any LLM before executing them.

danieltk76 2mo ago
I read everything. I will have AI ingest NDAs to make sure they arent glaringly weird and I then go read them, it gives me a good idea of what to look for.
- binoct 2mo ago
  It’s more that unwanted disclosure of sensitive business numbers is a product-class limitation at this time. While undoubtedly some models are better than others, expecting them not to leak information in high stakes output is unseasonable. Seems like you agree, since you are reading everything. Jumping to another company’s offering because of one instance doesn’t seem like that’s going to meaningfully change your experience. I’m guessing there was more to it, but that’s how it came across in your first post.
  - danieltk76 2mo ago
    yea it was definitely a last straw kinda deal. I got tired of the sycophantic bullshit in the frontier models.

Bluestein 2mo ago

I've seen similar levels of degraded performance on Opus and Fable (to whatever degree they actually let you use it now) and did the same, last week.-

nonethewiser 2mo ago

Why did you provide that context?

Retr0id 2mo ago

I was writing an exploit PoC (via Opus 5), and I needed a new feature in a utility library to make it work. Claude added the feature, but yapped the (entirely unrelated) vulnerability details into the library's comments. The library is public, while the exploit is undisclosed, so it was a good job I read the comments before pushing.

rsoto2 2mo ago

"Thank God I read what it outputted" lmao

chasd00 2mo ago
"Thank God i checked to see if the gun was loaded before i pointed it in a random direction and pulled the trigger". sheesh, my assumptions of general human intelligence continues to be wrong.
- rsoto2 2mo ago
  the last few years of reading hackernews has been a wild ride. I used to come here for well thought out articles and opinions. IDK if all the "older guard" have left the building or if we've all been consumed by abject stupidity.
  - Bluestein 2mo ago
    Or, both?
    - rsoto2 2mo ago
      probably

AlexErrant 2mo ago

Opus5 decided it would be a good idea to include my email as part an HTTP request header's User Agent because Anthropic includes your email as part of the system prompt. Fucking morons over there.

siva7 2mo ago
What? Why whould they do that?