3 ms·

They already have a track record of messing with internal system prompts (including those that affect the API) which obviously directly change outputs given the

by jjani 1y ago

They already have a track record of messing with internal system prompts (including those that affect the API) which obviously directly change outputs given the same prompts. So in effect, they've already been messing with the models for a long time. It's well known among founders who run services based on their products that this happened, everyone who does long output saw the same. It happened around November last year. If you had a set of evals running that expected an output of e.g. 6k tokens in length on 3.5 Sonnet, overnight it suddenly started cutting off at <2k, ending the message with something like "(Would you like me to continue?)". This is on raw API calls. Never seen or heard of (from people running services at scale, not just rumours) this kind of API behaviour change for a the same model from OpenAI and Google. Gemini 2.5 Pro did materially change at time of prod release despite them claiming they had simply "promoted the final preview endpoint to GA", but in that case you can give them the benefit of it being technically a new endpoint. Still lying, but less severe.

simonw 1y ago
Can you expand on "messing with internal system prompts" - this is the first I have heard of that.
- jjani 1y ago
  We talked about this a few weeks ago so it can't be the first time you're hearing about this :) [1] You hadn't heard about it before because it really only affected API customers running services including calls that required output of 2.5k+ (rough estimate) tokens in a single message. Which is pretty much just the small subset of AI founders/developers that are in the long-form content space. And then out of those, the ones using Sonnet 3.5 at the time for these tasks, which is an even smaller number. Back then it wasn't as favoured yet as it is now, especially for content. It's also expensive for high-output tasks, so only relevant to high-margin services, mostly B2B. Most of us in that small group aren't posting online about it - we urgently work to work around it, as we did back then. Still, as I showed you, some others did post about it as well. The only explanations were either internal system prompt changes, or updating the actual model. Since the only sharply different evals were those expecting 2.5k+ token outputs with all short ones remaining the same, and the consistency of the change was effectively 100%, it's unlikely to have been a stealth model update, though not impossible. [1] https://news.ycombinator.com/item?id=44844311 https://news.ycombinator.com/item?id=44844311
  - simonw 1y ago
    I didn't see that as being about changing system prompts, I thought it was about changing model weights.

by jjani 1y ago

simonw 1y ago

Can you expand on "messing with internal system prompts" - this is the first I have heard of that.

jjani 1y ago
We talked about this a few weeks ago so it can't be the first time you're hearing about this :) [1] You hadn't heard about it before because it really only affected API customers running services including calls that required output of 2.5k+ (rough estimate) tokens in a single message. Which is pretty much just the small subset of AI founders/developers that are in the long-form content space. And then out of those, the ones using Sonnet 3.5 at the time for these tasks, which is an even smaller number. Back then it wasn't as favoured yet as it is now, especially for content. It's also expensive for high-output tasks, so only relevant to high-margin services, mostly B2B. Most of us in that small group aren't posting online about it - we urgently work to work around it, as we did back then. Still, as I showed you, some others did post about it as well. The only explanations were either internal system prompt changes, or updating the actual model. Since the only sharply different evals were those expecting 2.5k+ token outputs with all short ones remaining the same, and the consistency of the change was effectively 100%, it's unlikely to have been a stealth model update, though not impossible. [1] https://news.ycombinator.com/item?id=44844311 https://news.ycombinator.com/item?id=44844311
- simonw 1y ago
  I didn't see that as being about changing system prompts, I thought it was about changing model weights.