6 ms·
The LLM Lobotomy?
- esafak 1y agoThis was the perfect opportunity to share the evidence. I think undisclosed quantization is definitely a thing. We need benchmarks to be periodically re-evaluated to ward against this. Providers should keep timestamped models fixed, and assign modified versions a new timestamp, and price, if they want. The model with the "latest" tag could change over time, like a Docker image. Then we can make an informed decision over which version to use. Companies want to cost optimize their cake and eat it too. edit: I have the same complaint about my Google Home devices. The models they use today are indisputably worse than the ones they used five whole years ago. And features have been removed without notice. Qualitatively, the devices are no longer what I bought.
- gregsadetsky 1y agoI commented on the forum asking Sarge whether they could share some of their test results. If they do, I think that it will add a lot to this conversation. Hope it happens!
- gregsadetsky 1y agoUpdate: Sarge responded in the forum and added more information. I asked them to share data/dates as much as that’s possible - fingers crossed
- colordrops 1y agoIn addition to quantization, I suspect the additions they make continually to their hidden system prompt for legal, business, and other reasons slowly degrade responses over time as well.
- jonplackett 1y agoThis is quite similar to all the modifications intel had to do due to spectre - I bet those system prompts have grown exponentially.
- icyfox 1y agoI guarantee you the weights are already versioned like you're describing. Each training run results in a static bundle of outputs and these are very much pinned (OpenAI has confirmed multiple times that they don't change the model weights once they issue a public release). > Not quantized. Weights are the same. If we did change the model, we’d release it as a new model with a new name in the API.” - [Ted Sanders](https://news.ycombinator.com/item?id=44242198 https://news.ycombinator.com/item?id=44242198) (OpenAI) The problem here is that most issues stem from broader infrastructure issues like numerical instability at inference time. Since this affects their whole service pipeline, the logic here can't really be encapsulated in a frozen environment like a Docker container. I suppose _technically_ they could maintain a separate inference cluster for each of their point releases, but that also means that previous models don't benefit from common infrastructure improvements / load balancing would be more difficult to shard across GPUs / might be logistically so hard to coordinate to effectively make it impossible. https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues https://www.anthropic.com/engineering/a-postmortem-of-three-... https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
- xg15 1y agoSorry, but this makes no sense. Numerical instability would lead to random fluctuations in output quality, but not to a continuous slow decline like the OP described. Heard of similar experiences from RL acquaintances, where a prompt worked reliably for hundreds of requests per day for several months - and then suddenly the model started to make mistakes, ignore parts of the prompt, etc when a newer model was released. I agree, it doesn't have to be deliberate malice like intentionally nerfing a model to make people switch to the newer one - it might just be that less resources are allocated to the older model once the newer one is available and so the inference parameters change - but some effect at the release of a newer model seems to be there.
- icyfox 1y agoI'm responding to the parent comment who's suggesting we version control the "model" in Docker. There are infra reasons why companies don't do that. Numerical instability is one class of inference issues, but there can be other bugs in the stack separate from them intentionally changing the weights or switching to a quantized model. As for the original forum post: - Multiple numerical computation bugs can compound to make things worse (we saw this in the latest Anthropic post-mortum) - OP didn't provide any details on eval methodology, so I don't think it's worth speculating on this anecdotal report until we see more data
- ProjectArcturis 1y agoI'm confused why this is addressed to Azure instead of OpenAI. Isn't Azure just offering a wrapper around chatGPT? That said, I would also love to see some examples or data, instead of just "it's getting worse".
- SubiculumCode 1y agoI don't remember where I saw it, but I remember a claim that Azure hosted models performed poorer than those hosted by openAI.
- transcriptase 1y agoExplains why the enterprise copilot ChatGPT wrapper that they shoehorn into every piece of office365 performs worse than a badly configured local LLM.
- bongodongobob 1y agoThey most definitely do. They have been lobotomized in some way to be ultra corporate friendly. I can only use their M365 Copilot at work and it's absolute dogshit at writing code more than maybe 100 lines. It can barely write correct PowerShell. Luckily, I really only need it for quick and dirty short PS scripts.
- sebazzz 1y agoI agree. I asked it for some help refactoring a database and some of the SQL is quite broken. It also doesn't help that their streaming code is broken so LLM responses sometimes end up broken in the web browser (both Firefox and Edge so it is not a browser issue), so you need to refresh after a response to make sure the LLMs response was not an indication of a drunk LLM.
- SBArbeit 1y agoI know that OpenAI has made computing deals with other companies, and as time goes on, the percentage of inference that they run their models on will shift, but I doubt that much, if any, of that has moved from Microsoft Azure data centers yet, so that's not a reason for difference in model performance. With that said, Microsoft has a different level of responsibility, both to its customers and to its stakeholders, to provide safety than OpenAI or any other frontier provider. That's not a criticism of OpenAI or Anthropic or anyone else, who I believe are all trying their best to provide safe usage. (Well, other than xAI and Grok, for which the lack of safety is a feature, not a bug.) The risk to Microsoft of getting this wrong is simply higher than it is for other companies, and what's why they have a strong focus on Responsible AI (RAI) [1]. I don't know the details, but I have to assume there's a layer of RAI processing on models through Azure OpenAI that's not there for just using OpenAI models directly through the OpenAI API. That layer is valuable for the companies who choose to run their inference through Azure, who also want to maximize safety. I wonder if that's where some of the observed changes are coming from. I hope the commenter posts their proof for further inspection. It would help everyone. [1]: https://www.microsoft.com/en-us/ai/responsible-ai https://www.microsoft.com/en-us/ai/responsible-ai
- gwynforthewyn 1y agoWhat's the conversation that you're looking to have here? There are fairly widespread claims that GPT-5 is worse than 4, and that's what the help article you've linked to says. I'm not sure how this furthers dialog about or understanding of LLMs, though, it reads to _me_ like this question just reinforces a notion that lots of people already agree with. What's your aim here, sgt3v? I'd love to positively contribute, but I don't see how this link gets us anywhere.
- arresin 1y agoMaybe to prompt more anecdotes on how gpt-$ is the money making gpt—where they gut quality and hold prices steady to reduce losses? I can tell you that the post describes is exactly what I’ve seen also: degraded performance and excruciatingly slow.
- bigchillin 1y agoThis is why we have open source. I noticed this with cursor, it’s not just an azure problem.
- ukFxqnLa2sBSBf6 1y agoIt’s a good thing the author provided no data or examples. Otherwise, there might be something to actually talk about.
- cush 1y agoIs setting temperature to 0 even a valid way to measure LLM performance over time, all else equal?
- jonplackett 1y agoIt could be that performance on temp zero has declined but performance on a normal temp is the same or better. I wonder if temp zero would be more influenced by changes to the system prompt too. I can imagine it making responses more brittle.
- fortyseven 1y agoI'd have assumed a fixed seed was used, but he doesn't mention that. Weird. Maybe he meant that?
- numpad0 1y agoPure sci-Fi idea: what if actually nothing was changed, but RNGs were becoming less random as we extract more randomness out of the universe?
- maxbond 1y agoI bet they did both. If I'm reading the documentation right you have to supply a seed in order to get "best effort" determinism. https://learn.microsoft.com/en-us/azure/ai-foundry/openai/reference https://learn.microsoft.com/en-us/azure/ai-foundry/openai/re...
- criemen 1y agoEven with temperature 0, the LLM output will not be deterministic. It will just have less randomness (not defined precisely) than with temperature 1. There was a recent post on the frontpage about fully deterministic sampling, but it turns out to be quite difficult.
- visarga 1y agoIt's because batch size is dynamic. So a different batch size will change the output even on temp 0.
- SirensOfTitan 1y agoI’m convinced all of the major LLM providers silently quantize their models. The absolute worst was Google’s transition from Gemini 2.5 Pro 3-25 checkpoint to the May checkpoint, but I’ve noticed this effect with Claude and GPT over the years too. I couldn’t imagine relying on any closed models for a business because of this highly dishonest and deceptive practice.
- arresin 1y agoYou can be clever with language also. You can say “we never intentionally degrade model performance” and then claim you had no idea a quant would make perf worse because it was meant to make it better (faster).
- briga 1y agoI have a theory: all these people reporting degrading model quality over time aren't actually seeing model quality deteriorate. What they are actually doing is discovering that these models aren't as powerful as they initially thought (ie. expanding their sample size for judging how good the model is). The probabilistic nature of LLM produces a lot of confused thinking about how good a model is, just because a model produces nine excellent responses doesn't mean the tenth response won't be garbage.
- chaos_emergent 1y agoYes exactly, my theory is that the novelty of a new generation of LLMs’ performances tends to cause an inflation in peoples’ perceptions of the model, with a reversion to a better calibrated expectation over time. If the developer reported numerical evaluations that drifted over time, I’d be more convinced of model change.
- colordrops 1y agoDid any of you read the article? They have a test framework that objectively shows the model getting worse over time.
- Aurornis 1y agoI read the article. No proof was included. Not even a graph of declining results.
- colordrops 1y agoOk fair, but not including the data is not the same as the article saying it was subjective "feel".
- yieldcrv 1y agofta: “I am glad I have proof of this with the test system” I think they have receipts, but did not post them there
- 1y ago
- zzzeek 1y agoI'm sure MSFT will offer this person some upgraded API tier that somewhat improves the issues, though not terrifically, for only ten times the price.
- jug 1y agoAt least on OpenRouter, you can often verify what quant a provider is using for a particular model.
- mehdibl 1y agoSince when LLM become deterministic?
- thomasmg 1y agoLLM are just software + data and can be made deterministic, in the same way a pseudo random number generator can be made deterministic by using the same seed. For an LLM, you typically set temperature to 0, or set the random seed to the same value, run it on the same hardware (or emulation) or otherwise ensure the (floating point) calculations get the exact same results. I think that's it. In reality, yes it's not that easy, but it's possible.
- mr_toad 1y agoUnfortunately because floating point addition isn’t always associative, and because GPUs don’t always perform calculations in the same order you won’t always get the same result even with a temperature of zero.
- ant6n 1y agoI used to think running your own local model is silly because it’s slow and expensive, but the nerfing of ChatGPT and Gemini is so aggressive it’s starting to make a lot more sense. I want the smartest model, and I don’t want to second guess some potentially quantized black box.
- bbminner 1y agoAm I the only person who can sense the exact moment an LLM-written response kicked in? :) "sharing some of the test results/numbers you have would truly help cement this case!" - c'mon :)
- gregsadetsky 1y agoI actually 100% wrote that comment myself haha!! See https://news.ycombinator.com/item?id=45316437 https://news.ycombinator.com/item?id=45316437 I think it would have sounded more reasonable in French, which is my actual native tongue. (i.e. I subconsciously translate from French when I'm writing in English) ((this comment was also written without AI!!)) :-)
- bbminner 1y agoOh, my honest apologies then, Greg! :) I am not a native speaker myself. And as far as i can tell, the phrasing is absolutely grammatically correct, but there's some quality to it that registers as LLM-speak to me. I wonder how the causal graph looks here: do people (esp those working with LLMs a lot) lean towards LLM-speak over time, or both LLMs and native speakers picked up this very particular sentence structure from a common source? (eg a large corpus of French-English translations in the same style?)
- gregsadetsky 1y agoNo apologies needed, but thanks for your kind words! I think that we’re all understandably “on edge” considering that so much content is now llm-generated, and it’s hard to know what’s real and what isn’t. I’ve been removing hyphens and bullet points from my own writing just to appear even less llm like! :) Great stylistic chicken and egg question! French definitely tends to use certain (I’m struggling to not say “fancier”) words even in informal contexts. I personally value using over-the-top ornate expressions in French: they both sound distinguished and a bit ridiculous, so I get to both ironically enjoy them and feel detached from them… but none of that really translates to casual English. :) Cheers
- mmh0000 1y agoI've noticed this with Claude Code recently. A few weeks ago, Claude was "amazing" in that I could feed it some context and a specification, and it could generate mostly correct code and refine it in a few prompts. Now, I can try the same things, and Claude gets it terribly wrong and works itself into problems it can't find its way out of. The cynical side of me thinks this is being done on purpose, not to save Anthropic money, but to make more money by burning tokens.
- cjtrowbridge 1y agoThis brings up a point many will not be aware of. If you know the random seed and the prompt, and the hash of the model's binary file; the output is completely deterministic. You can use this information to check whether they are in fact swapping your requests out to cheaper models than what you're paying for. This level of auditability is a strong argument for using open-source, commodified models, because you can easily check if the vendor is ripping you off.
- TZubiri 1y agoPretty sure this is wrong, requests are batched and size can affect the output, also gpus are highly parallel, there can be many race conditions.
- TeMPOraL 1y agoYup. Floating point math turns race conditions into numerical errors, reintroducing non-determinism regardless of inputs used.
- romperstomper 1y agoCould it be a result of a caching of some sort? I suppose in case of LLM they can't make a direct cache but they could group prompts using embeddings and produce some most common result maybe? (this is just a theory)
- juliangoldsmith 1y agoI've been using Azure AI Foundry for an ongoing project, and have been extremely dissatisfied. The first issue I ran into was with them not supporting LLaMA for tool calls. Microsoft stated in February that they were working on it [0], and they were just closing the ticket because they were tracking it internally. I'm not sure why they've been unable to do what took me two hours in over six months, but I am sure they wouldn't be upset by me using the much more expensive OpenAI models. There are also consistent performance issues, even on small models, as mentioned elsewhere. This is with a rate on the order of one per minute. You can solve that with provisioned throughput units. The cheapest option is one of the GPT models, at a minimum of $10k/month (a bit under half the cost of just renting an A100 server). DeepSeek was a minimum of around $72k/month. I don't remember there being any other non-OpenAI models with a provisioned option. Given that current usage without provisioning is approximately in the single dollars per month, I have some doubts as to whether we'd be getting our money's worth having to provision capacity.