2 ms·

I must admit I skimmed most of your comment because it is largely an incoherent rant, but I will address some points: > This is data. Nope. Because frequency

I must admit I skimmed most of your comment because it is largely an incoherent rant, but I will address some points: > This is data. Nope. Because frequency bias is a thing. If you hear on Twitter "model X got nerfed," your brain will look for that pattern and notice it more than usual. This will then confirm your suspicion, which leads to a vicious cycle. Then you tell your friends and the same phenomenon repeats. None of this requires the model to get worse. It's a well understood psychological phenomenon. > I can tell you what it means: models performing worse at coding tasks. So people report models being worse at coding tasks The perception of a model performing worse at some coding task is not what "different token distribution" means. You should ask AI to explain my comment ;) Latency and TPS can also tell you if you're getting a quant. Anyways you should really get some help. Praying for you!

arkadiytehgraet 5mo ago
> frequency bias Gaslighting again, Anthropic shill. What does frequency bias have to do with the objective fact that hundreds of people reported their own experiences with LLMs being degraded over a short period of time? The very same tasks that the very same LLMs could do, they no longer can? You seem to ignore this FACT, this DATA, and instead have to gaslight and divert into "frequency bias" nonsense. I do understand, why you are doing it, Anthropic shill, but at least have guts to admit it. > perception of a model You once again ignore what your LLM outputted and you typed yourself and divert into "perception", Anthropic shill. You do not need to sample entire output for tokens to notice the distribution moving. If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift. And as you said yourself, different token distribution = model being quantized. Which is reported in hundreds of separate instances. Which is more than enough to conclude that the model was, in fact, quantized, and no amount of gaslighting can change that. But you are an Anthropic shill, so you have to peddle your bullshit, trying to twist facts to support your employer's narrative. And you deserve being called out on that, Anthropic shill.
- solenoid0937 5mo ago
  > What does frequency bias have to do with the objective fact that hundreds of people reported This isn't hard to understand "Model is nerfed" claim hits social media Someone else sees it, frequency bias makes them think their model is also nerfed, and they amplify the claim Now it spreads, like a virus, even if the model never changed Social dynamics like this are well understood psychologically > If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift. The more likely explanation is that you're looking at older LLMs with rose tinted glasses, and misremembering what it could achieve Otherwise you could measure the token shift and see the better tps and latency Your own evals would trend down But no one, not one person, has presented empirical evidence of being served a quant. Just vibes.
  - arkadiytehgraet 5mo ago
    Did your LLM context get blown up or why does your comment read like linkedin-style post with one-liner sentences structure? Did you really just claim that people are so gullible that it was social media or whatever that made them believe their LLM could no longer achieve the tasks it yesterday could, and not the actual FACT of LLM not being able to do it that they, you know, verified before complaining online? I guess if you gaslight everything like that, then indeed no matter what the facts are, you will never be convinced in anything. You see, because of outlandish claims and reasoning (or rather lack of) like that, everyone sees that you are an Anthropic shill.
    - deleted 5mo ago
      [deleted]
    - liamwire 5mo ago
      My god, man. Go read the HN guidelines, this method of communication isn't only insufferable to read, but is actively making this place worse to be a participant of.
    - lukebitts 5mo ago
      Having read this whole conversation (much like one feels the need to stare at a car accident), you sound truly insane. I hope that’s you were aiming for, otherwise you really need help

arkadiytehgraet 5mo ago

> frequency bias Gaslighting again, Anthropic shill. What does frequency bias have to do with the objective fact that hundreds of people reported their own experiences with LLMs being degraded over a short period of time? The very same tasks that the very same LLMs could do, they no longer can? You seem to ignore this FACT, this DATA, and instead have to gaslight and divert into "frequency bias" nonsense. I do understand, why you are doing it, Anthropic shill, but at least have guts to admit it. > perception of a model You once again ignore what your LLM outputted and you typed yourself and divert into "perception", Anthropic shill. You do not need to sample entire output for tokens to notice the distribution moving. If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift. And as you said yourself, different token distribution = model being quantized. Which is reported in hundreds of separate instances. Which is more than enough to conclude that the model was, in fact, quantized, and no amount of gaslighting can change that. But you are an Anthropic shill, so you have to peddle your bullshit, trying to twist facts to support your employer's narrative. And you deserve being called out on that, Anthropic shill.

solenoid0937 5mo ago
> What does frequency bias have to do with the objective fact that hundreds of people reported This isn't hard to understand "Model is nerfed" claim hits social media Someone else sees it, frequency bias makes them think their model is also nerfed, and they amplify the claim Now it spreads, like a virus, even if the model never changed Social dynamics like this are well understood psychologically > If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift. The more likely explanation is that you're looking at older LLMs with rose tinted glasses, and misremembering what it could achieve Otherwise you could measure the token shift and see the better tps and latency Your own evals would trend down But no one, not one person, has presented empirical evidence of being served a quant. Just vibes.
- arkadiytehgraet 5mo ago
  Did your LLM context get blown up or why does your comment read like linkedin-style post with one-liner sentences structure? Did you really just claim that people are so gullible that it was social media or whatever that made them believe their LLM could no longer achieve the tasks it yesterday could, and not the actual FACT of LLM not being able to do it that they, you know, verified before complaining online? I guess if you gaslight everything like that, then indeed no matter what the facts are, you will never be convinced in anything. You see, because of outlandish claims and reasoning (or rather lack of) like that, everyone sees that you are an Anthropic shill.
  - deleted 5mo ago
    [deleted]
  - liamwire 5mo ago
    My god, man. Go read the HN guidelines, this method of communication isn't only insufferable to read, but is actively making this place worse to be a participant of.
  - lukebitts 5mo ago
    Having read this whole conversation (much like one feels the need to stare at a car accident), you sound truly insane. I hope that’s you were aiming for, otherwise you really need help