4 ms·

The prevailing commentary on Reddit is that GPT4 has been getting progressively nerfed for weeks.

by Solvency 3y ago

meghan_rain 3y ago
[flagged]
- orangepurple 3y ago
  Tell us more
  - meghan_rain 3y ago
    The more a model is lobotomized to fit a certain narrative ("don't return jokes about women"), the less it performs on normal/safe tasks as well ("summarize this article").
  - rootusrootus 3y ago
    Actually, please don't.
    - meghan_rain 3y ago
      [flagged]
- SomethingNew2 3y ago
  [flagged]
eurekin 3y ago
I use it only for sketching out some programming ideas in python. It takes a lot of back and forth to get anything useful and a lot of times, I have to workaround: > However, these approaches would be a lot more complex and beyond the scope of this assistant's capabilities Which seems very strange
moffkalast 3y ago
> OpenAI might be using speculative decoding on GPT-4's inference. (not sure 100%) The idea is to use a smaller faster model to decode several tokens in advance, and then feeds them into a large oracle model as a single batch. If the small model was right about its predictions – the larger model agrees and we can decode several tokens in a single batch. But if the larger model rejects the tokens predicted by the draft model then the rest of the batch is discarded. And we continue with the larger model. The conspiracy theory that the new GPT-4 quality had been deteriorated might be simply because they are letting the oracle model accept lower probability sequences from the speculative decoding model. [0] Well if the recent leaks are true it's because they're running a smaller, likely heavily quantized model to do the bulk of the generation now, with the full float one only stepping in on occasion to save GPU time. The interesting thing is that it seems that 3.5-turbo has slightly improved in recent months, while performance of 4 has deteriorated, it's like they're converging them to a middle ground or something. [0] https://archive.ph/2RQ8X https://archive.ph/2RQ8X

The prevailing commentary on Reddit is that GPT4 has been getting progressively nerfed for weeks.

meghan_rain 3y ago

[flagged]

orangepurple 3y ago
Tell us more
- meghan_rain 3y ago
  The more a model is lobotomized to fit a certain narrative ("don't return jokes about women"), the less it performs on normal/safe tasks as well ("summarize this article").
- rootusrootus 3y ago
  Actually, please don't.
  - meghan_rain 3y ago
    [flagged]
SomethingNew2 3y ago
[flagged]

eurekin 3y ago

I use it only for sketching out some programming ideas in python. It takes a lot of back and forth to get anything useful and a lot of times, I have to workaround: > However, these approaches would be a lot more complex and beyond the scope of this assistant's capabilities Which seems very strange

moffkalast 3y ago

> OpenAI might be using speculative decoding on GPT-4's inference. (not sure 100%) The idea is to use a smaller faster model to decode several tokens in advance, and then feeds them into a large oracle model as a single batch. If the small model was right about its predictions – the larger model agrees and we can decode several tokens in a single batch. But if the larger model rejects the tokens predicted by the draft model then the rest of the batch is discarded. And we continue with the larger model. The conspiracy theory that the new GPT-4 quality had been deteriorated might be simply because they are letting the oracle model accept lower probability sequences from the speculative decoding model. [0] Well if the recent leaks are true it's because they're running a smaller, likely heavily quantized model to do the bulk of the generation now, with the full float one only stepping in on occasion to save GPU time. The interesting thing is that it seems that 3.5-turbo has slightly improved in recent months, while performance of 4 has deteriorated, it's like they're converging them to a middle ground or something. [0] https://archive.ph/2RQ8X https://archive.ph/2RQ8X