6 ms·

So the original paper from Stanford and Berkley is also linked to this AI influencer? I am really amazed by this kind of dismissal. Its totally irrelevant who p

So the original paper from Stanford and Berkley is also linked to this AI influencer? I am really amazed by this kind of dismissal. Its totally irrelevant who posted the info and how framed it is, as long as you have access to the source.

gwd 3y ago
This comment has an interesting take on it, haven't read the paper to verify the take: https://news.ycombinator.com/item?id=36781968 https://news.ycombinator.com/item?id=36781968 EDIT: FWIW I haven't noticed any such regression. I don't generally use it to find prime numbers, but I do use it for coding, and have been really impressed with what it's able to do. 8<--- This paper is being misinterpreted. The degradations reported are somewhat peculiar to the authors' task selection and evaluation method and can easily result from fine tuning rather than intentionally degrading GPT-4's performance for cost saving reasons. They report 2 degradations: code generation & math problems. In both cases, they report a behavior change (likely fine tuning) rather than a capability decrease (possibly intentional degradation). The paper confuses these a bit: they mostly say behavior, including in the title, but the intro says capability in a couple of places. Code generation: the change they report is that the newer GPT-4 adds non-code text to its output. They don't evaluate the correctness of the code. They merely check if the code is directly executable. So the newer model's attempt to be more helpful counted against it. Math problems (primality checking): to solve this the model needs to do chain of thought. For some weird reason, the newer model doesn't seem to do so when asked to think step by step (but the current ChatGPT-4 does, as you can easily check). The paper doesn't say that the accuracy is worse conditional on doing CoT. The other two tasks are visual reasoning and answering sensitive questions. On the former, they report a slight improvement. On the latter, they report that the filters are much more effective — unsurprising since we know that OpenAI has been heavily tweaking these. In short, everything in the paper is consistent with fine tuning. It is possible that OpenAI is gaslighting everyone by denying that they degraded performance for cost saving purposes — but if so, this paper doesn't provide evidence of it. Still, it's a fascinating study of the unintended consequences of model updates.
- majkinetor 3y ago
  Fine tuning or not, its definitelly a proof one should not rely on it apart from very specific use cases (like lorem ipsum generator or something).
  - gwd 3y ago
    Just to be clear, you're saying that because they're tweaking GPT-4 to give more explanations of code, you shouldn't rely on it for coding? Obviously if that's your own preference, I'm not going to tell you that you're wrong; but I think in general, most people wouldn't agree with that statement.
    - agentgumshoe 3y ago
      I'm still wondering, why should anyone rely on AI generated answers? They are logically no better than search engine results. By that I mean, you can't tell if it's returning absolute trash or spot on correct. Building trust into it all is going to be either a) expensive or b) driven by all the wrong incentives.
      - gwd 3y ago
        > By that I mean, you can't tell if it's returning absolute trash or spot on correct. You use it for things which are 1) hard to write but easy to verify -- like doing drudge-work coding tasks for you, or rewording an email to be more diplomatic, or coming up with good tweets on some topic 2) things where it doesn't need to be perfect, just better than what you could do yourself. Here's an example of something last week that saved me some annoying drudge work in coding: https://gitlab.com/-/snippets/2567734 https://gitlab.com/-/snippets/2567734 And here's an example where it saved me having to skim through the massive documentation of a very "flexible" library to figure out how to do something: https://gitlab.com/-/snippets/2549955 https://gitlab.com/-/snippets/2549955 In the second category: I'm also learning two languages; I can paste a sentence into GPT-4 and ask it, "Can you explain the grammar to me?" Sure, there's a chance it might be wrong about something; but it's less wrong than the random guesses I'd be making by myself. As I gain experience, I'll eventually correct all the mistakes -- both the ones I got from making my own guesses, and the ones I got from GPT-4; and the help I've gotten from GPT-4 makes the mistakes worth it.
deleted 3y ago
[deleted]
ambrozk 3y ago
The paper does things like ask GPT-4 to write code and then check if that code compiles. Since March, they've fine-tuned GPT-4 to add back-ticks around code, which improves human-readable formatting but stops the code compiling. This is interpreted as "degraded performance" in the paper even though it's improved performance from a human perspective.
- josefx 3y ago
  The query explicitly asks it to add no other text to the code. > it's improved performance from a human perspective. Ignoring explicit requirements is the kind of thing that makes modern day search engines a pain to use.
  - lolinder 3y ago
    If I'm using it from the web UI, this is exactly what I would want—this allows the language model to define the output language so I get correct syntax highlighting, without an error-prone secondary step of language detection. If I'm using it from the API, then all I have to do is strip out the leading backticks and language name if I don't need to check the language, or alternatively parse it to determine what the output language is. It seems to me that in either case this is actually strictly better, and annotating the computer programming language used doesn't feel to me like extra text—I would think of that requirement as prohibiting a plaintext explanation before or after the code.
    - lukas099 3y ago
      It is still ignoring an explicit requirement, which is almost always bad. The user should be able to override what the creator/application thinks is 'strictly better'. Exceptions probably exist, but this isn't one of them.
      - lolinder 3y ago
        Like I said, I read their requirement differently—the phrase they use is "the code only". I don't think that including backticks is a violation of this requirement. It's still readily parseable and serves as metadata for interpreting the code. In the context of ChatGPT, which typically will provide a full explanation for the code snippet, I think this is a reasonable interpretation of the instruction.
phillipcarter 3y ago
Like clockwork, it's coming out that the original paper was wildly misinterpreted: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim...

gwd 3y ago

This comment has an interesting take on it, haven't read the paper to verify the take: https://news.ycombinator.com/item?id=36781968 https://news.ycombinator.com/item?id=36781968 EDIT: FWIW I haven't noticed any such regression. I don't generally use it to find prime numbers, but I do use it for coding, and have been really impressed with what it's able to do. 8<--- This paper is being misinterpreted. The degradations reported are somewhat peculiar to the authors' task selection and evaluation method and can easily result from fine tuning rather than intentionally degrading GPT-4's performance for cost saving reasons. They report 2 degradations: code generation & math problems. In both cases, they report a behavior change (likely fine tuning) rather than a capability decrease (possibly intentional degradation). The paper confuses these a bit: they mostly say behavior, including in the title, but the intro says capability in a couple of places. Code generation: the change they report is that the newer GPT-4 adds non-code text to its output. They don't evaluate the correctness of the code. They merely check if the code is directly executable. So the newer model's attempt to be more helpful counted against it. Math problems (primality checking): to solve this the model needs to do chain of thought. For some weird reason, the newer model doesn't seem to do so when asked to think step by step (but the current ChatGPT-4 does, as you can easily check). The paper doesn't say that the accuracy is worse conditional on doing CoT. The other two tasks are visual reasoning and answering sensitive questions. On the former, they report a slight improvement. On the latter, they report that the filters are much more effective — unsurprising since we know that OpenAI has been heavily tweaking these. In short, everything in the paper is consistent with fine tuning. It is possible that OpenAI is gaslighting everyone by denying that they degraded performance for cost saving purposes — but if so, this paper doesn't provide evidence of it. Still, it's a fascinating study of the unintended consequences of model updates.

majkinetor 3y ago
Fine tuning or not, its definitelly a proof one should not rely on it apart from very specific use cases (like lorem ipsum generator or something).
- gwd 3y ago
  Just to be clear, you're saying that because they're tweaking GPT-4 to give more explanations of code, you shouldn't rely on it for coding? Obviously if that's your own preference, I'm not going to tell you that you're wrong; but I think in general, most people wouldn't agree with that statement.
  - agentgumshoe 3y ago
    I'm still wondering, why should anyone rely on AI generated answers? They are logically no better than search engine results. By that I mean, you can't tell if it's returning absolute trash or spot on correct. Building trust into it all is going to be either a) expensive or b) driven by all the wrong incentives.
    - gwd 3y ago
      > By that I mean, you can't tell if it's returning absolute trash or spot on correct. You use it for things which are 1) hard to write but easy to verify -- like doing drudge-work coding tasks for you, or rewording an email to be more diplomatic, or coming up with good tweets on some topic 2) things where it doesn't need to be perfect, just better than what you could do yourself. Here's an example of something last week that saved me some annoying drudge work in coding: https://gitlab.com/-/snippets/2567734 https://gitlab.com/-/snippets/2567734 And here's an example where it saved me having to skim through the massive documentation of a very "flexible" library to figure out how to do something: https://gitlab.com/-/snippets/2549955 https://gitlab.com/-/snippets/2549955 In the second category: I'm also learning two languages; I can paste a sentence into GPT-4 and ask it, "Can you explain the grammar to me?" Sure, there's a chance it might be wrong about something; but it's less wrong than the random guesses I'd be making by myself. As I gain experience, I'll eventually correct all the mistakes -- both the ones I got from making my own guesses, and the ones I got from GPT-4; and the help I've gotten from GPT-4 makes the mistakes worth it.

deleted 3y ago

[deleted]

ambrozk 3y ago

The paper does things like ask GPT-4 to write code and then check if that code compiles. Since March, they've fine-tuned GPT-4 to add back-ticks around code, which improves human-readable formatting but stops the code compiling. This is interpreted as "degraded performance" in the paper even though it's improved performance from a human perspective.

josefx 3y ago
The query explicitly asks it to add no other text to the code. > it's improved performance from a human perspective. Ignoring explicit requirements is the kind of thing that makes modern day search engines a pain to use.
- lolinder 3y ago
  If I'm using it from the web UI, this is exactly what I would want—this allows the language model to define the output language so I get correct syntax highlighting, without an error-prone secondary step of language detection. If I'm using it from the API, then all I have to do is strip out the leading backticks and language name if I don't need to check the language, or alternatively parse it to determine what the output language is. It seems to me that in either case this is actually strictly better, and annotating the computer programming language used doesn't feel to me like extra text—I would think of that requirement as prohibiting a plaintext explanation before or after the code.
  - lukas099 3y ago
    It is still ignoring an explicit requirement, which is almost always bad. The user should be able to override what the creator/application thinks is 'strictly better'. Exceptions probably exist, but this isn't one of them.
    - lolinder 3y ago
      Like I said, I read their requirement differently—the phrase they use is "the code only". I don't think that including backticks is a violation of this requirement. It's still readily parseable and serves as metadata for interpreting the code. In the context of ChatGPT, which typically will provide a full explanation for the code snippet, I think this is a reasonable interpretation of the instruction.

phillipcarter 3y ago

Like clockwork, it's coming out that the original paper was wildly misinterpreted: https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-time https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim...