5 ms·

ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy

by valine 3y ago

ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy to personify the model and assume it gets bored the same way we get bored, but the truth is it literally couldn't care less. The model's "motivation" comes from the cost function we design, and I imagine this will hold true regardless of its intelligence. You have zero evidence to show models boredom is correlated with model intelligence. It's most certainly not a "real problem we face with A(G)I".

gabelschlager 3y ago
Even you are still personifying the model. There is no motivation. An input sentence is multiplied by a series of weights and the resulting output vector transformed back into a text. At the end of the day, it's a program like any other. There are no emotions or a consciousness.
- valine 3y ago
  That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".
ipaddr 3y ago
Rarely will it do what it is asked without me coming up with a backstory. q: What is an average price of x. a: This is a language model that only has access to information before 2021 q: what was the price range in 2021 a: we don't have access to realtime pricing information q: who does a: we don't have access to that information. q: How to do x a: step 1 research x step 2 follow research step 3 do something obvious q: write me an app that does.. a: code from a basic tutorial I don't understand what people with a higher than average IQ are getting from llms. The answers are basic and are common sense. It's stories are like a student trying to fill the page with words to hit a word count. It can write code as well as tutorials. Everything is surface level information
- valine 3y ago
  > I don't understand what people with a higher than average IQ are getting from llms People with higher than average IQ are smart enough to ask detailed and specific questions like “Fix the race condition in this [giant block of code]” rather than asking dumb, overly general things like “build me an app!!”. Also, chatgpt models with internet access are completely happy to help you price compare. Go try bing chat, or ChatGPT plugins.
  - ipaddr 3y ago
    So it's limited to people smart enough to know there is a race condition but not smart enough to fix it also trusting enough to accept the answer (how could this group audit the fix?). If you know enough to ask the right series of questions you already know the answer.
    - michaelmrose 3y ago
      Knowing that there is an error can be as trivial as reading a failed test or a bug report something even someone grossly insufficient to the task could do. Fixing the problem could be a 5 minute or a 5 hour tour. For some subset of the tasks that fall towards the hard end of the range validation and or adaptation of a proposed solution is liable to be faster than starting from scratch. I don't know how this could possibly be in dispute with capability rapidly increasing.
    - valine 3y ago
      I’m smart enough to fix a race condition, but why would I spend a hour debugging when ChatGPT can find the issue in 10 seconds. Sure it might not be perfect, but having some initial code to fix the issue is a huge time save. Making ChatGPT write a whole app is problematic because small issues are hidden by the large quantity of generated code. Having ChatGPT fix an issue in your own code is comparatively easy to spot check for correctness.
      - ipaddr 3y ago
        "but why would I spend a hour debugging" So you won't make the same mistake again
mumblemumble 3y ago
> refuses to do what you ask Even that wording is personification. We don't know all the details of how the GPT series of models are trained, but what we do know suggests to me that it doesn't refuse to do what you ask, per se, so much as produce an output that looks like refusing to do a task in when the input most resembles examples from its training data where the expected output was something along the lines of, "I'm sorry, Dave, I'm afraid I can't do that." But it's exceedingly unlikely that GPT series models have anything like a module that is making a specific decision to do or not do what you ask, unless you count the sort of anti-toxicity wrapper the have in the publicly-available chatbots built around these models.

by valine 3y ago

gabelschlager 3y ago

Even you are still personifying the model. There is no motivation. An input sentence is multiplied by a series of weights and the resulting output vector transformed back into a text. At the end of the day, it's a program like any other. There are no emotions or a consciousness.

valine 3y ago
That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".

ipaddr 3y ago

Rarely will it do what it is asked without me coming up with a backstory. q: What is an average price of x. a: This is a language model that only has access to information before 2021 q: what was the price range in 2021 a: we don't have access to realtime pricing information q: who does a: we don't have access to that information. q: How to do x a: step 1 research x step 2 follow research step 3 do something obvious q: write me an app that does.. a: code from a basic tutorial I don't understand what people with a higher than average IQ are getting from llms. The answers are basic and are common sense. It's stories are like a student trying to fill the page with words to hit a word count. It can write code as well as tutorials. Everything is surface level information

valine 3y ago
> I don't understand what people with a higher than average IQ are getting from llms People with higher than average IQ are smart enough to ask detailed and specific questions like “Fix the race condition in this [giant block of code]” rather than asking dumb, overly general things like “build me an app!!”. Also, chatgpt models with internet access are completely happy to help you price compare. Go try bing chat, or ChatGPT plugins.
- ipaddr 3y ago
  So it's limited to people smart enough to know there is a race condition but not smart enough to fix it also trusting enough to accept the answer (how could this group audit the fix?). If you know enough to ask the right series of questions you already know the answer.
  - michaelmrose 3y ago
    Knowing that there is an error can be as trivial as reading a failed test or a bug report something even someone grossly insufficient to the task could do. Fixing the problem could be a 5 minute or a 5 hour tour. For some subset of the tasks that fall towards the hard end of the range validation and or adaptation of a proposed solution is liable to be faster than starting from scratch. I don't know how this could possibly be in dispute with capability rapidly increasing.
  - valine 3y ago
    I’m smart enough to fix a race condition, but why would I spend a hour debugging when ChatGPT can find the issue in 10 seconds. Sure it might not be perfect, but having some initial code to fix the issue is a huge time save. Making ChatGPT write a whole app is problematic because small issues are hidden by the large quantity of generated code. Having ChatGPT fix an issue in your own code is comparatively easy to spot check for correctness.
    - ipaddr 3y ago
      "but why would I spend a hour debugging" So you won't make the same mistake again

mumblemumble 3y ago

> refuses to do what you ask Even that wording is personification. We don't know all the details of how the GPT series of models are trained, but what we do know suggests to me that it doesn't refuse to do what you ask, per se, so much as produce an output that looks like refusing to do a task in when the input most resembles examples from its training data where the expected output was something along the lines of, "I'm sorry, Dave, I'm afraid I can't do that." But it's exceedingly unlikely that GPT series models have anything like a module that is making a specific decision to do or not do what you ask, unless you count the sort of anti-toxicity wrapper the have in the publicly-available chatbots built around these models.