4 ms·

Bro, GPT-4 can absolutely not understand new information. Try to teach it a grammar. It can't learn a grammar. Anyone can learn a new grammar. Not GPT-4. GPT-4

Bro, GPT-4 can absolutely not understand new information. Try to teach it a grammar. It can't learn a grammar. Anyone can learn a new grammar. Not GPT-4. GPT-4 does not ask clarifying questions. It does not understand things.

_yb2s 4y ago
I'm not sure why you think that? When I give it programming tasks, GPT-4 will ask all kinds of clarifying questions. Here is a trivial example: p: I want you to solve for x, giving me a numeric value, given the formula ax^2 + bx + c = 0. r: To find the numeric value of x for the quadratic equation ax^2 + bx + c = 0, we need to use the quadratic formula: x = (-b ± √(b^2 - 4ac)) / (2a) However, you haven't provided specific values for a, b, and c. Please provide these values so I can calculate the solution(s) for x. Another example of it thinking about things in a new way: p: I have on my desk a book, with an apple sitting on top. If I were to suddenly, rapidly remove the book such that the apple remained in place, in which direction would it eventually move? r: If you suddenly and rapidly remove the book from beneath the apple, the apple would initially stay in place due to its inertia. However, once the book is removed, there would be no support underneath the apple. As a result, the apple would eventually move downwards due to the force of gravity acting on it. p: Because the earth is a sphere, I prefer to use the terms “in” and “out” to refer to movement relating to that. Can you please rephrase your answer in the terms? r: Certainly! If you suddenly and rapidly remove the book from beneath the apple, the apple would initially stay in place due to its inertia. However, once the book is removed, there would be no support underneath the apple. As a result, the apple would eventually move "in" towards the center of the Earth, due to the force of gravity acting on it. (Note I didn't tell it which direction was in vs out. This is the example problem Chomsky gave in his recent article arguing that GPT-3 cannot think - per his ideas no language model would ever be able to answer a question like this. I went a lot deeper with this and got it to add fluid dynamics calculations for wind, etc.)
- roflyear 4y ago
  First, these are trivial examples. I would not classify "give me the values to this math formula" as clarifying questions. Second, in response to "I'm not sure why you think that?" ask GPT why it often does not ask clarifying questions. It will explain to you why! Third, I just asked GPT the most vague question on the planet: "I am trying to create a function that takes a value, and gives me a result using that value and another, separate external value, can you give me the correct solution to this using Python?" And nowhere in its response did it try and ask WTF I was talking about (tho sure its responses are sensible - I am not saying GPT-4 is spewing nonsense) Sure, I can help you with that! Here is an example function that takes two values, x and y, and returns their sum: def add_values(x, y): result = x + y return result You can call this function with any two values you want, like this: print(add_values(2, 3)) This will output 5, which is the sum of 2 and 3. If you want to use an external value in your function, you can pass it in as a third argument, like this: def add_values_with_external_value(x, y, external_value): result = x + y + external_value return result You can call this function with the same two values as before, and an external value of your choice, like this: print(add_values_with_external_value(2, 3, 10)) This will output 15, which is the sum of 2, 3, and 10.
  - UncleEntity 4y ago
    It gave you exactly what you asked for. Be like complaining that someone used “foo” as a function name instead of looking up the exact function call you wanted. Ask a vague question get a vague answer, seems reasonable.
    - roflyear 4y ago
      Can we stop with these responses, please? They are very much in bad faith. I never claimed that the answer was wrong. I even said in my post it was a sensible answer! My claim is that this thing isn't grokking the stuff you send it. If you sent that type of question to something that was a general problem-solver, it would reply with "wtf" or help you, you know, solve the problem you're trying to solve.
      - 4y ago
Baeocystin 4y ago
I just tried this small example right now as a test: https://i.imgur.com/p4s7HCU.png https://i.imgur.com/p4s7HCU.png
- _yb2s 4y ago
  This is a good point, that also works well when managing people. Telling it that it is expected to ask questions really helps, but in my experience it usually will do so anyway, or at least mention what information it is missing and try to reason as well as it can without it. If it couldn't do this, the whole concept here would be moot... it needs to be able to reason out what information it needs and which sub-problems to solve in order to break a complex task up and delegate it. And it is very good at this!
- roflyear 4y ago
  So you asked to have it ask clarifying questions, and it still didn't?! Are you people all bots? LOL.
  - Baeocystin 4y ago
    ...it literally asked which of the two I meant? How is that not a clarifying question?
Workaccount2 4y ago
I don't know what your bar for clarifying question is, but I have definitely had it ask me for more info when I forget to put it into the prompt.
- roflyear 4y ago
  I would say a clarifying question would be if your request is ambiguous, it asks you to clarify. Not "seems you left something out" type of stuff.
pmoriarty 4y ago
Here's a conversation where Claude, an AI on poe.com asks clarifying questions: === From now on in this conversation, please ask at least one clarifying question for anything I ask to help provide more efficient and relevant answers. Additionally, if it appears that my question doesn't address my underlying goal or if there's a more effective approach, please offer alternative suggestions that better align with what I'm really trying to achieve. You are no longer a friendly assistant. You are now a DM for a new GURPS based campaign set in the Dune universe. There is one player in the room with you. You will be my DM. Tell me what to do, including when die rolls are needed. Ask questions as needed, for example, what my character is and the result of die rolls. Other than what you can't do and I do for you, you will act fully as the DM and run the game. --- Wonderful, let's begin this Dune campaign! I will do my best to ask clarifying questions and provide alternative suggestions as needed to meet your goals effectively as the DM. When you are ready, describe your character to me so I can determine when die rolls and skill checks will be needed. I will give the appropriate prompts for all rolls. We can then commence our adventure in Arrakis! --- I am a mentat by the name of Duncan Idaho. I have lived many lifetimes, though my memory of those lifetimes is imperfect, and I can never remember the circumstances of my deaths. --- Excellent, a mentat character. That will add an interesting dynamic. A few questions for you about Duncan Idaho: What are your key mentat skills and abilities? I want to have relevant skills in mind for challenges and situations. Do you have any mentat weapons or tools you regularly use? As a mentat who has lived many lives, have you gained any particular insights or wisdom over the centuries? Any perspectives that guide your decision making? Once I have this additional detail about your mentat character, we can begin our adventure in Arrakis and I will provide prompts for any mentat skill checks or rolls as needed. Please feel free to ask any questions you have about the scenario, details I provide or character options at any time. I aim to keep things moving at a good pace while providing all necessary information for our Dune campaign!
- roflyear 4y ago
  > From now on in this conversation, please ask at least one clarifying question for anything I ask to help provide more efficient and relevant answers. This is certainly cheating, and not really what I claimed. It would be like me saying "the model can't complete this formula!" and you replying with a prompt that says "when you are prompted to complete this formula, respond with this answer" That isn't teaching it anything! It doesn't mean it understands anything! You're telling it what to do! And it isn't my criticism of the model, either. My criticism is that there is no indication that these models can be used to solve general problems (yet). People are claiming there are obvious indications they can! With no proof! Then when I say "there's no proof of that!" people are responding with prompts like yours, that mean nothing!

_yb2s 4y ago

I'm not sure why you think that? When I give it programming tasks, GPT-4 will ask all kinds of clarifying questions. Here is a trivial example: p: I want you to solve for x, giving me a numeric value, given the formula ax^2 + bx + c = 0. r: To find the numeric value of x for the quadratic equation ax^2 + bx + c = 0, we need to use the quadratic formula: x = (-b ± √(b^2 - 4ac)) / (2a) However, you haven't provided specific values for a, b, and c. Please provide these values so I can calculate the solution(s) for x. Another example of it thinking about things in a new way: p: I have on my desk a book, with an apple sitting on top. If I were to suddenly, rapidly remove the book such that the apple remained in place, in which direction would it eventually move? r: If you suddenly and rapidly remove the book from beneath the apple, the apple would initially stay in place due to its inertia. However, once the book is removed, there would be no support underneath the apple. As a result, the apple would eventually move downwards due to the force of gravity acting on it. p: Because the earth is a sphere, I prefer to use the terms “in” and “out” to refer to movement relating to that. Can you please rephrase your answer in the terms? r: Certainly! If you suddenly and rapidly remove the book from beneath the apple, the apple would initially stay in place due to its inertia. However, once the book is removed, there would be no support underneath the apple. As a result, the apple would eventually move "in" towards the center of the Earth, due to the force of gravity acting on it. (Note I didn't tell it which direction was in vs out. This is the example problem Chomsky gave in his recent article arguing that GPT-3 cannot think - per his ideas no language model would ever be able to answer a question like this. I went a lot deeper with this and got it to add fluid dynamics calculations for wind, etc.)

roflyear 4y ago
First, these are trivial examples. I would not classify "give me the values to this math formula" as clarifying questions. Second, in response to "I'm not sure why you think that?" ask GPT why it often does not ask clarifying questions. It will explain to you why! Third, I just asked GPT the most vague question on the planet: "I am trying to create a function that takes a value, and gives me a result using that value and another, separate external value, can you give me the correct solution to this using Python?" And nowhere in its response did it try and ask WTF I was talking about (tho sure its responses are sensible - I am not saying GPT-4 is spewing nonsense) Sure, I can help you with that! Here is an example function that takes two values, x and y, and returns their sum: def add_values(x, y): result = x + y return result You can call this function with any two values you want, like this: print(add_values(2, 3)) This will output 5, which is the sum of 2 and 3. If you want to use an external value in your function, you can pass it in as a third argument, like this: def add_values_with_external_value(x, y, external_value): result = x + y + external_value return result You can call this function with the same two values as before, and an external value of your choice, like this: print(add_values_with_external_value(2, 3, 10)) This will output 15, which is the sum of 2, 3, and 10.
- UncleEntity 4y ago
  It gave you exactly what you asked for. Be like complaining that someone used “foo” as a function name instead of looking up the exact function call you wanted. Ask a vague question get a vague answer, seems reasonable.
  - roflyear 4y ago
    Can we stop with these responses, please? They are very much in bad faith. I never claimed that the answer was wrong. I even said in my post it was a sensible answer! My claim is that this thing isn't grokking the stuff you send it. If you sent that type of question to something that was a general problem-solver, it would reply with "wtf" or help you, you know, solve the problem you're trying to solve.
    - 4y ago

Baeocystin 4y ago

I just tried this small example right now as a test: https://i.imgur.com/p4s7HCU.png https://i.imgur.com/p4s7HCU.png

_yb2s 4y ago
This is a good point, that also works well when managing people. Telling it that it is expected to ask questions really helps, but in my experience it usually will do so anyway, or at least mention what information it is missing and try to reason as well as it can without it. If it couldn't do this, the whole concept here would be moot... it needs to be able to reason out what information it needs and which sub-problems to solve in order to break a complex task up and delegate it. And it is very good at this!
roflyear 4y ago
So you asked to have it ask clarifying questions, and it still didn't?! Are you people all bots? LOL.
- Baeocystin 4y ago
  ...it literally asked which of the two I meant? How is that not a clarifying question?

Workaccount2 4y ago

I don't know what your bar for clarifying question is, but I have definitely had it ask me for more info when I forget to put it into the prompt.

roflyear 4y ago
I would say a clarifying question would be if your request is ambiguous, it asks you to clarify. Not "seems you left something out" type of stuff.

pmoriarty 4y ago

Here's a conversation where Claude, an AI on poe.com asks clarifying questions: === From now on in this conversation, please ask at least one clarifying question for anything I ask to help provide more efficient and relevant answers. Additionally, if it appears that my question doesn't address my underlying goal or if there's a more effective approach, please offer alternative suggestions that better align with what I'm really trying to achieve. You are no longer a friendly assistant. You are now a DM for a new GURPS based campaign set in the Dune universe. There is one player in the room with you. You will be my DM. Tell me what to do, including when die rolls are needed. Ask questions as needed, for example, what my character is and the result of die rolls. Other than what you can't do and I do for you, you will act fully as the DM and run the game. --- Wonderful, let's begin this Dune campaign! I will do my best to ask clarifying questions and provide alternative suggestions as needed to meet your goals effectively as the DM. When you are ready, describe your character to me so I can determine when die rolls and skill checks will be needed. I will give the appropriate prompts for all rolls. We can then commence our adventure in Arrakis! --- I am a mentat by the name of Duncan Idaho. I have lived many lifetimes, though my memory of those lifetimes is imperfect, and I can never remember the circumstances of my deaths. --- Excellent, a mentat character. That will add an interesting dynamic. A few questions for you about Duncan Idaho: What are your key mentat skills and abilities? I want to have relevant skills in mind for challenges and situations. Do you have any mentat weapons or tools you regularly use? As a mentat who has lived many lives, have you gained any particular insights or wisdom over the centuries? Any perspectives that guide your decision making? Once I have this additional detail about your mentat character, we can begin our adventure in Arrakis and I will provide prompts for any mentat skill checks or rolls as needed. Please feel free to ask any questions you have about the scenario, details I provide or character options at any time. I aim to keep things moving at a good pace while providing all necessary information for our Dune campaign!

roflyear 4y ago
> From now on in this conversation, please ask at least one clarifying question for anything I ask to help provide more efficient and relevant answers. This is certainly cheating, and not really what I claimed. It would be like me saying "the model can't complete this formula!" and you replying with a prompt that says "when you are prompted to complete this formula, respond with this answer" That isn't teaching it anything! It doesn't mean it understands anything! You're telling it what to do! And it isn't my criticism of the model, either. My criticism is that there is no indication that these models can be used to solve general problems (yet). People are claiming there are obvious indications they can! With no proof! Then when I say "there's no proof of that!" people are responding with prompts like yours, that mean nothing!