3 ms·

a lot of people trying to get a language model to do maths and coding. obviously this is not actually within its capabilities to properly reason about... You c

a lot of people trying to get a language model to do maths and coding. obviously this is not actually within its capabilities to properly reason about... You can ask it non-trivial things about maths (from a beginner at maths perspective....) and it will just fabricate nonsense.. like 'find the missing number' for: 5,10,?,50,122 It said this: starting with 5, add 5 to get 10 starting with 10, add 10 to get 'x' starting with 'x' add 40 to get 50 starting with 50, add 72 to get 122 then it goes on to say 'using this pattern we can find the value of x by subtracting 10 from the third term. x = 10 + (50-10)/2 x = 10 + 20 x = 30 Amazing, complete and utter nonsense. The real answer of course is a bit difficult here for normal people like me.. But any person can clearly see that even given its own logic, it still violates its own logic... x + 40 would be 50 in its initial logic. 30 + 40 is clearly not 50... - if it understood _anything_ about numbers and values, it would never have said such a thing. This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer....

sim7c00 4y ago
(if u wanna know, the pattern is n^2 + 1 and the answer is 26)
sebzim4500 4y ago
One more than the squared primes? So 26 is the missing number? That's pretty hard, I'm not suprised it couldn't find that. >This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer.... Yeah, this is the biggest problem IMO. Just making it answer "I don't know" when it doesn't know something would make it enormously more useful, even if it occasionally said it didn't know in cases where it currently gets the answer right.
- throwaway4aday 4y ago
  It doesn't have a concept of not knowing. You can literally type gibberish like "goronne domkiry awerquin mernacb" and it will produce matching output.
  - Sunhold 4y ago
    Did you try? It replies "I'm sorry, but I don't recognize the language or the words you've written. Could you please provide more context or information about the language or the origin of these words? That would help me understand better and provide a more accurate response."
    - throwaway4aday 4y ago
      You must be using gpt-3.5-turbo, try it with text-davinci-003. They did... something to 3.5 beyond just RLHF and it's somewhat lobotomized now or strongly biased towards producing canned output for prompts that aren't middle of the road.
hackinthebochs 4y ago
> This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer.... GPT-3 has some "meta-cognitive" access and can communicate it when prompted to: https://www.lesswrong.com/posts/ADwayvunaJqBLzawa/contra-hofstadter-on-gpt-3-nonsense https://www.lesswrong.com/posts/ADwayvunaJqBLzawa/contra-hof...

sim7c00 4y ago

(if u wanna know, the pattern is n^2 + 1 and the answer is 26)

sebzim4500 4y ago

One more than the squared primes? So 26 is the missing number? That's pretty hard, I'm not suprised it couldn't find that. >This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer.... Yeah, this is the biggest problem IMO. Just making it answer "I don't know" when it doesn't know something would make it enormously more useful, even if it occasionally said it didn't know in cases where it currently gets the answer right.

throwaway4aday 4y ago
It doesn't have a concept of not knowing. You can literally type gibberish like "goronne domkiry awerquin mernacb" and it will produce matching output.
- Sunhold 4y ago
  Did you try? It replies "I'm sorry, but I don't recognize the language or the words you've written. Could you please provide more context or information about the language or the origin of these words? That would help me understand better and provide a more accurate response."
  - throwaway4aday 4y ago
    You must be using gpt-3.5-turbo, try it with text-davinci-003. They did... something to 3.5 beyond just RLHF and it's somewhat lobotomized now or strongly biased towards producing canned output for prompts that aren't middle of the road.

hackinthebochs 4y ago

> This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer.... GPT-3 has some "meta-cognitive" access and can communicate it when prompted to: https://www.lesswrong.com/posts/ADwayvunaJqBLzawa/contra-hofstadter-on-gpt-3-nonsense https://www.lesswrong.com/posts/ADwayvunaJqBLzawa/contra-hof...