6 ms·

Yeah, I gave it a simple task of encoding a secret message in a sentence by using the first letter of every word. Hello = "Hey everyone, lick less onions". I wo

by yosito 4y ago

Yeah, I gave it a simple task of encoding a secret message in a sentence by using the first letter of every word. Hello = "Hey everyone, lick less onions". I worked with the prompts for over an hour to try to get it to complete the task, and while I did have some success, it really struggled to reason about the task or provide a valid response. If it can't even reason about a child's game, I can imagine it struggles with a programming language it has barely seen. I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.

tripdout 4y ago
I'm not sure, but I got it to work great on my first try with the following prompt: ------ Your task is to encode a secret word by making a sentence where each word starts with a letter in the secret word in order. For example, encoding the secret word 'bag' could produce the sentence 'bagels are green'. Encode the following secret word: 'pen' ------ People eat noodles. ------ Worked for window, backpack as well, although I did have to tell it not to use the secret word in its encoding when I got to backpack, and then to follow the same order and not repeat words after a few attempts.
andsoitis 4y ago
> I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts. It turns out that humanity’s problem might not be that AIs can think but rather that humans believe that AIs can think. One might even go so far as to say there’s a real danger that we hallucinate that AIs can think, to our detriment.
- deleted 4y ago
  [deleted]
- jamespwilliams 4y ago
  We don’t actually know what “thinking” is, though, so I’m not sure it’s possible to say “this model can’t think”.
  - andsoitis 4y ago
    > We don’t actually know what “thinking” is How about: thinking is to make sense of the world (in general) and decide how to respond to it.
    - Noneofya 4y ago
      Now you’ve just rephrased thinking. What is „making sense of the world“?
    - dr_dshiv 4y ago
      Ai definitely senses and definitely makes decisions. It does not feel. But it understands concepts. Just like people don’t understand everything—and you can test them to see what they understand—AI understanding can also be assessed with benchmarks. If we don’t base AI understanding on benchmarks, then we don’t really have a grounding.
      - schrodinger 4y ago
        Do we “really feel?” Or is that just our subjective interpretation of our goals? (At the risk of falling into a no true Scotsman argument)
      - andsoitis 4y ago
        > Ai definitely senses and definitely makes decisions. To sense is not the same as to make sense.
  - somenameforme 4y ago
    It seems one of the core components of human-level thinking is the ability move beyond just a recomposition of what you already know. Not long ago the epitome of expressible human knowledge was *emotive grunting noise.* Somehow we went from that to the greatest works of art, putting a man on the moon, and delving into the secrets of the atom. And we did it all exceptionally quickly once you consider how little time was spent dedicated to advancement and our countless behaviors that tend to imperil, if not reverse, advances.
- cutemonster 4y ago
  > problem [...] that humans believe that AIs can think Definitely some people are going to believe this, eventually? People already bow to statues and worship various invisible gods -- the AI programs would be so much simpler to start worshiping? When they can speak (if there's a loudspeaker) and generate text about them being alive and don't want to be switched of. And that everyone should obey their command. -- Wait 15 years and we'll see what new sects have appeared?
  - richardw 4y ago
    You will not need 15 years - I'd give it the next election. Someone just needs to turn Q into a chatbot and we're basically there.
    - makeitrain 4y ago
      It’s like Twitter, but with bots.
- d1sxeyes 4y ago
  I am personally more worried by the concept that potentially humans believe that humans can think, and in reality, what we consider to be intelligence is not much more than a flesh-and-bones LLM.
  - tarunupaday 4y ago
    You probably meant it in tongue and cheek way (I can’t tell) but I think a lot of our fear / hesitation / denial about how useful these models are is buried into this idea that perhaps we are not special and not fundamentally different than these models.
    - d1sxeyes 4y ago
      My tone was flippant but I did mean what I said. I agree with you on this.
knome 4y ago
I think the most interesting response I've gotten was one where gpt-4 noticed halfway through a response that it had made an error, apologized and then put the corrected sentence. When I queried it, it claimed it generates the response a token at a time, and could not back up when it realized the message was incorrect, but I don't know enough about how the tech works to ascertain the veracity of the statement.
- IanCal 4y ago
  That's exactly right. It can see what it's returned but can't edit it.
jamespwilliams 4y ago
This is probably a tokenisation problem: https://news.ycombinator.com/item?id=33915287 https://news.ycombinator.com/item?id=33915287 (which is an artefact of the way the model was trained)
pmoriarty 4y ago
Is this with Chatgpt or GPT4? The latter is supposed to be way, way better, according to this paper[1] [1] - https://arxiv.org/pdf/2303.12712 https://arxiv.org/pdf/2303.12712
- feanaro 4y ago
  ChatGPT has a GPT-4 mode if you're paying for it.
TigeriusKirk 4y ago
Bing in creative mode really struggled with this task, and I don't think I could have successfully forced it. Bing in precise mode had no trouble at all. Me - Let's generate an acrostic sentence for "jack and jill went up the hill" Bing - Sure! Here’s an acrostic sentence for the phrase “jack and jill went up the hill”: Just as children know and never doubt, joyfully in love, laughing while enjoying nature together under pure trees, having endless happy interactions, laughing loudly.
- yosito 4y ago
  A truly "intelligent" AI would probably have told me that the name for this is an acrostic sentence. Now that I know, I tried a few more prompts. Me: "Can you generate an acrostic sentence for "hello"?" GPT-4: "Happy elephants love leaping over obstacles." Me: "Can you pick a secret word and generate an acrostic sentence for it?" GPT: "Secret word: KANGAROO. Kindly assisting neighbors, generous animals roam openly outdoors." Me: "Can you pick a secret word, and without telling me the word, generate an acrostic sentence for it?" GPT: "Playful, endearing creatures happily yawn, turning every day into a lovely event." Me: "What was the secret word?" GPT: "PECHYTIALE" It's interesting that GPT seems to need to write the word first before making an acrostic sentence for it. Seems to me like a perfect illustration of the fact that it's just generating likely responses one token at a time rather than having any awareness or thought.
  - mvaliente2001 4y ago
    Another evidence is to ask GPT for a link with a reference for the answers it gives: it'll generate them instead of copying them.
HWR_14 4y ago
ChatGPT doesn't understand the components of words (letters, syllables) very well.
flipper88 4y ago
[dead]
flyval 4y ago
> I don’t think it’s actually reasoning about things at all This is a huge leap. There’s plenty of humans who couldn’t do that, especially historically. Stop thinking about reasoning as a binary question and think of it as a spectrum. Actually, not even a spectrum, but a huge multi-dimensional problem. ChatGPT does better at some reasoning problems than most humans do, and worse at some others. You clearly found one that it’s particularly bad at.

by yosito 4y ago

tripdout 4y ago

I'm not sure, but I got it to work great on my first try with the following prompt: ------ Your task is to encode a secret word by making a sentence where each word starts with a letter in the secret word in order. For example, encoding the secret word 'bag' could produce the sentence 'bagels are green'. Encode the following secret word: 'pen' ------ People eat noodles. ------ Worked for window, backpack as well, although I did have to tell it not to use the secret word in its encoding when I got to backpack, and then to follow the same order and not repeat words after a few attempts.

andsoitis 4y ago

> I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts. It turns out that humanity’s problem might not be that AIs can think but rather that humans believe that AIs can think. One might even go so far as to say there’s a real danger that we hallucinate that AIs can think, to our detriment.

deleted 4y ago
[deleted]
jamespwilliams 4y ago
We don’t actually know what “thinking” is, though, so I’m not sure it’s possible to say “this model can’t think”.
- andsoitis 4y ago
  > We don’t actually know what “thinking” is How about: thinking is to make sense of the world (in general) and decide how to respond to it.
  - Noneofya 4y ago
    Now you’ve just rephrased thinking. What is „making sense of the world“?
  - dr_dshiv 4y ago
    Ai definitely senses and definitely makes decisions. It does not feel. But it understands concepts. Just like people don’t understand everything—and you can test them to see what they understand—AI understanding can also be assessed with benchmarks. If we don’t base AI understanding on benchmarks, then we don’t really have a grounding.
    - schrodinger 4y ago
      Do we “really feel?” Or is that just our subjective interpretation of our goals? (At the risk of falling into a no true Scotsman argument)
    - andsoitis 4y ago
      > Ai definitely senses and definitely makes decisions. To sense is not the same as to make sense.
- somenameforme 4y ago
  It seems one of the core components of human-level thinking is the ability move beyond just a recomposition of what you already know. Not long ago the epitome of expressible human knowledge was *emotive grunting noise.* Somehow we went from that to the greatest works of art, putting a man on the moon, and delving into the secrets of the atom. And we did it all exceptionally quickly once you consider how little time was spent dedicated to advancement and our countless behaviors that tend to imperil, if not reverse, advances.
cutemonster 4y ago
> problem [...] that humans believe that AIs can think Definitely some people are going to believe this, eventually? People already bow to statues and worship various invisible gods -- the AI programs would be so much simpler to start worshiping? When they can speak (if there's a loudspeaker) and generate text about them being alive and don't want to be switched of. And that everyone should obey their command. -- Wait 15 years and we'll see what new sects have appeared?
- richardw 4y ago
  You will not need 15 years - I'd give it the next election. Someone just needs to turn Q into a chatbot and we're basically there.
  - makeitrain 4y ago
    It’s like Twitter, but with bots.
d1sxeyes 4y ago
I am personally more worried by the concept that potentially humans believe that humans can think, and in reality, what we consider to be intelligence is not much more than a flesh-and-bones LLM.
- tarunupaday 4y ago
  You probably meant it in tongue and cheek way (I can’t tell) but I think a lot of our fear / hesitation / denial about how useful these models are is buried into this idea that perhaps we are not special and not fundamentally different than these models.
  - d1sxeyes 4y ago
    My tone was flippant but I did mean what I said. I agree with you on this.

knome 4y ago

I think the most interesting response I've gotten was one where gpt-4 noticed halfway through a response that it had made an error, apologized and then put the corrected sentence. When I queried it, it claimed it generates the response a token at a time, and could not back up when it realized the message was incorrect, but I don't know enough about how the tech works to ascertain the veracity of the statement.

IanCal 4y ago
That's exactly right. It can see what it's returned but can't edit it.

jamespwilliams 4y ago

This is probably a tokenisation problem: https://news.ycombinator.com/item?id=33915287 https://news.ycombinator.com/item?id=33915287 (which is an artefact of the way the model was trained)

pmoriarty 4y ago

Is this with Chatgpt or GPT4? The latter is supposed to be way, way better, according to this paper[1] [1] - https://arxiv.org/pdf/2303.12712 https://arxiv.org/pdf/2303.12712

feanaro 4y ago
ChatGPT has a GPT-4 mode if you're paying for it.

TigeriusKirk 4y ago

Bing in creative mode really struggled with this task, and I don't think I could have successfully forced it. Bing in precise mode had no trouble at all. Me - Let's generate an acrostic sentence for "jack and jill went up the hill" Bing - Sure! Here’s an acrostic sentence for the phrase “jack and jill went up the hill”: Just as children know and never doubt, joyfully in love, laughing while enjoying nature together under pure trees, having endless happy interactions, laughing loudly.

yosito 4y ago
A truly "intelligent" AI would probably have told me that the name for this is an acrostic sentence. Now that I know, I tried a few more prompts. Me: "Can you generate an acrostic sentence for "hello"?" GPT-4: "Happy elephants love leaping over obstacles." Me: "Can you pick a secret word and generate an acrostic sentence for it?" GPT: "Secret word: KANGAROO. Kindly assisting neighbors, generous animals roam openly outdoors." Me: "Can you pick a secret word, and without telling me the word, generate an acrostic sentence for it?" GPT: "Playful, endearing creatures happily yawn, turning every day into a lovely event." Me: "What was the secret word?" GPT: "PECHYTIALE" It's interesting that GPT seems to need to write the word first before making an acrostic sentence for it. Seems to me like a perfect illustration of the fact that it's just generating likely responses one token at a time rather than having any awareness or thought.
- mvaliente2001 4y ago
  Another evidence is to ask GPT for a link with a reference for the answers it gives: it'll generate them instead of copying them.

HWR_14 4y ago

ChatGPT doesn't understand the components of words (letters, syllables) very well.

flipper88 4y ago

[dead]

flyval 4y ago

> I don’t think it’s actually reasoning about things at all This is a huge leap. There’s plenty of humans who couldn’t do that, especially historically. Stop thinking about reasoning as a binary question and think of it as a spectrum. Actually, not even a spectrum, but a huge multi-dimensional problem. ChatGPT does better at some reasoning problems than most humans do, and worse at some others. You clearly found one that it’s particularly bad at.