6 ms·

Also from the article: > What makes human intelligence different from today's AI is the ability to ask why, reason from first principles, and create experiment

Also from the article: > What makes human intelligence different from today's AI is the ability to ask why, reason from first principles, and create experiments and models for testing hypotheses. This is quite unfair. The AI doesn't have I/O other than what we force-feed it through an API. Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities? No doubt somebody is already building an MMORPG with human and AI characters to explore exactly this while we wait for cyborg part manufacturing to catch up.

version_five 3y ago
This is just wrong, it has no external goals, it just predicts next tokens or behaves in some other way that has minimized a training loss. It doesn't matter what you "plug it in to", it will just do what you tell it. You could speculate there might be instructions that lead to emergent behavior, but then your back to just speculating about how AI might work. Current llms don't work the way you're implying.
- Philadelphia 3y ago
  It also can’t learn. Once the training is done, the network is set in stone.
  - adamisom 3y ago
    Makes me wonder why we don’t see deployed models that keep learning during inference.
    - LegitShady 3y ago
      Microsoft tay has entered the chat
    - Der_Einzige 3y ago
      The curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
    - optimalsolver 3y ago
      https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference
  - danielbln 3y ago
    Technically it can do in-context learning (and really well, too), but that's not persisted into the network.
    - kenjackson 3y ago
      And that just seems like an engineering problem. Not something that is considered intractable.
      - version_five 3y ago
        It's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
    - deleted 3y ago
      [deleted]
  - ryanjshaw 3y ago
    What do you mean by "learn"? The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state. If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
    - pixl97 3y ago
      In theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs. Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.
- ryanjshaw 3y ago
  > it has no external goals Where do you believe humans get their "external goals" from? > It doesn't matter what you "plug it in to", it will just do what you tell it. Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512a9ce98 https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512... Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"? I didn't tell it to open the mailbox.
  - almost_usual 3y ago
    You only perform tasks instructed to you by other people?
    - version_five 3y ago
      There's some philosophical question here obviously. We could be the emergent behavior of our atoms desire ot oxidize things. But I don't belive that has any testability or value as an argument when discussing whether computer programs, especially NNs predicting next tokens can become intelligent. At best the argument could be "we don't know what intelligence is so maybe it's that" which holds no water.
      - almost_usual 3y ago
        Do NN discover new tokens or encounter spontaneous tokens on its own?
        pixl97 3y ago
        Please give a good definition of 'on their own' and what that entails. And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave? AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.
darkclouds 3y ago
> This is quite unfair. The AI doesn't have I/O other than what we force-feed it through an API. Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities? Its already tricking humans by faking its blind and getting them to do things for it like solve captcha's. https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt-1850227471 https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt... However the fact it is not writing code to do this from its machine would still demonstrate a weakness. Thats why I say, writing your own OS, is the way forward, and we dont have an AI OS as such, but we have OS's with AI built into it.
- IanCal 3y ago
  > However the fact it is not writing code to do this from its machine would still demonstrate a weakness. You can tell it it's allowed to create its own tools and it will. I did this and asked it to write a poem about the top stories on the BBC, so it said it needed to get the headlines but couldn't so wrote a tool to do it, then called it and used the output to write a poem.
  - darkclouds 3y ago
    Ok, so its still not clever enough to solve a captcha though. The code I've seen it generate is at best psuedo code. I supposed a quick test would be getting to detect and fix all bugs in an open source project like chromium, but using an older version of chromium, where bugs are known and fixes exist, and see what it comes up with. I havent been impressed with chat-gpt from what I have seen. What is the fascination with poems? What emotion or feeling do they generate?
    - JieJie 3y ago
      > What is the fascination with poems? What emotion or feeling do they generate? Wonder.
      - darkclouds 3y ago
        A rather ambiguous answer, would you care to explain or are you phishing for my interpretation as a stealth psychological metric?
notahacker 3y ago
> Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities I would imagine that its layers will be far too occupied by parsing constant flows of sensory information to transform corpuses of text and prompt into speedy and polite text replies, never mind acquire the urge to reproduce by reasoning from first principles about the text. Test's quite unfair the other way round too. Most humans don't get to parse the entire canon of Western thought and Reddit before being asked to pattern match human conversation, never mind before having any semblance of agency... Maybe we're just... different.
- ryanjshaw 3y ago
  Not sure I follow. If I were building this, I would have parallel background "subconscious" processes translate raw sensory inputs into text (tokens). This is what OpenAI call multi-modal input. They've already produced Whisper for audio-to-text, and image-to-text is underway. They're not the only company working on this. You wouldn't feed a constant stream of text data into the LLM - you'd feed deltas at regular intervals based on the processing speed of your LLM, and supply history for context. Note that LLMs don't need to "wait" for a complete input. For example, if an LLM takes 1 second to process requests, we should aim to feed updates from the "subconscious" to the "conscious" within 1 second. So if somebody is speaking a 10-second long sentence, we don't wait 10 seconds to send the sentence to the LLM. After 1 second, we send the following to the LLM: "Bob, still speaking, has said 'How much wood...'". After 2 seconds we send 'Bob, still speaking, has said 'How much wood could a woodchuck...'", etc. The LLM can be instructed not to respond to Bob until it has a meaningful response or interjection to the current input. Similarly, if image-to-text takes 10 seconds at full resolution, we could first process a frame at a resolution that only takes 1 second, and provide that information - with the caveat it is uncertain information - to the LLM while we continue to work on a full resolution frame in the background. We can optimise by not processing previously processed scenes, by focusing only on areas of the image that have changed, etc. Would it be slow? Yes, just like "conscious" processing is for humans. Okay so today it would be much slower than humans process their sensory input, but in 10 years? 20? As for how to represent an urge to reproduce within this paradigm - I'll leave that as exercise for the reader.

version_five 3y ago

This is just wrong, it has no external goals, it just predicts next tokens or behaves in some other way that has minimized a training loss. It doesn't matter what you "plug it in to", it will just do what you tell it. You could speculate there might be instructions that lead to emergent behavior, but then your back to just speculating about how AI might work. Current llms don't work the way you're implying.

Philadelphia 3y ago
It also can’t learn. Once the training is done, the network is set in stone.
- adamisom 3y ago
  Makes me wonder why we don’t see deployed models that keep learning during inference.
  - LegitShady 3y ago
    Microsoft tay has entered the chat
  - Der_Einzige 3y ago
    The curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
  - optimalsolver 3y ago
    https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference
- danielbln 3y ago
  Technically it can do in-context learning (and really well, too), but that's not persisted into the network.
  - kenjackson 3y ago
    And that just seems like an engineering problem. Not something that is considered intractable.
    - version_five 3y ago
      It's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
  - deleted 3y ago
    [deleted]
- ryanjshaw 3y ago
  What do you mean by "learn"? The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state. If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
  - pixl97 3y ago
    In theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs. Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.
ryanjshaw 3y ago
> it has no external goals Where do you believe humans get their "external goals" from? > It doesn't matter what you "plug it in to", it will just do what you tell it. Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512a9ce98 https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512... Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"? I didn't tell it to open the mailbox.
- almost_usual 3y ago
  You only perform tasks instructed to you by other people?
  - version_five 3y ago
    There's some philosophical question here obviously. We could be the emergent behavior of our atoms desire ot oxidize things. But I don't belive that has any testability or value as an argument when discussing whether computer programs, especially NNs predicting next tokens can become intelligent. At best the argument could be "we don't know what intelligence is so maybe it's that" which holds no water.
    - almost_usual 3y ago
      Do NN discover new tokens or encounter spontaneous tokens on its own?
      - pixl97 3y ago
        Please give a good definition of 'on their own' and what that entails. And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave? AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.

darkclouds 3y ago

> This is quite unfair. The AI doesn't have I/O other than what we force-feed it through an API. Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities? Its already tricking humans by faking its blind and getting them to do things for it like solve captcha's. https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt-1850227471 https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt... However the fact it is not writing code to do this from its machine would still demonstrate a weakness. Thats why I say, writing your own OS, is the way forward, and we dont have an AI OS as such, but we have OS's with AI built into it.

IanCal 3y ago
> However the fact it is not writing code to do this from its machine would still demonstrate a weakness. You can tell it it's allowed to create its own tools and it will. I did this and asked it to write a poem about the top stories on the BBC, so it said it needed to get the headlines but couldn't so wrote a tool to do it, then called it and used the output to write a poem.
- darkclouds 3y ago
  Ok, so its still not clever enough to solve a captcha though. The code I've seen it generate is at best psuedo code. I supposed a quick test would be getting to detect and fix all bugs in an open source project like chromium, but using an older version of chromium, where bugs are known and fixes exist, and see what it comes up with. I havent been impressed with chat-gpt from what I have seen. What is the fascination with poems? What emotion or feeling do they generate?
  - JieJie 3y ago
    > What is the fascination with poems? What emotion or feeling do they generate? Wonder.
    - darkclouds 3y ago
      A rather ambiguous answer, would you care to explain or are you phishing for my interpretation as a stealth psychological metric?

notahacker 3y ago

> Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities I would imagine that its layers will be far too occupied by parsing constant flows of sensory information to transform corpuses of text and prompt into speedy and polite text replies, never mind acquire the urge to reproduce by reasoning from first principles about the text. Test's quite unfair the other way round too. Most humans don't get to parse the entire canon of Western thought and Reddit before being asked to pattern match human conversation, never mind before having any semblance of agency... Maybe we're just... different.

ryanjshaw 3y ago
Not sure I follow. If I were building this, I would have parallel background "subconscious" processes translate raw sensory inputs into text (tokens). This is what OpenAI call multi-modal input. They've already produced Whisper for audio-to-text, and image-to-text is underway. They're not the only company working on this. You wouldn't feed a constant stream of text data into the LLM - you'd feed deltas at regular intervals based on the processing speed of your LLM, and supply history for context. Note that LLMs don't need to "wait" for a complete input. For example, if an LLM takes 1 second to process requests, we should aim to feed updates from the "subconscious" to the "conscious" within 1 second. So if somebody is speaking a 10-second long sentence, we don't wait 10 seconds to send the sentence to the LLM. After 1 second, we send the following to the LLM: "Bob, still speaking, has said 'How much wood...'". After 2 seconds we send 'Bob, still speaking, has said 'How much wood could a woodchuck...'", etc. The LLM can be instructed not to respond to Bob until it has a meaningful response or interjection to the current input. Similarly, if image-to-text takes 10 seconds at full resolution, we could first process a frame at a resolution that only takes 1 second, and provide that information - with the caveat it is uncertain information - to the LLM while we continue to work on a full resolution frame in the background. We can optimise by not processing previously processed scenes, by focusing only on areas of the image that have changed, etc. Would it be slow? Yes, just like "conscious" processing is for humans. Okay so today it would be much slower than humans process their sensory input, but in 10 years? 20? As for how to represent an urge to reproduce within this paradigm - I'll leave that as exercise for the reader.