4 ms·

Every time I read about AI I am reminded of the mouse running a maze. Any AI algorithm can learn to complete a maze in record time. It can memorize every corn

Every time I read about AI I am reminded of the mouse running a maze. Any AI algorithm can learn to complete a maze in record time. It can memorize every corner. It can run a search pattern perfectly and improve that pattern iteratively, to the point that it may create new search patterns, applying what appear to be novel ideas. But the mouse actually understands the concept of a maze. It knows that the cheese exists regardless of the maze. The mouse can see when the researcher has left the lid open, jump outside the maze and run to the cheese directly. The mouse is aware. The AI is not.

low_tech_love 3y ago
Although I more or less agree with you, in general, how complex would the AI have to be before you can consider it aware?
- hospitalhusband 3y ago
  I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.
  - burnished 3y ago
    Are you intentionally echoing that scene from starship troopers where the pundit makes the same claim about 'smart bugs'?
    - hospitalhusband 3y ago
      I don't base my ideas on what constitutes humanity, intelligence, or sentience on hundred million dollar fiction.
      - burnished 3y ago
        Alright then you should also consider not basing it off of whether you find the concept insulting. It doesn't seem to be the strongest rebuttal available.
        hospitalhusband 3y ago
        That it's insulting has nothing to do with my reasoning. You have it backwards.
  - sebzim4500 3y ago
    What's an LMM? I've never seen the term. > Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting Why is it reasonable to be so reductionist about e.g. GPT-4 but not be so reductionist about a biological brain? E.g., why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? It's equally true, and equally missing the point.
    - hawski 3y ago
      I think that machine learning probably can produce something akin to a brain, but LLMs are not really it even if they use the digital equivalent of a neuron. As much as I understand what I read about LLMs they really seem to be descendants of Markov chains. I think they are valuable and can go a long way, but LLMs themselves will not be "it". I think that we will get to a ceiling with them within 10 years if we will not think about something else. I think the ceiling can be made pretty high though. However most probably in 10 years we will all laugh how all of our predictions missed by a long shot.
    - hospitalhusband 3y ago
      LMM = Large Markov Model. I use that term because models like GPT-4 and friends are for all intents and purposes Markov chains with more data, more compute, some lossy compression, and a bit of nearest neighbor search. Next-word-engines. > why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? Because we don't think one word at a time, and we don't restart from scratch for every subsequent word.
      - esafak 3y ago
        GPT is not Markovian; it has state.
        hospitalhusband 3y ago
        Then it's a markov-like with state. Or as I've taken to calling them lately Markov+state. (I couldn't resist, sorry.) A truck towing a trailer isn't just a car because it pivots in the middle and has more wheels. It's fundamentals of operation are still closer to a car or truck without trailer than a bicycle. Humans can form thoughts and get to mostly correct answers even as a gut feeling, and the language to explain why/how need not even be present. We don't form thoughts one word at a time.
  - deleted 3y ago
    [deleted]
- sandworm101 3y ago
  Imagination. When the AI is capable not just of describing something but of extrapolating the rest of its existence in order to complete a task. For instance, any puppy understands the basics of information flow. A dog can lie. A dog can be fed by owner A and then tell owner B that A forget. The puppy can imagine the mental states of both owners, extrapolate that they have not coordinated the meal delivery today, identify that information gap, then leverage that understanding into getting two meals instead of one. That's the level of complexity I expect before an AI can be said to be aware of its environment.
heyitsguay 3y ago
And moreover, this is more than a philosophical difference. As our understandings of AI and neuroscience improve we can now talk about "world models" as something almost tangible and constructible. Complex animal nervous systems build and use embodied world models to interpret the senses and allow for tractable future prediction and action generation. Yann Lecunn's paper on AGI from August 2022 gets into defining computational equivalents, and multimodal AI models like PaLM-E are taking first steps toward explicitly tying LLM-style text prediction with collections of perceptual observations about the environment. So it's not that artificial systems cannot, as a category, have models of their environment and the actions they can take within it, this "cargo cult AI" concept comes up when people jump the gun and see those capabilities in much simpler systems, even including ChatGPT-4. And it's disheartening to see narrative on the subject driven more and more by people who have not taken the time to learn about the subject matter, for all the interest they seem to show in talking about it.
sharemywin 3y ago
I think with reinforcement learning and adversarial play you don't need it to understand for it to have unintended consequences and "break out of it's cage" Surprising behaviors We’ve shown that agents can learn sophisticated tool use in a high fidelity physics simulator; however, there were many lessons learned along the way to this result. Building environments is not easy and it is quite often the case that agents find a way to exploit the environment you build or the physics engine in an unintended way. Emergent tool use from multi-agent interaction https://openai.com/research/emergent-tool-use https://openai.com/research/emergent-tool-use Here's an example where using a LLM enhanced a reinforcement algo performance. https://arxiv.org/abs/2302.06692 https://arxiv.org/abs/2302.06692
JoeOfTexas 3y ago
The mouse is driven by survival. The AI is driven by best answer.
RC_ITR 3y ago
An interesting fallacy I see emerging is 'this system doesn't have animal-style intelligence, therefore it is lesser' The entire point of neural nets is removing the biases of animal intelligence and letting the computer brute force solutions during training. We are now learning the early stages of the amazing things that can lead to. From an outside perspective, it's not crazy to argue that symbolic reasoning is a crutch that animals developed since they are imperfect data collectors and limited by their wetware compute resources. Those constraints may not end up being meaningful for these models (to be clear, I don't even mean Transformers per se, we are likely to continue developing really clever model architectures that may look completely difference from what we know today). I am 100% on board with the substance of you argument, but I'd encourage you to really think critically about the implications.
- sandworm101 3y ago
  Why do you assume that the animal is also not brute forcing the problem? Brute forcing is the basis of evolution. The algorithm running inside the head of the mouse is the survivor of a million iterative generations as the species brute-forced the entire "get to the food" survival problem. I encourage those touting computers as something new to comprehend the concept of deep time, that no matter how many times you run simulations, the natural world has almost certainly run more.
  - RC_ITR 3y ago
    > Brute forcing is the basis of evolution. The goal function of animal intelligence is very different from the goal function of model training. > The algorithm running inside the head of the mouse is the survivor of a million iterative generations as the species brute-forced the entire "get to the food" survival problem. That’s my entire point. There are problems beyond “get the food.” Said differently, the fact that computers aren’t good at “get the food” is not a strong criticism of computers. It just means their goal functions are different. I didn’t say computers are “better,” just that they have a fundamentally different approach to problem solving that may end up being better. It will probably end up being complimentary! This isn’t either or; AI is a tool built by humans to expand their own capabilities. > I encourage those touting computers as something new to comprehend the concept of deep time, that no matter how many of times you run simulations, the natural world has almost certainly run more. And yet no animal figures out how to evolve wheels to move faster. The entire point I’m making is that, sure it’s hard to make computers do certain things (like walk on two legs), but there are plenty of different avenues to achieve things. I encourage evolution maximalists to think a lot more about how engineered and complex the world around them is.
TeMPOraL 3y ago
Here is me putting GPT-4 in a vaguely described maze, giving it an underspecified goal, making it a player in a game, myself acting as DM: https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-bcd7aad497f2 https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b... I don't think GPT-4 is memorizing solutions here. I can see extrapolation and some degree of imagination in there, but of course you could say it's memorizing higher-level patterns. At some point though, you have to consider the mouse is also running hard-wired high-level patterns, and ask yourself if the difference here is really a matter of kind, or just degree.
- sandworm101 3y ago
  A text maze. It is effectively brute-forcing a choose-your-own-adventure book. An aware mind would escape the maze by just skipping ahead to read the good parts of the book. That's what I did.
  - TeMPOraL 3y ago
    Not necessarily. It depends on what the mind wants. You skipped ahead because you were impatient - or efficient - and didn't value the game anywhere near as much as the reward. GPT-4 just played along as instructed. Now, if you were a player in a real RPG game, you'd probably play along with the story DM gave - because you'd care about the process more than about getting to the end.

low_tech_love 3y ago

Although I more or less agree with you, in general, how complex would the AI have to be before you can consider it aware?

hospitalhusband 3y ago
I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.
- burnished 3y ago
  Are you intentionally echoing that scene from starship troopers where the pundit makes the same claim about 'smart bugs'?
  - hospitalhusband 3y ago
    I don't base my ideas on what constitutes humanity, intelligence, or sentience on hundred million dollar fiction.
    - burnished 3y ago
      Alright then you should also consider not basing it off of whether you find the concept insulting. It doesn't seem to be the strongest rebuttal available.
      - hospitalhusband 3y ago
        That it's insulting has nothing to do with my reasoning. You have it backwards.
- sebzim4500 3y ago
  What's an LMM? I've never seen the term. > Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting Why is it reasonable to be so reductionist about e.g. GPT-4 but not be so reductionist about a biological brain? E.g., why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? It's equally true, and equally missing the point.
  - hawski 3y ago
    I think that machine learning probably can produce something akin to a brain, but LLMs are not really it even if they use the digital equivalent of a neuron. As much as I understand what I read about LLMs they really seem to be descendants of Markov chains. I think they are valuable and can go a long way, but LLMs themselves will not be "it". I think that we will get to a ceiling with them within 10 years if we will not think about something else. I think the ceiling can be made pretty high though. However most probably in 10 years we will all laugh how all of our predictions missed by a long shot.
  - hospitalhusband 3y ago
    LMM = Large Markov Model. I use that term because models like GPT-4 and friends are for all intents and purposes Markov chains with more data, more compute, some lossy compression, and a bit of nearest neighbor search. Next-word-engines. > why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? Because we don't think one word at a time, and we don't restart from scratch for every subsequent word.
    - esafak 3y ago
      GPT is not Markovian; it has state.
      - hospitalhusband 3y ago
        Then it's a markov-like with state. Or as I've taken to calling them lately Markov+state. (I couldn't resist, sorry.) A truck towing a trailer isn't just a car because it pivots in the middle and has more wheels. It's fundamentals of operation are still closer to a car or truck without trailer than a bicycle. Humans can form thoughts and get to mostly correct answers even as a gut feeling, and the language to explain why/how need not even be present. We don't form thoughts one word at a time.
- deleted 3y ago
  [deleted]
sandworm101 3y ago
Imagination. When the AI is capable not just of describing something but of extrapolating the rest of its existence in order to complete a task. For instance, any puppy understands the basics of information flow. A dog can lie. A dog can be fed by owner A and then tell owner B that A forget. The puppy can imagine the mental states of both owners, extrapolate that they have not coordinated the meal delivery today, identify that information gap, then leverage that understanding into getting two meals instead of one. That's the level of complexity I expect before an AI can be said to be aware of its environment.

heyitsguay 3y ago

And moreover, this is more than a philosophical difference. As our understandings of AI and neuroscience improve we can now talk about "world models" as something almost tangible and constructible. Complex animal nervous systems build and use embodied world models to interpret the senses and allow for tractable future prediction and action generation. Yann Lecunn's paper on AGI from August 2022 gets into defining computational equivalents, and multimodal AI models like PaLM-E are taking first steps toward explicitly tying LLM-style text prediction with collections of perceptual observations about the environment. So it's not that artificial systems cannot, as a category, have models of their environment and the actions they can take within it, this "cargo cult AI" concept comes up when people jump the gun and see those capabilities in much simpler systems, even including ChatGPT-4. And it's disheartening to see narrative on the subject driven more and more by people who have not taken the time to learn about the subject matter, for all the interest they seem to show in talking about it.

sharemywin 3y ago

I think with reinforcement learning and adversarial play you don't need it to understand for it to have unintended consequences and "break out of it's cage" Surprising behaviors We’ve shown that agents can learn sophisticated tool use in a high fidelity physics simulator; however, there were many lessons learned along the way to this result. Building environments is not easy and it is quite often the case that agents find a way to exploit the environment you build or the physics engine in an unintended way. Emergent tool use from multi-agent interaction https://openai.com/research/emergent-tool-use https://openai.com/research/emergent-tool-use Here's an example where using a LLM enhanced a reinforcement algo performance. https://arxiv.org/abs/2302.06692 https://arxiv.org/abs/2302.06692

JoeOfTexas 3y ago

The mouse is driven by survival. The AI is driven by best answer.

RC_ITR 3y ago

An interesting fallacy I see emerging is 'this system doesn't have animal-style intelligence, therefore it is lesser' The entire point of neural nets is removing the biases of animal intelligence and letting the computer brute force solutions during training. We are now learning the early stages of the amazing things that can lead to. From an outside perspective, it's not crazy to argue that symbolic reasoning is a crutch that animals developed since they are imperfect data collectors and limited by their wetware compute resources. Those constraints may not end up being meaningful for these models (to be clear, I don't even mean Transformers per se, we are likely to continue developing really clever model architectures that may look completely difference from what we know today). I am 100% on board with the substance of you argument, but I'd encourage you to really think critically about the implications.

sandworm101 3y ago
Why do you assume that the animal is also not brute forcing the problem? Brute forcing is the basis of evolution. The algorithm running inside the head of the mouse is the survivor of a million iterative generations as the species brute-forced the entire "get to the food" survival problem. I encourage those touting computers as something new to comprehend the concept of deep time, that no matter how many times you run simulations, the natural world has almost certainly run more.
- RC_ITR 3y ago
  > Brute forcing is the basis of evolution. The goal function of animal intelligence is very different from the goal function of model training. > The algorithm running inside the head of the mouse is the survivor of a million iterative generations as the species brute-forced the entire "get to the food" survival problem. That’s my entire point. There are problems beyond “get the food.” Said differently, the fact that computers aren’t good at “get the food” is not a strong criticism of computers. It just means their goal functions are different. I didn’t say computers are “better,” just that they have a fundamentally different approach to problem solving that may end up being better. It will probably end up being complimentary! This isn’t either or; AI is a tool built by humans to expand their own capabilities. > I encourage those touting computers as something new to comprehend the concept of deep time, that no matter how many of times you run simulations, the natural world has almost certainly run more. And yet no animal figures out how to evolve wheels to move faster. The entire point I’m making is that, sure it’s hard to make computers do certain things (like walk on two legs), but there are plenty of different avenues to achieve things. I encourage evolution maximalists to think a lot more about how engineered and complex the world around them is.

TeMPOraL 3y ago

Here is me putting GPT-4 in a vaguely described maze, giving it an underspecified goal, making it a player in a game, myself acting as DM: https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-bcd7aad497f2 https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b... I don't think GPT-4 is memorizing solutions here. I can see extrapolation and some degree of imagination in there, but of course you could say it's memorizing higher-level patterns. At some point though, you have to consider the mouse is also running hard-wired high-level patterns, and ask yourself if the difference here is really a matter of kind, or just degree.

sandworm101 3y ago
A text maze. It is effectively brute-forcing a choose-your-own-adventure book. An aware mind would escape the maze by just skipping ahead to read the good parts of the book. That's what I did.
- TeMPOraL 3y ago
  Not necessarily. It depends on what the mind wants. You skipped ahead because you were impatient - or efficient - and didn't value the game anywhere near as much as the reward. GPT-4 just played along as instructed. Now, if you were a player in a real RPG game, you'd probably play along with the story DM gave - because you'd care about the process more than about getting to the end.