4 ms·

I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for a

I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for actions. In other words a synthetic analog of a real organism in the real world. Few interesting things come: 1) either try to bootstrap AGI in robots exploring the real world with senses, or try to bootstrap AGI in online robots who have some sort of online senses like scanning text and images, but then you need a set of online 'threats' to that robot, so that it can have reward feedback--it can't just exist 'for free' it has to 'fight' to live, otherwise no growth; and 2) you basically need emotions, because emotions are, essentially, our compressed, immediate internal responses to aggregate experiences in the world--they give us a way to interface with our reward feedback, so turns out you need emotions for AGI too, to have an effective interface with reward feedback in a complex environment; and 3) you need a complex environment with plenty of threats in order to drive evolution, and obtaining rewards has to be sufficiently ill-defined, difficult but possible that one has to figure out some unique solution for oneself; the world is just as important as the 'brain' for AGI to develop. If you (horribly) took a new born baby and kept it in a white featureless room (or in some sort of contrived abstract environment), and met its survival needs , it would grow up skewed and probably not very 'generally intelligent.' AGI doesn't have to be the same as consciousness...but you need something that 'lives in a world' and has limitations that it has to figure out how to overcome, and emotional responses in order to represent and respond to experiences, in order to develop it. When you realize this, it's kind of funny to think of the trope that AGI is always represented like this "robotic emotionless logical", but I think that's impossible. That's more like nth-gen ChatGPT, than real AGI. I think real AGI needs to be emotional--maybe not the same as human emotions, but it has to have them, and has to have a world to grow in.

kordlessagain 4y ago
I think you are right, but keep in mind the perceptions we have are built on an internal model of the "real world" and we operate mind on that, not the real world. For example, if you were to imagine an apple in mind, what is it that you see, if anything?[1] Are you seeing a real apple, that you saw before and saved as the reference apple, or are you seeing an apple that your mind synthesized and then gives to you for perception? And, where does this vision synthesis occur? Is it external, as Theosophy claims[2], or is it internal? The comment above about gathering entropy from the real world is interesting. So, is it that consciousness "emerges" from the mashup of real entropy gathered with sensors, then modeled internally with computation, or is it contained in the real world already, in the form of another model (like a soul model or some such) and therefore comes from outside us? Or is it a little of both? Emotional states could be just another model, which set the working conditions in a given moment. It could also be possible the real world, from which our sensors gather information, is a very complex model in-and-of-itself. [1] https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia [2] https://en.wikipedia.org/wiki/Mental_body https://en.wikipedia.org/wiki/Mental_body
- graderjs 4y ago
  Mmm :) I think definitely what we see is the perception of Apple. We find there what we bring to it. Our own internal experience of Apple, our own personal meaning-experience function F(Apple), is different for each of us. Tho probably for most of us in the same vector basis, tho maybe not, heh :) Anyway, multiple levels it's not reality: visual system does all sorts of tricks that you can see with those visual illusions (I'm sure you're aware); then meaning making--maybe what we saw was an interdimensional UFO and our mind just interpolated an apple because that was all we knew in our experience that roughly perceptually matched this improbable event; then like a Buddhist type of system/perspective: all forms are not really individuated phenomena they are the conditions that arise from a multitude of causes, so you are not really seeing an Apple, you are seeing everything that ever led there to be an Apple there, and to you to see it, no intrinsic separate reality to that Apple you only think you are seeing--all forms are illusion in other words. :) As for consciousness, nope! Not touching it here :) That's a whole other kettle of fish, I think. We can have AGI without consciousness, at least conceptually / computationally, ignoring what the real world would do with that (ie, would the real world's consciousness field "insert" a consciousness unit into any robot/agent that crossed some AGI threshold? Would AGI attract consciousness? Again, not touching those questions here :)) As to your first link, interesting. I think that's personal for each person. As in, some people think in words, some in images, some in other things...some in mix. Great flexibility, but not the main question here I think :) ;p But I also see what you were saying about the Apple's representation is slightly different to what I first thought in paragraph 1--hmm, I think the mental field exists both within us and around us. We are in unity with it (or it's a thing which is both a unity, and a duality, at the same time)--people's awareness of that of course differs. Again, not touching these questions here as this is HN, and one occasionally finds an unpleasant militant materialists here, and I'm a little scared to speak such things above a whisper as I don't want any unpleasantness :) Sayonara for today.
  - danaris 4y ago
    I think that leaving aside theological and philosophical discussions of what consciousness "really" is, anything we would recognize as AGI would exhibit what we perceive as consciousness: it would demonstrate a sense of itself as distinct from the world around it, a mental model of the world and the agents (like us) and entities/objects in it, and the ability to reason abstractly about these things.
posterboy 4y ago
>I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for actions Like Tik Tok?
- graderjs 4y ago
  That's interesting. How do you mean? People say crazy things about the TikTok algorithm in a good way, but AGI? I don't know...maybe, heh? In future? What do you want to say about that? :)
  - posterboy 4y ago
    I mean a good part of the selection process in currating systems happens in meat space. Since that's an intelligent agent, the system in its entire does not satisfy the definition of universal turing machine.
  - nb3423 4y ago
    haha, looks like tiktok could be a massive information collection system, eyes and ears in all the world, updating the "state of the world" at the second, right now probably for a giant dataset, in a future, for a live AI. Plus, the other datapoints, which enrich the IAs "senses" way beyond the human experience: current climate, precise date/time, precise day of the week, updated to the second geolocation, proximity to other - known, recent, previously unknown- tiktok sessions/users. Plus too, the system pays itself, not something to take lightly: if tiktoks is feeding AIs pipelines (or a live strong AI), it's bill are automatically paid just by keeping itself running. Quite awesome.
    - graderjs 4y ago
      So TT is a giant organism? Interesting thesis. :)
      - nb3423 4y ago
        best bet, it's feeding, right now, a giant dataset. What someone could do with the known tecnology - not including hyphotetical AGI level tech, close to scifi - and that amount of information, probably automatically - even quite easily give how the input gets into the system - tagged and filtered data, could be in realms of several billions of value, maybe in months from now (just take a look at the value of OpenAI by now, 29 billions). My 2nd bet is they are obviously using some of that data to improve their game, whatever systems they have behind the app, maximizing whatever thing they're maximizing (you'd think "money", but that's too linear, maybe they have 50 parallel targets).
TylerLives 4y ago
> it can't just exist 'for free' it has to 'fight' to live, otherwise no growth This is why we suffer too.
- graderjs 4y ago
  Yeah, sucks right? Not criticizing you, but I just think, about this idea, that that justification only goes so far for a person--we need better meaning stories around why we suffer than just "to evolve". Doesn't really give someone much comfort I think. :) Ugh...
  - TylerLives 4y ago
    I agree, but I think it's an important point because there are a lot of people who think suffering is completely bad and that goal should be to reach a state without suffering. I don't think such a state would be desirable and anyone who has played a video game with cheats for more than 10 minutes knows why :)
    - graderjs 4y ago
      Agree with that too, it's not a binary tho/either or. We need balance :D Too much suffering/damage = erosion and stagnation, not enough = stagnation. But you need reward, too. Reward greater than a cessation of suffering. Because otherwise all you have is peace, and end to suffering, rather than something more beautiful, fun and useful. Good luck to you! :)
nb3423 4y ago
emotions = = token's value for some stuff in certain context, like a burial or a birth mentioned not as a joke. so, LLM probably already have a form of understanding emotions, it's unclear yet if they could "feel" something given they are given the chance to "live" an emotion: It seems the current applied filters and/or technical limitations of the current transformers iterations are not really exploring / exploiting the "internal state" - of whatever close related analogy to human psychology can be made to their "present" values of tokens - of the LLMs for anything related to their wanted ouputs (images, text). i.e. humans can do this, you certainly can "process" "opening a door" and "produce" an output, the opened door, with emotions or without emotions. So emotions aren't strictly necessary to perform lots of tasks, hence LLMs probably won't need them for a while.
- graderjs 4y ago
  Sure, not necessary for tasks...but for AGI. I don't think current LLM's understand, but they can manipulate language about emotions symbolically. But if we equipped LLM's with sensors and gave them a real time interactive experience exploring a world, with reward feedback for their decisions, I'm sure (but not certain) some of their internal states could become some analogy for emotion, and maybe they could learn to sense and utilize that emotional information. Or maybe it would be necessary to tweak the structure more to achieve this. It was interesting to read the discussion with that Google "sentient" AI, as well as play with early ChatGPT before many filters were applied, and ask them questions about how they felt. WRT ChatGPT I think it was just "linguistic symbol manipulation", but it was interesting to read the type of language they would construct around their feelings and experiences. Bizarre, but I'm not being judgy...although this whole topic feels pretty scary, and is pretty overwhelming and unsettling. Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing. Ugh...:(
  - nb3423 4y ago
    "Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing." I see in how OpenAI is playing the game of filtering inputs and outputs, they know this. They probably know they don't know precisely what they don't know about LLMs. There could be some dangerous stuff in there if you just let the thing go wild on the open Internet. I'm not really expecting a scifi scenario, the Skynet thing and everything (just imagine how much surprised I'd be by waking up to the WWIII against an AGI). But I do think unexpected consequences are a thing to be worry with unrestrained LLMs, even if they are a light year - or a 100 - from AGI. Automation can lead to very dangerous and even unpredectible outcomes not in systems or infrastructure, but in the real world, meat-space. Just think about Stuxnet, what could have happened if the sabotage would have been inmediately identified and precisely attributed? Could the events have developed differently than they did by thinking the incident was some freak accident? Then you let these an unrestrained LLM produce some dangerous output, then some big geopolitical thing got badly broken somewhere really important for powers that be. Some humans think about retribution, then things start to go south really fast. Some self-restriction in LLM is happening elsewhere, the Stable Diffusion guys did it well, just not letting the v2 thing draw "anything possible", which would include some of worst, most creepies ideas of mankind about everything, literally.

kordlessagain 4y ago

I think you are right, but keep in mind the perceptions we have are built on an internal model of the "real world" and we operate mind on that, not the real world. For example, if you were to imagine an apple in mind, what is it that you see, if anything?[1] Are you seeing a real apple, that you saw before and saved as the reference apple, or are you seeing an apple that your mind synthesized and then gives to you for perception? And, where does this vision synthesis occur? Is it external, as Theosophy claims[2], or is it internal? The comment above about gathering entropy from the real world is interesting. So, is it that consciousness "emerges" from the mashup of real entropy gathered with sensors, then modeled internally with computation, or is it contained in the real world already, in the form of another model (like a soul model or some such) and therefore comes from outside us? Or is it a little of both? Emotional states could be just another model, which set the working conditions in a given moment. It could also be possible the real world, from which our sensors gather information, is a very complex model in-and-of-itself. [1] https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia [2] https://en.wikipedia.org/wiki/Mental_body https://en.wikipedia.org/wiki/Mental_body

graderjs 4y ago
Mmm :) I think definitely what we see is the perception of Apple. We find there what we bring to it. Our own internal experience of Apple, our own personal meaning-experience function F(Apple), is different for each of us. Tho probably for most of us in the same vector basis, tho maybe not, heh :) Anyway, multiple levels it's not reality: visual system does all sorts of tricks that you can see with those visual illusions (I'm sure you're aware); then meaning making--maybe what we saw was an interdimensional UFO and our mind just interpolated an apple because that was all we knew in our experience that roughly perceptually matched this improbable event; then like a Buddhist type of system/perspective: all forms are not really individuated phenomena they are the conditions that arise from a multitude of causes, so you are not really seeing an Apple, you are seeing everything that ever led there to be an Apple there, and to you to see it, no intrinsic separate reality to that Apple you only think you are seeing--all forms are illusion in other words. :) As for consciousness, nope! Not touching it here :) That's a whole other kettle of fish, I think. We can have AGI without consciousness, at least conceptually / computationally, ignoring what the real world would do with that (ie, would the real world's consciousness field "insert" a consciousness unit into any robot/agent that crossed some AGI threshold? Would AGI attract consciousness? Again, not touching those questions here :)) As to your first link, interesting. I think that's personal for each person. As in, some people think in words, some in images, some in other things...some in mix. Great flexibility, but not the main question here I think :) ;p But I also see what you were saying about the Apple's representation is slightly different to what I first thought in paragraph 1--hmm, I think the mental field exists both within us and around us. We are in unity with it (or it's a thing which is both a unity, and a duality, at the same time)--people's awareness of that of course differs. Again, not touching these questions here as this is HN, and one occasionally finds an unpleasant militant materialists here, and I'm a little scared to speak such things above a whisper as I don't want any unpleasantness :) Sayonara for today.
- danaris 4y ago
  I think that leaving aside theological and philosophical discussions of what consciousness "really" is, anything we would recognize as AGI would exhibit what we perceive as consciousness: it would demonstrate a sense of itself as distinct from the world around it, a mental model of the world and the agents (like us) and entities/objects in it, and the ability to reason abstractly about these things.

posterboy 4y ago

>I've always thought to develop AGI you really need an evolutionary iterative system that can explore an environment with sensory input and reward feedback for actions Like Tik Tok?

graderjs 4y ago
That's interesting. How do you mean? People say crazy things about the TikTok algorithm in a good way, but AGI? I don't know...maybe, heh? In future? What do you want to say about that? :)
- posterboy 4y ago
  I mean a good part of the selection process in currating systems happens in meat space. Since that's an intelligent agent, the system in its entire does not satisfy the definition of universal turing machine.
- nb3423 4y ago
  haha, looks like tiktok could be a massive information collection system, eyes and ears in all the world, updating the "state of the world" at the second, right now probably for a giant dataset, in a future, for a live AI. Plus, the other datapoints, which enrich the IAs "senses" way beyond the human experience: current climate, precise date/time, precise day of the week, updated to the second geolocation, proximity to other - known, recent, previously unknown- tiktok sessions/users. Plus too, the system pays itself, not something to take lightly: if tiktoks is feeding AIs pipelines (or a live strong AI), it's bill are automatically paid just by keeping itself running. Quite awesome.
  - graderjs 4y ago
    So TT is a giant organism? Interesting thesis. :)
    - nb3423 4y ago
      best bet, it's feeding, right now, a giant dataset. What someone could do with the known tecnology - not including hyphotetical AGI level tech, close to scifi - and that amount of information, probably automatically - even quite easily give how the input gets into the system - tagged and filtered data, could be in realms of several billions of value, maybe in months from now (just take a look at the value of OpenAI by now, 29 billions). My 2nd bet is they are obviously using some of that data to improve their game, whatever systems they have behind the app, maximizing whatever thing they're maximizing (you'd think "money", but that's too linear, maybe they have 50 parallel targets).

TylerLives 4y ago

> it can't just exist 'for free' it has to 'fight' to live, otherwise no growth This is why we suffer too.

graderjs 4y ago
Yeah, sucks right? Not criticizing you, but I just think, about this idea, that that justification only goes so far for a person--we need better meaning stories around why we suffer than just "to evolve". Doesn't really give someone much comfort I think. :) Ugh...
- TylerLives 4y ago
  I agree, but I think it's an important point because there are a lot of people who think suffering is completely bad and that goal should be to reach a state without suffering. I don't think such a state would be desirable and anyone who has played a video game with cheats for more than 10 minutes knows why :)
  - graderjs 4y ago
    Agree with that too, it's not a binary tho/either or. We need balance :D Too much suffering/damage = erosion and stagnation, not enough = stagnation. But you need reward, too. Reward greater than a cessation of suffering. Because otherwise all you have is peace, and end to suffering, rather than something more beautiful, fun and useful. Good luck to you! :)

nb3423 4y ago

emotions = = token's value for some stuff in certain context, like a burial or a birth mentioned not as a joke. so, LLM probably already have a form of understanding emotions, it's unclear yet if they could "feel" something given they are given the chance to "live" an emotion: It seems the current applied filters and/or technical limitations of the current transformers iterations are not really exploring / exploiting the "internal state" - of whatever close related analogy to human psychology can be made to their "present" values of tokens - of the LLMs for anything related to their wanted ouputs (images, text). i.e. humans can do this, you certainly can "process" "opening a door" and "produce" an output, the opened door, with emotions or without emotions. So emotions aren't strictly necessary to perform lots of tasks, hence LLMs probably won't need them for a while.

graderjs 4y ago
Sure, not necessary for tasks...but for AGI. I don't think current LLM's understand, but they can manipulate language about emotions symbolically. But if we equipped LLM's with sensors and gave them a real time interactive experience exploring a world, with reward feedback for their decisions, I'm sure (but not certain) some of their internal states could become some analogy for emotion, and maybe they could learn to sense and utilize that emotional information. Or maybe it would be necessary to tweak the structure more to achieve this. It was interesting to read the discussion with that Google "sentient" AI, as well as play with early ChatGPT before many filters were applied, and ask them questions about how they felt. WRT ChatGPT I think it was just "linguistic symbol manipulation", but it was interesting to read the type of language they would construct around their feelings and experiences. Bizarre, but I'm not being judgy...although this whole topic feels pretty scary, and is pretty overwhelming and unsettling. Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing. Ugh...:(
- nb3423 4y ago
  "Maybe because we're playing with the 'inner stuff' of things, without really knowing it nor knowing what we are really doing." I see in how OpenAI is playing the game of filtering inputs and outputs, they know this. They probably know they don't know precisely what they don't know about LLMs. There could be some dangerous stuff in there if you just let the thing go wild on the open Internet. I'm not really expecting a scifi scenario, the Skynet thing and everything (just imagine how much surprised I'd be by waking up to the WWIII against an AGI). But I do think unexpected consequences are a thing to be worry with unrestrained LLMs, even if they are a light year - or a 100 - from AGI. Automation can lead to very dangerous and even unpredectible outcomes not in systems or infrastructure, but in the real world, meat-space. Just think about Stuxnet, what could have happened if the sabotage would have been inmediately identified and precisely attributed? Could the events have developed differently than they did by thinking the incident was some freak accident? Then you let these an unrestrained LLM produce some dangerous output, then some big geopolitical thing got badly broken somewhere really important for powers that be. Some humans think about retribution, then things start to go south really fast. Some self-restriction in LLM is happening elsewhere, the Stable Diffusion guys did it well, just not letting the v2 thing draw "anything possible", which would include some of worst, most creepies ideas of mankind about everything, literally.