5 ms·

Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than t

by yojo 5y ago

Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than the one we enjoy. I’m not saying you’re wrong, only that I can imagine a reality where you can’t just arbitrarily scale components of cognition and still have a functioning system.

gonehome 5y ago
It's more likely (imo) that humans are constrained by biological limits (energy, head size) than some true maximum. A simple thought experiment is the same architecture of the human brain (whatever that is), but just with way more power to run at more operations per second.
- breuleux 5y ago
  Would that actually work, though, or would it just cook itself? I think the general question to ask is, for a given organism or machine, at what point does it stop being cost effective to make it smarter? Nature doesn't just keep human intelligence at the level it currently is, it's doing so for all other organisms. There seems to be tradeoffs. Which ones would apply to AI? And which ones are not currently relevant but will apply to future AI? For example, AI has no physical predators currently, but you could imagine a future where they do, or where nanobots or special bacteria attack silicon. In such a future, it is possible that very intelligent AI would suddenly become uncompetitive, relative to dumber and leaner ones. Viable superintelligent AI might then turn out to be just as difficult or circumstantial to "evolve" as humans were.
  - eru 5y ago
    > Would that actually work, though, or would it just cook itself? You could add better cooling than what the human hardware provides. Even just taking a normal human head and dunking it in cold water dissipates a lot more head. (If you try this at home, I suggest getting a snorkel.)
  - gonehome 5y ago
    Why would it cook itself if it was replicated in silicon? I think it's a lot less likely humanity just happens to be at some natural limit of intelligence rather than being bounded by other unrelated constraints. AGI tradeoffs would be different because it'd be unconstrained from biological natural selection. If you're interested in this stuff it's worth checking out the work that specifies the AGI goal alignment problem more specifically. The most commonly recommended book is Bostrom's Superintelligence, but it's fairly dry. Tegmark's Life 3.0 is on the other extreme of too pop-sci like. I think Yudkowsky's writing is the best for explaining the issue (specifically AGI goal alignment and what the specific problem is) in a way that's accessible, but not dumbed down.
    - coderenegade 5y ago
      The human experience is fundamentally chemical in nature (which is why alcohol has an impact on cognition, for example). What would it even mean to replicate that in silicon? I know food is bad because I can hold it, smell it, and taste it, none of which a computer can do. If you tried to upload someone's consciousness to a machine, you would need to emulate the full sensory experience as well, because there would be zero context otherwise. What's missing from discussions on AGI is the lack of a measure that relates human intelligence with machine intelligence, and the recognition that our own intelligence is fundamentally tied in to our physical bodies. To what extent does it make sense to talk about intelligence without also talking about its embodiment?
      - gonehome 5y ago
        Inputs can be replicated in other forms (cameras for example for visual input and computer vision). There’s a common misconception that AGI would be human like, consciousness is a mostly poorly defined orthogonal rabbit hole to the AGI question (and the goal alignment issue). That said, you’re right the ability to train is dependent on inputs. If you’re curious about this stuff it’s really worth reading the stuff I mentioned to get a sense for what the problem is. I only suggest them because I would have made a similar comment to you a few years ago before getting a better sense for what the problems are actually describing.

by yojo 5y ago

gonehome 5y ago

It's more likely (imo) that humans are constrained by biological limits (energy, head size) than some true maximum. A simple thought experiment is the same architecture of the human brain (whatever that is), but just with way more power to run at more operations per second.

breuleux 5y ago
Would that actually work, though, or would it just cook itself? I think the general question to ask is, for a given organism or machine, at what point does it stop being cost effective to make it smarter? Nature doesn't just keep human intelligence at the level it currently is, it's doing so for all other organisms. There seems to be tradeoffs. Which ones would apply to AI? And which ones are not currently relevant but will apply to future AI? For example, AI has no physical predators currently, but you could imagine a future where they do, or where nanobots or special bacteria attack silicon. In such a future, it is possible that very intelligent AI would suddenly become uncompetitive, relative to dumber and leaner ones. Viable superintelligent AI might then turn out to be just as difficult or circumstantial to "evolve" as humans were.
- eru 5y ago
  > Would that actually work, though, or would it just cook itself? You could add better cooling than what the human hardware provides. Even just taking a normal human head and dunking it in cold water dissipates a lot more head. (If you try this at home, I suggest getting a snorkel.)
- gonehome 5y ago
  Why would it cook itself if it was replicated in silicon? I think it's a lot less likely humanity just happens to be at some natural limit of intelligence rather than being bounded by other unrelated constraints. AGI tradeoffs would be different because it'd be unconstrained from biological natural selection. If you're interested in this stuff it's worth checking out the work that specifies the AGI goal alignment problem more specifically. The most commonly recommended book is Bostrom's Superintelligence, but it's fairly dry. Tegmark's Life 3.0 is on the other extreme of too pop-sci like. I think Yudkowsky's writing is the best for explaining the issue (specifically AGI goal alignment and what the specific problem is) in a way that's accessible, but not dumbed down.
  - coderenegade 5y ago
    The human experience is fundamentally chemical in nature (which is why alcohol has an impact on cognition, for example). What would it even mean to replicate that in silicon? I know food is bad because I can hold it, smell it, and taste it, none of which a computer can do. If you tried to upload someone's consciousness to a machine, you would need to emulate the full sensory experience as well, because there would be zero context otherwise. What's missing from discussions on AGI is the lack of a measure that relates human intelligence with machine intelligence, and the recognition that our own intelligence is fundamentally tied in to our physical bodies. To what extent does it make sense to talk about intelligence without also talking about its embodiment?
    - gonehome 5y ago
      Inputs can be replicated in other forms (cameras for example for visual input and computer vision). There’s a common misconception that AGI would be human like, consciousness is a mostly poorly defined orthogonal rabbit hole to the AGI question (and the goal alignment issue). That said, you’re right the ability to train is dependent on inputs. If you’re curious about this stuff it’s really worth reading the stuff I mentioned to get a sense for what the problem is. I only suggest them because I would have made a similar comment to you a few years ago before getting a better sense for what the problems are actually describing.