4 ms·

It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) as well as having faster access

It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) as well as having faster access to more knowledge in general (because you can make a bigger brain with more memory) it's just pretty obvious that AGI will beat humans. If all they had was human brains but could transfer thoughts faster they'd still beat us.

yojo 5y ago
Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than the one we enjoy. I’m not saying you’re wrong, only that I can imagine a reality where you can’t just arbitrarily scale components of cognition and still have a functioning system.
- gonehome 5y ago
  It's more likely (imo) that humans are constrained by biological limits (energy, head size) than some true maximum. A simple thought experiment is the same architecture of the human brain (whatever that is), but just with way more power to run at more operations per second.
  - breuleux 5y ago
    Would that actually work, though, or would it just cook itself? I think the general question to ask is, for a given organism or machine, at what point does it stop being cost effective to make it smarter? Nature doesn't just keep human intelligence at the level it currently is, it's doing so for all other organisms. There seems to be tradeoffs. Which ones would apply to AI? And which ones are not currently relevant but will apply to future AI? For example, AI has no physical predators currently, but you could imagine a future where they do, or where nanobots or special bacteria attack silicon. In such a future, it is possible that very intelligent AI would suddenly become uncompetitive, relative to dumber and leaner ones. Viable superintelligent AI might then turn out to be just as difficult or circumstantial to "evolve" as humans were.
    - eru 5y ago
      > Would that actually work, though, or would it just cook itself? You could add better cooling than what the human hardware provides. Even just taking a normal human head and dunking it in cold water dissipates a lot more head. (If you try this at home, I suggest getting a snorkel.)
    - gonehome 5y ago
      Why would it cook itself if it was replicated in silicon? I think it's a lot less likely humanity just happens to be at some natural limit of intelligence rather than being bounded by other unrelated constraints. AGI tradeoffs would be different because it'd be unconstrained from biological natural selection. If you're interested in this stuff it's worth checking out the work that specifies the AGI goal alignment problem more specifically. The most commonly recommended book is Bostrom's Superintelligence, but it's fairly dry. Tegmark's Life 3.0 is on the other extreme of too pop-sci like. I think Yudkowsky's writing is the best for explaining the issue (specifically AGI goal alignment and what the specific problem is) in a way that's accessible, but not dumbed down.
      - coderenegade 5y ago
        The human experience is fundamentally chemical in nature (which is why alcohol has an impact on cognition, for example). What would it even mean to replicate that in silicon? I know food is bad because I can hold it, smell it, and taste it, none of which a computer can do. If you tried to upload someone's consciousness to a machine, you would need to emulate the full sensory experience as well, because there would be zero context otherwise. What's missing from discussions on AGI is the lack of a measure that relates human intelligence with machine intelligence, and the recognition that our own intelligence is fundamentally tied in to our physical bodies. To what extent does it make sense to talk about intelligence without also talking about its embodiment?
        gonehome 5y ago
        Inputs can be replicated in other forms (cameras for example for visual input and computer vision). There’s a common misconception that AGI would be human like, consciousness is a mostly poorly defined orthogonal rabbit hole to the AGI question (and the goal alignment issue). That said, you’re right the ability to train is dependent on inputs. If you’re curious about this stuff it’s really worth reading the stuff I mentioned to get a sense for what the problem is. I only suggest them because I would have made a similar comment to you a few years ago before getting a better sense for what the problems are actually describing.
shkkmo 5y ago
I suspect that intra network latency has a significant impact on intelligent systems. Interconnectivity also seems crucial but as you add nodes the number of connections needed to keep that closeness in the networks scales dramatically. These connections also need space so if you keep the same level of interconnection as you add more nodes, your density drops and so your latency goes up. There is thus a fundamental tradeoff between network size, network latency, and network interconnectivity that can't be avoided. While we may eventually beat the human brain on some or all of these measures, there is good reason to believe there are fundamental limits on the scalability of intelligence.
- yojo 5y ago
  Thank you for this. I was hoping someone would contribute a better explanation than my own vague intuition.
- FeepingCreature 5y ago
  There is simply no reason to believe that the human brain is anywhere near optimal on this tradeoff.
breuleux 5y ago
> It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) I don't think it's necessarily inevitable. It is not a given that future hardware architectures for AI would be inspectable or copiable, because the extra wiring required to do that is space and energy overhead. It may also be the case that greater intelligence comes through better distributed representations and that it simply isn't possible to cheaply translate knowledge from an inferior representation to a superior one (you may need to relearn from scratch). The ability to transfer thoughts may therefore require a sort of lowest-common-denominator representation, in other words, a language. I imagine that language could be more efficient, though.
- eru 5y ago
  You are right that it's not inevitable, but still pretty likely. Also keep in mind: Human nerves work (roughly) at the speed of sound. Computers work (roughly) at the speed of light. We already make computers that are much, much faster than humans at sequential processing. About the representation thing: if you start with AIs that are copies from each other, you can probably keep the 'superior' internal representation the same or nearly the same. So I would expect AI clones to be able to exchange knowledge much quicker than unrelated AIs.

yojo 5y ago

Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than the one we enjoy. I’m not saying you’re wrong, only that I can imagine a reality where you can’t just arbitrarily scale components of cognition and still have a functioning system.

gonehome 5y ago
It's more likely (imo) that humans are constrained by biological limits (energy, head size) than some true maximum. A simple thought experiment is the same architecture of the human brain (whatever that is), but just with way more power to run at more operations per second.
- breuleux 5y ago
  Would that actually work, though, or would it just cook itself? I think the general question to ask is, for a given organism or machine, at what point does it stop being cost effective to make it smarter? Nature doesn't just keep human intelligence at the level it currently is, it's doing so for all other organisms. There seems to be tradeoffs. Which ones would apply to AI? And which ones are not currently relevant but will apply to future AI? For example, AI has no physical predators currently, but you could imagine a future where they do, or where nanobots or special bacteria attack silicon. In such a future, it is possible that very intelligent AI would suddenly become uncompetitive, relative to dumber and leaner ones. Viable superintelligent AI might then turn out to be just as difficult or circumstantial to "evolve" as humans were.
  - eru 5y ago
    > Would that actually work, though, or would it just cook itself? You could add better cooling than what the human hardware provides. Even just taking a normal human head and dunking it in cold water dissipates a lot more head. (If you try this at home, I suggest getting a snorkel.)
  - gonehome 5y ago
    Why would it cook itself if it was replicated in silicon? I think it's a lot less likely humanity just happens to be at some natural limit of intelligence rather than being bounded by other unrelated constraints. AGI tradeoffs would be different because it'd be unconstrained from biological natural selection. If you're interested in this stuff it's worth checking out the work that specifies the AGI goal alignment problem more specifically. The most commonly recommended book is Bostrom's Superintelligence, but it's fairly dry. Tegmark's Life 3.0 is on the other extreme of too pop-sci like. I think Yudkowsky's writing is the best for explaining the issue (specifically AGI goal alignment and what the specific problem is) in a way that's accessible, but not dumbed down.
    - coderenegade 5y ago
      The human experience is fundamentally chemical in nature (which is why alcohol has an impact on cognition, for example). What would it even mean to replicate that in silicon? I know food is bad because I can hold it, smell it, and taste it, none of which a computer can do. If you tried to upload someone's consciousness to a machine, you would need to emulate the full sensory experience as well, because there would be zero context otherwise. What's missing from discussions on AGI is the lack of a measure that relates human intelligence with machine intelligence, and the recognition that our own intelligence is fundamentally tied in to our physical bodies. To what extent does it make sense to talk about intelligence without also talking about its embodiment?
      - gonehome 5y ago
        Inputs can be replicated in other forms (cameras for example for visual input and computer vision). There’s a common misconception that AGI would be human like, consciousness is a mostly poorly defined orthogonal rabbit hole to the AGI question (and the goal alignment issue). That said, you’re right the ability to train is dependent on inputs. If you’re curious about this stuff it’s really worth reading the stuff I mentioned to get a sense for what the problem is. I only suggest them because I would have made a similar comment to you a few years ago before getting a better sense for what the problems are actually describing.

shkkmo 5y ago

I suspect that intra network latency has a significant impact on intelligent systems. Interconnectivity also seems crucial but as you add nodes the number of connections needed to keep that closeness in the networks scales dramatically. These connections also need space so if you keep the same level of interconnection as you add more nodes, your density drops and so your latency goes up. There is thus a fundamental tradeoff between network size, network latency, and network interconnectivity that can't be avoided. While we may eventually beat the human brain on some or all of these measures, there is good reason to believe there are fundamental limits on the scalability of intelligence.

yojo 5y ago
Thank you for this. I was hoping someone would contribute a better explanation than my own vague intuition.
FeepingCreature 5y ago
There is simply no reason to believe that the human brain is anywhere near optimal on this tradeoff.

breuleux 5y ago

> It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) I don't think it's necessarily inevitable. It is not a given that future hardware architectures for AI would be inspectable or copiable, because the extra wiring required to do that is space and energy overhead. It may also be the case that greater intelligence comes through better distributed representations and that it simply isn't possible to cheaply translate knowledge from an inferior representation to a superior one (you may need to relearn from scratch). The ability to transfer thoughts may therefore require a sort of lowest-common-denominator representation, in other words, a language. I imagine that language could be more efficient, though.

eru 5y ago
You are right that it's not inevitable, but still pretty likely. Also keep in mind: Human nerves work (roughly) at the speed of sound. Computers work (roughly) at the speed of light. We already make computers that are much, much faster than humans at sequential processing. About the representation thing: if you start with AIs that are copies from each other, you can probably keep the 'superior' internal representation the same or nearly the same. So I would expect AI clones to be able to exchange knowledge much quicker than unrelated AIs.