6 ms·

The cognitive capacity of ants prevents trade proposed. And the cognitive capacity of humans relative to AGI would also prevent trade in the same fashion.

aidenn0 3y ago
I don't see that the cognitive capacity of humans relative to AGI would prevent such trade anymore than the cognitive capacity of e.g. dogs prevent trade between humans and dogs. It's not a relationship with equal power, but there is some level of reciprocity there.
- TheOtherHobbes 3y ago
  Dogs and humans are relatively close. Humans and AGI might not be so close. It depends where the S curve for AGI plateaus. It might be somewhere relatively comprehensible, or it might be far beyond that. If it's the latter no trade is possible. AGI is going to be doing things we literally can't begin to imagine, driven by motivations we literally can't begin to imagine, using super-meta-everything conceptual engineering we literally can't begin to imagine. There may be some perceptual fall-out for us to experience, but we won't have any idea what it means. If this isn't obvious, consider that humans have symbolic abstraction skills which other animals (mostly) lack. What's the next type of skill up from that? You could say something like much broader and deeper pattern recognition, the ability to handle far more percepts at once, and so on. But those are still qualitatively familiar. They're just human skills improved. What would an entirely different super-class of skills and abilities look like? It's impossible for us to answer that.
  - aidenn0 3y ago
    I stated elsewhere (after your reply) that a human-created AGI is likely to have at least vestigial ability to communicate with humans, since humans seem to want very much to communicate with AIs.
anonymouskimmer 3y ago
"Trade" can exist outside of the literal meaning. Humans set up many situation where animals just do their thing, but in a manner that benefits the humans (and ideally the animal for this to be considered a trade, such as living in a protected environment). Using Giant African Rats as landmine detectors is one such mutually-beneficial trade.
jamilton 3y ago
I don't think it's as simple as "cognitive capacity". I think the author's angle of communication being the critical issue is more accurate. It could be the case that we won't be able to communicate with AGI, but I don't see reason to assume that is the most likely outcome.
eru 3y ago
Why? To do trade you need to clear an absolute minimum threshold of intelligence. It's not a relative threshold. For example, people can 'trade' with smart contracts on some blockchain. Despite the name, those are really 'dumb' computer programs.

aidenn0 3y ago

I don't see that the cognitive capacity of humans relative to AGI would prevent such trade anymore than the cognitive capacity of e.g. dogs prevent trade between humans and dogs. It's not a relationship with equal power, but there is some level of reciprocity there.

TheOtherHobbes 3y ago
Dogs and humans are relatively close. Humans and AGI might not be so close. It depends where the S curve for AGI plateaus. It might be somewhere relatively comprehensible, or it might be far beyond that. If it's the latter no trade is possible. AGI is going to be doing things we literally can't begin to imagine, driven by motivations we literally can't begin to imagine, using super-meta-everything conceptual engineering we literally can't begin to imagine. There may be some perceptual fall-out for us to experience, but we won't have any idea what it means. If this isn't obvious, consider that humans have symbolic abstraction skills which other animals (mostly) lack. What's the next type of skill up from that? You could say something like much broader and deeper pattern recognition, the ability to handle far more percepts at once, and so on. But those are still qualitatively familiar. They're just human skills improved. What would an entirely different super-class of skills and abilities look like? It's impossible for us to answer that.
- aidenn0 3y ago
  I stated elsewhere (after your reply) that a human-created AGI is likely to have at least vestigial ability to communicate with humans, since humans seem to want very much to communicate with AIs.

anonymouskimmer 3y ago

"Trade" can exist outside of the literal meaning. Humans set up many situation where animals just do their thing, but in a manner that benefits the humans (and ideally the animal for this to be considered a trade, such as living in a protected environment). Using Giant African Rats as landmine detectors is one such mutually-beneficial trade.

jamilton 3y ago

I don't think it's as simple as "cognitive capacity". I think the author's angle of communication being the critical issue is more accurate. It could be the case that we won't be able to communicate with AGI, but I don't see reason to assume that is the most likely outcome.

eru 3y ago

Why? To do trade you need to clear an absolute minimum threshold of intelligence. It's not a relative threshold. For example, people can 'trade' with smart contracts on some blockchain. Despite the name, those are really 'dumb' computer programs.