4 ms·

> Cognitive engineering, via direct edits to embryonic human DNA, will eventually produce individuals who are well beyond all historical figures in cognitive ab

> Cognitive engineering, via direct edits to embryonic human DNA, will eventually produce individuals who are well beyond all historical figures in cognitive ability. Now that would be truly foolhardy. Blindly augmenting intelligence before cognitive moral reasoning is completely understood would be a recipe for incredibly dangerous psychopaths. At least with AI, you don't have a system that has been fine-tuned by eons of evolution to manipulate people at least partly against their own interests.

SpaceManNabs 11y ago
Are you assuming ethics and intelligence disparate? Intelligent humans aren't necessarily void of empathy, morality, and emotion.
- Tenobrus 11y ago
  They seem to be orthogonal at least. Smart people are not necessarily ethical and vice versa. I'm unsure as to whether the parent thinks that any increase in intelligence would correspond with a decrease in morality, which I disagree with. But there is a pretty clear issue, being that if we manage to augment intelligence without first understanding ethics there's a not-insignifianct chance of creating a sociopath who's smarter than any previous human. That wouldn't exactly be world ending, but it could certainly be seriously damaging.
  - Kalium 11y ago
    I'm willing to be generous to parent to the point of allowing that a specimen of sufficiently greater intelligence may have difficulty empathizing with your average human and regard them as lesser. Much in the same way that we tend to regard apes or mice as lesser than ourselves. Some would argue that we already have that problem emerging in Silicon Valley. I don't agree, since I think that's more of a case of many people refusing to value a society that ill-treated me.
  - saiya-jin 11y ago
    something like Khan in Star trek?
eli_gottlieb 11y ago
>At least with AI, you don't have a system that has been fine-tuned by eons of evolution to manipulate people at least partly against their own interests. As a smart person, I actually find this remarkably insulting. Not only do I have little desire to manipulate others, I have little ability. I'm much better with computers than with manipulating people.
- deleted 11y ago
  [deleted]
- woodchuck64 11y ago
  That you have little desire to manipulate people isn't really here or there; the fact is that a social brain has extensive hardwiring that is designed to manipulate other social beings in blatant and subtle ways, and that hardwiring has been shaped and fine-tuned by eons of evolution. The desire restricting the extent of manipulation against another's interest is going to be constrained by moral beliefs and instincts, empathy, which seems to be of much less concern in the OP than maximizing raw intelligence.
  - eli_gottlieb 11y ago
    But theory-of-mind is a distinct cognitive module from intellectual intelligence. You can enhance one without enhancing the other.
    - woodchuck64 11y ago
      Let me make sure I understand you since I didn't understand why you were "remarkably insulted". The system fine-tuned by evolution is the human mind. Sociopathy and psychopathy seem to be genetically based and may occur in anywhere from 1% to 10% of the population based on various ballpark estimates. IQ is in no way related to moral behavior, smart people can be sincerely kind people or sociopathic, people of low-IQ can be sincerely kind or sociopathic. A person of low-IQ who is a sociopath ends up in jail most likely, a person of high-IQ who is a sociopath damages society far more by cleverly staying out of jail and may often land themselves high positions as they manipulate people's social intuitions without remorse. I suspect (but can't prove) that sociopathy has no correlation with IQ whatsoever. > But theory-of-mind is a distinct cognitive module from intellectual intelligence. You can enhance one without enhancing the other. I agree. However, I think theory-of-mind is one part of the puzzle. Another is moral reasoning and how it shapes social interaction. This, too, needs to be carefully understood because we know that authority and purity concerns, for example, both allow people to effectively turn off empathy. Turning off theory-of-mind when it is inconvenient is a potentially frightening ability.

SpaceManNabs 11y ago

Are you assuming ethics and intelligence disparate? Intelligent humans aren't necessarily void of empathy, morality, and emotion.

Tenobrus 11y ago
They seem to be orthogonal at least. Smart people are not necessarily ethical and vice versa. I'm unsure as to whether the parent thinks that any increase in intelligence would correspond with a decrease in morality, which I disagree with. But there is a pretty clear issue, being that if we manage to augment intelligence without first understanding ethics there's a not-insignifianct chance of creating a sociopath who's smarter than any previous human. That wouldn't exactly be world ending, but it could certainly be seriously damaging.
- Kalium 11y ago
  I'm willing to be generous to parent to the point of allowing that a specimen of sufficiently greater intelligence may have difficulty empathizing with your average human and regard them as lesser. Much in the same way that we tend to regard apes or mice as lesser than ourselves. Some would argue that we already have that problem emerging in Silicon Valley. I don't agree, since I think that's more of a case of many people refusing to value a society that ill-treated me.
- saiya-jin 11y ago
  something like Khan in Star trek?

eli_gottlieb 11y ago

>At least with AI, you don't have a system that has been fine-tuned by eons of evolution to manipulate people at least partly against their own interests. As a smart person, I actually find this remarkably insulting. Not only do I have little desire to manipulate others, I have little ability. I'm much better with computers than with manipulating people.

deleted 11y ago
[deleted]
woodchuck64 11y ago
That you have little desire to manipulate people isn't really here or there; the fact is that a social brain has extensive hardwiring that is designed to manipulate other social beings in blatant and subtle ways, and that hardwiring has been shaped and fine-tuned by eons of evolution. The desire restricting the extent of manipulation against another's interest is going to be constrained by moral beliefs and instincts, empathy, which seems to be of much less concern in the OP than maximizing raw intelligence.
- eli_gottlieb 11y ago
  But theory-of-mind is a distinct cognitive module from intellectual intelligence. You can enhance one without enhancing the other.
  - woodchuck64 11y ago
    Let me make sure I understand you since I didn't understand why you were "remarkably insulted". The system fine-tuned by evolution is the human mind. Sociopathy and psychopathy seem to be genetically based and may occur in anywhere from 1% to 10% of the population based on various ballpark estimates. IQ is in no way related to moral behavior, smart people can be sincerely kind people or sociopathic, people of low-IQ can be sincerely kind or sociopathic. A person of low-IQ who is a sociopath ends up in jail most likely, a person of high-IQ who is a sociopath damages society far more by cleverly staying out of jail and may often land themselves high positions as they manipulate people's social intuitions without remorse. I suspect (but can't prove) that sociopathy has no correlation with IQ whatsoever. > But theory-of-mind is a distinct cognitive module from intellectual intelligence. You can enhance one without enhancing the other. I agree. However, I think theory-of-mind is one part of the puzzle. Another is moral reasoning and how it shapes social interaction. This, too, needs to be carefully understood because we know that authority and purity concerns, for example, both allow people to effectively turn off empathy. Turning off theory-of-mind when it is inconvenient is a potentially frightening ability.