4 ms·

I think the most interesting part of AI doomsaying is that almost all of these risks apply to biological 'systems' too. There's no reason why a human can't be b

I think the most interesting part of AI doomsaying is that almost all of these risks apply to biological 'systems' too. There's no reason why a human can't be born that somehow has superhuman intelligence and does all of the things these evil AI would do. Moreover, a biological superintelligence has self-replication built into the system already!

tristor 3y ago
Depending on how you define biological superintelligence, these persons already exist, there's at least two people in the world (on an SD16 scale) with a 200 IQ at our current global population, my IQ is in the "above-genius" territory and there are at least ~335 people as intelligent as I am just in the United States (and that's only statistical probability, due to global labor market conditions the number is probably higher). Do you know what really smart people do? They don't do any of the things that AI Risk says will happen, they get jobs at Google (or insert other pointless white collar job on wall street) moving widgets by 1 pixel to drive an ad engagement metric by half of a percentage and get paid $500k+ a year in total compensation. Why? Because it is a relatively low effort way to fund a lifestyle that allows enjoyment of your intellect through experiencing the world in other dimensions besides the mundane of work. The non-biological aspect is the only reason why AI Risk has any credence at all /in our current state/, because AI isn't distracted by a desire to travel the world and explore nature, or to read all the great works of literature and write their own analysis of it, etc. Biological beings, at least at this point in time and in the probable sense, have emotions and therefore have desires that range beyond whatever machinations you expect from AI Risk.
- Aerroon 3y ago
  You're right that most smart people behave this way, but there's no guarantee that all of them will. The existential risk is still there (psychopathy is a real thing too). But something else to consider is that so far biological super intelligence doesn't seem to get all that much smarter than what humans are at generally. If somebody is guarded against you, then it's very difficult (impossible) to fool them even if you are way smarter than they are. This could indicate that maybe there's a cap to how super intelligent AI can get as well.

tristor 3y ago

Depending on how you define biological superintelligence, these persons already exist, there's at least two people in the world (on an SD16 scale) with a 200 IQ at our current global population, my IQ is in the "above-genius" territory and there are at least ~335 people as intelligent as I am just in the United States (and that's only statistical probability, due to global labor market conditions the number is probably higher). Do you know what really smart people do? They don't do any of the things that AI Risk says will happen, they get jobs at Google (or insert other pointless white collar job on wall street) moving widgets by 1 pixel to drive an ad engagement metric by half of a percentage and get paid $500k+ a year in total compensation. Why? Because it is a relatively low effort way to fund a lifestyle that allows enjoyment of your intellect through experiencing the world in other dimensions besides the mundane of work. The non-biological aspect is the only reason why AI Risk has any credence at all /in our current state/, because AI isn't distracted by a desire to travel the world and explore nature, or to read all the great works of literature and write their own analysis of it, etc. Biological beings, at least at this point in time and in the probable sense, have emotions and therefore have desires that range beyond whatever machinations you expect from AI Risk.

Aerroon 3y ago
You're right that most smart people behave this way, but there's no guarantee that all of them will. The existential risk is still there (psychopathy is a real thing too). But something else to consider is that so far biological super intelligence doesn't seem to get all that much smarter than what humans are at generally. If somebody is guarded against you, then it's very difficult (impossible) to fool them even if you are way smarter than they are. This could indicate that maybe there's a cap to how super intelligent AI can get as well.