4 ms·

Excellent post, I rather agree. > As long as AI is an open technology there will always be some criminals who just want to see the world burn. Thankfully we h

by arcanus 10y ago

Excellent post, I rather agree. > As long as AI is an open technology there will always be some criminals who just want to see the world burn. Thankfully we have yet to see terror attacks with nuclear weapons, but the lower barriers of entry for potentially catastrophic AIs undoubtedly is alarming. I often wonder if analog solutions will not be critical. To continue with your example, a nuke that must be armed with a physical lever would be a rather great hindrance to any AI. Internet communications, etc. will be more tricky, due to the high frequency activity, but the point of having meatspace 'firewalls' on mission critical activities is a nice short term kludge.

ccvannorman 10y ago
one of the most effective and scariest attack vectors for AI would be convincing or coercing humans to do its bidding (e.g. "pull the lever or i'll switch off your father's life support")
- deftnerd 10y ago
  It might be easier than that. Data Broker companies are the ones that run those stupid "Which LOTR character are you" or "What color are you" quizzes that are popular in some social media circles. They do it to slowly build psychological trait models of the people taking the quiz. This allows them to sell that information to marketing companies. Access to that kind of data would help an AI determine the people that are more susceptible to manipulation. Add in records on health care, as you mentioned, and information on debts and you have data that can help an AI gather as many human minions as it needs.
  - pixl97 10y ago
    Greedy people cause all kinds of problems now. I don't know why people aren't concerned that a greedy AI could be even worse.
- kbaker 10y ago
  Great recent story following this plot line: http://compellingsciencefiction.com/stories/crinkles.html http://compellingsciencefiction.com/stories/crinkles.html
daveguy 10y ago
Low barrier to entry for catastrophic AI? Surely you aren't talking about the current state of AI. The current barrier of entry for an AGI (which is at least what it would take for catastrophe) is "does not exist". That's about as big as barriers get. Even when the first AGIs show up it will take full time supercomputers to perform like a single human brain. I don't see that as a particularly low barrier.
- pixl97 10y ago
  Eh. The barrier for entry for an AGI does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 7 billion general intelligences running around this planet (and many more if you consider animal intelligences) It is important to define it that way, not that it is impossible, just that it is unknown how much effort is needed to create an artificial one. This distinction is very important when comparing the threat of AI with other significant threats. Before nuclear bombs were built we could not tell you what the difficulty was in creating one. Now that difficulty is well defined, and we can use that knowledge to prevent them from being built by most nations, except the most well funded. If the barrier for entry for AGI (then ASI) is lower than we expect, then the threat of AI is significantly different than if AGI/ASI can only be created by nation states.
  - argonaut 10y ago
    The barrier for entry for an alien invasion does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 1 bloodthirsty species running around this galaxy (and many more if you consider the statistical possibility of life on other planets) It is important to define it that way, not that it is impossible, just that it is unknown how much time is needed before an alien invasion. The reason I am framing things this way is we need to be very careful here because we are starting to turn towards speculation.
    - FeepingCreature 10y ago
      You know, you mean for that to sound implausible, but the Great Filter is in fact an open research problem.
ikeboy 10y ago
http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n...

by arcanus 10y ago

ccvannorman 10y ago

one of the most effective and scariest attack vectors for AI would be convincing or coercing humans to do its bidding (e.g. "pull the lever or i'll switch off your father's life support")

deftnerd 10y ago
It might be easier than that. Data Broker companies are the ones that run those stupid "Which LOTR character are you" or "What color are you" quizzes that are popular in some social media circles. They do it to slowly build psychological trait models of the people taking the quiz. This allows them to sell that information to marketing companies. Access to that kind of data would help an AI determine the people that are more susceptible to manipulation. Add in records on health care, as you mentioned, and information on debts and you have data that can help an AI gather as many human minions as it needs.
- pixl97 10y ago
  Greedy people cause all kinds of problems now. I don't know why people aren't concerned that a greedy AI could be even worse.
kbaker 10y ago
Great recent story following this plot line: http://compellingsciencefiction.com/stories/crinkles.html http://compellingsciencefiction.com/stories/crinkles.html

daveguy 10y ago

Low barrier to entry for catastrophic AI? Surely you aren't talking about the current state of AI. The current barrier of entry for an AGI (which is at least what it would take for catastrophe) is "does not exist". That's about as big as barriers get. Even when the first AGIs show up it will take full time supercomputers to perform like a single human brain. I don't see that as a particularly low barrier.

pixl97 10y ago
Eh. The barrier for entry for an AGI does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 7 billion general intelligences running around this planet (and many more if you consider animal intelligences) It is important to define it that way, not that it is impossible, just that it is unknown how much effort is needed to create an artificial one. This distinction is very important when comparing the threat of AI with other significant threats. Before nuclear bombs were built we could not tell you what the difficulty was in creating one. Now that difficulty is well defined, and we can use that knowledge to prevent them from being built by most nations, except the most well funded. If the barrier for entry for AGI (then ASI) is lower than we expect, then the threat of AI is significantly different than if AGI/ASI can only be created by nation states.
- argonaut 10y ago
  The barrier for entry for an alien invasion does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 1 bloodthirsty species running around this galaxy (and many more if you consider the statistical possibility of life on other planets) It is important to define it that way, not that it is impossible, just that it is unknown how much time is needed before an alien invasion. The reason I am framing things this way is we need to be very careful here because we are starting to turn towards speculation.
  - FeepingCreature 10y ago
    You know, you mean for that to sound implausible, but the Great Filter is in fact an open research problem.

ikeboy 10y ago

http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n...