5 ms·

I think that velocity is the fascinating part. I remember being intrigued while reading "Influx" by Daniel Suarez (great tin foil hat thriller that is hitting

I think that velocity is the fascinating part. I remember being intrigued while reading "Influx" by Daniel Suarez (great tin foil hat thriller that is hitting scarily close to home these days) about the training of the various AIs that were used by the government. I wonder what happens once we grow an AI whose purpose is to excel at training other AI's. Will we see an exponential leap forward in the speed with which they are trained?

catshirt 11y ago
start here https://en.m.wikipedia.org/wiki/Technological_singularity https://en.m.wikipedia.org/wiki/Technological_singularity
pdkl95 11y ago
> exponential leap forward A recent(-ish) Computerphile video has a good overview of the AI-self-improvement problem. https://www.youtube.com/watch?v=5qfIgCiYlfY https://www.youtube.com/watch?v=5qfIgCiYlfY I recommend watching the previous video first, which introduces the problem of strong-AI rapidly finding solutions (especially with poorly-specified questions) that may not be in humanity's best interest. https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4 Before anybody jumps to overly-sensational conclusions, note that the last video in that series, Rob Miles explains how exponential self-improvement is an extreme point in the space of possible AI development. We don't know how to predict discoveries[1], so we need more research into AI, so we can hopefully make something that isn't exponentially growing beyond our understanding. https://www.youtube.com/watch?v=IB1OvoCNnWY https://www.youtube.com/watch?v=IB1OvoCNnWY [1] see James Burke's non-teleological view of change
- Retra 11y ago
  Finding solutions doesn't imply that those solutions are implemented. I find a dozen solutions that are not in humanities interest every day. Does that put anybody at risk? Maybe if you were stupid enough to make me Dictator of the World or something. And you certainly shouldn't do that until I convincingly demonstrate that I am very very committed to finding solutions that are in humanity's best interests. I'm baffled at how easily people assuming a computer thinking something means it's going to happen. There are a trillion pieces that have to fall in place for that to happen accidentally, and if it doesn't happen accidentally, your problem is a human social problem, not an AI problem. And I know what someone is going to think: "The AI might be smart enough to figure out how to make those trillion pieces fall in place." But then, who cares? So what if it figures out how to do something. It still has to be done. And we're the ones who have to do it.
  - AgentME 11y ago
    The AI could come up with ideas that benefit the people that enact them in the short term. If the AI can earn its own money, then it's not that hard for it to use that money to pay people. For an extreme example, you could imagine an AI getting rich from the stock market (or mechanical turk, etc), then buying up ridiculous amounts of land for paperclip factories and paying workers. The people that want to feed their families or get rich from selling their factories are the ones who will enact the AI's plans. How many conscientious objectors do you expect?
    - nefitty 11y ago
      The "rich AI" problem didn't seem feasible to me until I realized how powerful and flexible bitcoin is. Now add the power of contracts through ethereum and a machine could actually harness a significant amount of leverage over human actors. With traditional contracts, enforcement would have left the power in government hands, and with traditional banking as well. We've now stepped into an era where we might literally have built the bat and shovel AI will use to get humanity into its grave. /panic
    - Retra 11y ago
      How many objections would there be a machine collecting a significant amount of money in the stock market and redistributing it among humans? How is that different from what is happening today? We already have rich, selfish people. And they already pay people to get their way. And plenty of people object to it.
      - AgentME 11y ago
        I was alluding to the idea of a paperclip maximizer AI[1], which over time redirects increasing amounts of resources to making useless paperclips. Following the thought experiment further, it continually buys more factories for the purpose of building paperclips or technologies specifically for building paperclips (including improving itself). It probably does some charity in order to be seen as benevolent by people while the people are still in control. Soon many countries are doing nothing but building paperclips and making the minimum necessities to feed their workers. Every other human endeavor is decreasingly profitable as the AI orchestrates the markets to optimize for paperclips. When the AI reaches enough automation and humans are no longer useful or a threat, it drops all benevolent pretenses and replaces all of its human workers, leaving them to fend for themselves while it owns and defends all of the planet's resources. [1] https://wiki.lesswrong.com/wiki/Paperclip_maximizer https://wiki.lesswrong.com/wiki/Paperclip_maximizer

catshirt 11y ago

start here https://en.m.wikipedia.org/wiki/Technological_singularity https://en.m.wikipedia.org/wiki/Technological_singularity

pdkl95 11y ago

> exponential leap forward A recent(-ish) Computerphile video has a good overview of the AI-self-improvement problem. https://www.youtube.com/watch?v=5qfIgCiYlfY https://www.youtube.com/watch?v=5qfIgCiYlfY I recommend watching the previous video first, which introduces the problem of strong-AI rapidly finding solutions (especially with poorly-specified questions) that may not be in humanity's best interest. https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4 Before anybody jumps to overly-sensational conclusions, note that the last video in that series, Rob Miles explains how exponential self-improvement is an extreme point in the space of possible AI development. We don't know how to predict discoveries[1], so we need more research into AI, so we can hopefully make something that isn't exponentially growing beyond our understanding. https://www.youtube.com/watch?v=IB1OvoCNnWY https://www.youtube.com/watch?v=IB1OvoCNnWY [1] see James Burke's non-teleological view of change

Retra 11y ago
Finding solutions doesn't imply that those solutions are implemented. I find a dozen solutions that are not in humanities interest every day. Does that put anybody at risk? Maybe if you were stupid enough to make me Dictator of the World or something. And you certainly shouldn't do that until I convincingly demonstrate that I am very very committed to finding solutions that are in humanity's best interests. I'm baffled at how easily people assuming a computer thinking something means it's going to happen. There are a trillion pieces that have to fall in place for that to happen accidentally, and if it doesn't happen accidentally, your problem is a human social problem, not an AI problem. And I know what someone is going to think: "The AI might be smart enough to figure out how to make those trillion pieces fall in place." But then, who cares? So what if it figures out how to do something. It still has to be done. And we're the ones who have to do it.
- AgentME 11y ago
  The AI could come up with ideas that benefit the people that enact them in the short term. If the AI can earn its own money, then it's not that hard for it to use that money to pay people. For an extreme example, you could imagine an AI getting rich from the stock market (or mechanical turk, etc), then buying up ridiculous amounts of land for paperclip factories and paying workers. The people that want to feed their families or get rich from selling their factories are the ones who will enact the AI's plans. How many conscientious objectors do you expect?
  - nefitty 11y ago
    The "rich AI" problem didn't seem feasible to me until I realized how powerful and flexible bitcoin is. Now add the power of contracts through ethereum and a machine could actually harness a significant amount of leverage over human actors. With traditional contracts, enforcement would have left the power in government hands, and with traditional banking as well. We've now stepped into an era where we might literally have built the bat and shovel AI will use to get humanity into its grave. /panic
  - Retra 11y ago
    How many objections would there be a machine collecting a significant amount of money in the stock market and redistributing it among humans? How is that different from what is happening today? We already have rich, selfish people. And they already pay people to get their way. And plenty of people object to it.
    - AgentME 11y ago
      I was alluding to the idea of a paperclip maximizer AI[1], which over time redirects increasing amounts of resources to making useless paperclips. Following the thought experiment further, it continually buys more factories for the purpose of building paperclips or technologies specifically for building paperclips (including improving itself). It probably does some charity in order to be seen as benevolent by people while the people are still in control. Soon many countries are doing nothing but building paperclips and making the minimum necessities to feed their workers. Every other human endeavor is decreasingly profitable as the AI orchestrates the markets to optimize for paperclips. When the AI reaches enough automation and humans are no longer useful or a threat, it drops all benevolent pretenses and replaces all of its human workers, leaving them to fend for themselves while it owns and defends all of the planet's resources. [1] https://wiki.lesswrong.com/wiki/Paperclip_maximizer https://wiki.lesswrong.com/wiki/Paperclip_maximizer