4 ms·

AI doesn't work that way.

CPLX 9y ago
Not yet.
dws 9y ago
You might find the history of Doug Lennat's Eurisko informative. "Lenat fed the massive Traveller rulebook into the system and asked it to find the best way of winning. Each night, after several hours of trial and error, Eurisko would spit out a few strategies—some interesting, some impractical, some ridiculous. At one point, Lenat remembers, it suggested he could win the game by changing the rules." https://www.wired.com/2016/03/doug-lenat-artificial-intelligence-common-sense-engine/ https://www.wired.com/2016/03/doug-lenat-artificial-intellig...
- lostboys67 9y ago
  ah this was Traveller book 5 Trillion Credit Squadron Publish in 1980 or so. The authors commented that having a computer (when Pc's where ultra rare and expensive) would be useful :-) I commented at that time that a PC was essential and working at the cutting edge of AI would be "useful" Traveller (book3) was the basis for Elite as well
- cabalamat 9y ago
  Bear in mind that Lenat won't release the Eurisko source code, and some people have speculated that it didn't do all that he said it did.
- gumby 9y ago
  I used to work for Doug and let’s say there is some “dramatization” in that characterization. Also Eurisko and other symbolic/semantic systems are fundamentally different from the NNs popular today.
ynniv 9y ago
AI will eventually be better at doing what the best people do.
- schnevets 9y ago
  Although I agree with this sentiment normally, this doomsday scenario would require a great degree of cunning and deception. An AI can succeed without practicing this kind of manipulation. I'm certain other humans/AIs would be able to identify this and build around it (or maybe even pull the plug) The scenario may work with a human actor/corporation working with the AI, but then it's no longer that cyberpunk (and could probably be done with a large enough grid today).
- clavalle 9y ago
  AI is generally strictly bounded. It only looks at certain data and can only take certain actions based on that data.
kleer001 9y ago
Which is how I feel about most of the supposed Ai paradoxes like "The stamp collector problem" or even "Grey goo". They're like a 10 ten ton block of ice in the middle of a sun. Sure, it's physically plausible, there's nothing in the laws of physics that says it can't happen. It's only that there's no path leading up to it happening.
- adrianN 9y ago
  I think grey goo is fairly realistic. Having small, self-replicating machines has many useful applications. Flaws during reproduction could trigger a runaway replication scenario. The latter already happens with the existing small replicating machines that we know. We usually call that cancer.
  - Retric 9y ago
    Life is already tiny self replicating machines, it still can't replicate inside of a wood wall very well. Further, life and nano-machines are stuck with the same building blocks. Outside of organic chemistry there are not a lot of options for self replication. At which point your nano machines are also just food for existing nano-machines with billions of years of evolution behind them. TLDR; The problem is chemistry not design.
    - adrianN 9y ago
      Wood wasn't digested by anything for millions of years. It seems not completely impossible that whatever we use to create the nanomachines wouldn't be eaten by anything. But of course you're right in stating that it's unlikely that we'll produce machines that are capable of self-replicating using virtually any available substrate. So the runaway scenario where the whole Earth is turned into grey goo is unrealistic.
      - Retric 9y ago
        Small bits of Wood get eaten by any number of organisms and much faster when wet. https://en.wikipedia.org/wiki/Dry_rot https://en.wikipedia.org/wiki/Dry_rot So, it's probably a poor choice for building blocks if you don't want the robots eaten especially if you want the to move around..
fenwick67 9y ago
AI doesn't actually know what it's doing. It just optimizes for hidden patterns. I wrote a whole blog post on this but tl;dr AI/ML can do unethical things without anybody knowing.
have_faith 9y ago
Of course, the trading bot AI assumedly would only have outputs as either buy or sell shares. But what if the AI after a long time of analysing news and correlating it with stock prices determines that the best returns are somehow by buying and selling in a way that crashes the market temporarily. I guess the central point is that even if they aren't advanced enough to enact OP's scenario, they do not understand negative repercussions outside of their specific domain and so can unscrupulously cause havoc by proxy once they get smart enough.
stupidcar 9y ago
AI works exactly that way. Reinforcement learning involves training a model using some input features, some outputs, and a value function (usually, how closely the output(s) match a known value in the training set). Consider a model whose inputs are market prices, outputs are market transactions, and which has a naive value function that simply optimises for the amount of profit is trades result in in a relative short space of time. It's plausible that it could detect trading patterns that correspond to market crashes that can be exploited for profit. These trading patterns might actually be causing or supporting political / economic shocks, such as bubbles, corrupt or unethical behaviour or inter-state conflict that then cause these crashes, but, to the model, this would be invisible or irrelevant. It would learn to perform trades that encourage these same shocks (e.g. undermining a large employer, causing job losses that cause political anger), and profit from the consequences. All without ever "understanding" what is happening outside of its model or having malevolent intent.
cujic9 9y ago
Unfortunately, humans do.

CPLX 9y ago

Not yet.

dws 9y ago

You might find the history of Doug Lennat's Eurisko informative. "Lenat fed the massive Traveller rulebook into the system and asked it to find the best way of winning. Each night, after several hours of trial and error, Eurisko would spit out a few strategies—some interesting, some impractical, some ridiculous. At one point, Lenat remembers, it suggested he could win the game by changing the rules." https://www.wired.com/2016/03/doug-lenat-artificial-intelligence-common-sense-engine/ https://www.wired.com/2016/03/doug-lenat-artificial-intellig...

lostboys67 9y ago
ah this was Traveller book 5 Trillion Credit Squadron Publish in 1980 or so. The authors commented that having a computer (when Pc's where ultra rare and expensive) would be useful :-) I commented at that time that a PC was essential and working at the cutting edge of AI would be "useful" Traveller (book3) was the basis for Elite as well
cabalamat 9y ago
Bear in mind that Lenat won't release the Eurisko source code, and some people have speculated that it didn't do all that he said it did.
gumby 9y ago
I used to work for Doug and let’s say there is some “dramatization” in that characterization. Also Eurisko and other symbolic/semantic systems are fundamentally different from the NNs popular today.

ynniv 9y ago

AI will eventually be better at doing what the best people do.

schnevets 9y ago
Although I agree with this sentiment normally, this doomsday scenario would require a great degree of cunning and deception. An AI can succeed without practicing this kind of manipulation. I'm certain other humans/AIs would be able to identify this and build around it (or maybe even pull the plug) The scenario may work with a human actor/corporation working with the AI, but then it's no longer that cyberpunk (and could probably be done with a large enough grid today).
clavalle 9y ago
AI is generally strictly bounded. It only looks at certain data and can only take certain actions based on that data.

kleer001 9y ago

Which is how I feel about most of the supposed Ai paradoxes like "The stamp collector problem" or even "Grey goo". They're like a 10 ten ton block of ice in the middle of a sun. Sure, it's physically plausible, there's nothing in the laws of physics that says it can't happen. It's only that there's no path leading up to it happening.

adrianN 9y ago
I think grey goo is fairly realistic. Having small, self-replicating machines has many useful applications. Flaws during reproduction could trigger a runaway replication scenario. The latter already happens with the existing small replicating machines that we know. We usually call that cancer.
- Retric 9y ago
  Life is already tiny self replicating machines, it still can't replicate inside of a wood wall very well. Further, life and nano-machines are stuck with the same building blocks. Outside of organic chemistry there are not a lot of options for self replication. At which point your nano machines are also just food for existing nano-machines with billions of years of evolution behind them. TLDR; The problem is chemistry not design.
  - adrianN 9y ago
    Wood wasn't digested by anything for millions of years. It seems not completely impossible that whatever we use to create the nanomachines wouldn't be eaten by anything. But of course you're right in stating that it's unlikely that we'll produce machines that are capable of self-replicating using virtually any available substrate. So the runaway scenario where the whole Earth is turned into grey goo is unrealistic.
    - Retric 9y ago
      Small bits of Wood get eaten by any number of organisms and much faster when wet. https://en.wikipedia.org/wiki/Dry_rot https://en.wikipedia.org/wiki/Dry_rot So, it's probably a poor choice for building blocks if you don't want the robots eaten especially if you want the to move around..

fenwick67 9y ago

AI doesn't actually know what it's doing. It just optimizes for hidden patterns. I wrote a whole blog post on this but tl;dr AI/ML can do unethical things without anybody knowing.

have_faith 9y ago

Of course, the trading bot AI assumedly would only have outputs as either buy or sell shares. But what if the AI after a long time of analysing news and correlating it with stock prices determines that the best returns are somehow by buying and selling in a way that crashes the market temporarily. I guess the central point is that even if they aren't advanced enough to enact OP's scenario, they do not understand negative repercussions outside of their specific domain and so can unscrupulously cause havoc by proxy once they get smart enough.

stupidcar 9y ago

AI works exactly that way. Reinforcement learning involves training a model using some input features, some outputs, and a value function (usually, how closely the output(s) match a known value in the training set). Consider a model whose inputs are market prices, outputs are market transactions, and which has a naive value function that simply optimises for the amount of profit is trades result in in a relative short space of time. It's plausible that it could detect trading patterns that correspond to market crashes that can be exploited for profit. These trading patterns might actually be causing or supporting political / economic shocks, such as bubbles, corrupt or unethical behaviour or inter-state conflict that then cause these crashes, but, to the model, this would be invisible or irrelevant. It would learn to perform trades that encourage these same shocks (e.g. undermining a large employer, causing job losses that cause political anger), and profit from the consequences. All without ever "understanding" what is happening outside of its model or having malevolent intent.

cujic9 9y ago

Unfortunately, humans do.