13 ms·
I will know we should start fearing AI when AI is writing pieces on Medium that says we shouldn't fear it :-) But more importantly, the good/bad conversations
by ChuckMcM 10y ago
I will know we should start fearing AI when AI is writing pieces on Medium that says we shouldn't fear it :-)
But more importantly, the good/bad conversations are really about sentient[1] AI and not programmatic AI. Essentially the argument boils down to, "If a sentient AI is created that exists in, and can control, inter-connected computer systems, that AI will be able to take action to satisfy any particular 'want' it might have."
So it gets "scary" if you create an AI that is "wanting" something. And I could imagine this done innocuously, for example creating a trading system like deepmind what "wanted" to "win the securities game" by maximizing the value of its holdings. And the risk being that there are very destructive ways to achieve that (killing off your competition for example). But in those scenarios, when it is clear the program is operating outside of its control parameters, the question would be how would it have already figured out that it could be turned off/deleted and so protected itself against that? And that is why sentience is required, by recognizing itself apart from its programming, it recognizes threats to its existence.
I tend to lean more toward the "unlikely to be an issue in the near future." camp.
[1] Sentience being the effect that the program actually understands it's own existence apart from the environment.