3 ms·

To me, the real proof that AI cab be an existential threat would be that it can predict the weather to a very good degree. If an AI can handle the complexity a

• mdp2021 5y ago
  Following the article, AI can become an existential threat if empowered humans which cannot "«predict the weather to a very good degree»" in turn empower AI, which may decide unchecked something along the lines of "to be accurate about predicting the weather a nuclear holocaust could be a good move" (very stretched image for rhetoric purpose). More literally: what happens if, for example, weapons systems are automated by black boxes? So-called "expert systems", for example, remaining within automation, can be instructed more clearly, explicitly, deterministically, about values and boundaries. That the results of "black box decision systems" can be surprising is also a risk.
  • kkjjkgjjgg 5y ago
    Not a "paperclip optimizer", but a "weather prediction optimizer" - nice!
    • mdp2021 5y ago
      My expression fell under the vibes of the more generic "paperclip maximizer argument" because I wanted to remain faithful to the "weather" idea, but could not get a better image. Nonetheless, the details of the "paperclip maximizer argument" are valid, though not primary, for the real point: AI applied to strategy, tactics, defence and weapon management can suffer from the "Orthogonality thesis" (the decision may be out of a process unaware of human values) and "Instrumental convergence" (the decision may be out of a process unaware of its legitimate boundaries). The primary point remains "You do not put an "enfant prodige" in charge of systems with overextensive consequences".