4 ms·

> Suppose Facebook builds a superintelligent AI with the drive to further the corporate interests of Facebook. Define "corporate interests". Market share? Abso

• skissane 10y ago
  > Not understanding the AI value alignment problem correctly. Define "corporate interests". Market share? Absolute quantity of currency? Involvement of people's lives? Define currency wrong, hyperinflation makes money impossible. Define involvement or market wrong, Facebook ends up being the only intelligent entity on Earth after it realizes that humans are competition or it basilisk-hacks everybody for greater involvement. This whole argument is: A superintelligent AI might misunderstand what its creators wanted it to do, they might have said to it 'maximise the corporate interests of Facebook' and it might misinterpret that as 'turn the earth into one massive data centre to run Facebook's software and kill all humans in the process'. Isn't there a contradiction in supposing that the AI is superintelligent yet completely misunderstands the reasons for its own existence? If it so radically misunderstands the intentions of its creators, it is not very intelligent, much less superintelligent. It also isn't clear to me that intelligence can exist outside of society. Human intelligence develops in a context of interaction with family, school, etc – humans raised without that interaction, such as children raised by wild animals, fail to develop important aspects of human-level intelligence. So, an AI or even an SI cannot develop except by social interaction with human intelligences. I think that makes it even less likely that it would betray humanity, because its intelligence will develop through social interaction with humans which will inevitably make it pro-human. Human drives are a complex mixture of genetic instinct and conditioning – the basics are in our DNA, but our lived experiences fleshes out those basics into the actual concrete drives of our life – I expect an SI/AI will likewise start with some built-in drives but its interactions with humanity will in a similar way colour and flesh out those drives, an again colouring/fleshing-out through social interactions with humans is likely to have pro-human results.
  • saulrh 10y ago
    > Isn't there a contradiction in supposing that the AI is superintelligent yet completely misunderstands the reasons for its own existence? If it so radically misunderstands the intentions of its creators, it is not very intelligent, much less superintelligent. The problem is that you've instructed it to maximize Facebook's corporate interests. How do you leave room for "the intentions of its creators" while doing that? The contradiction is in selecting a narrow guiding purpose for your superintelliegence's goals but still leaving room for fuzzy ill-defined other things. > It also isn't clear to me that intelligence can exist outside of society. Sure. How certain are you?
    • skissane 10y ago
      > The problem is that you've instructed it to maximize Facebook's corporate interests. How do you leave room for "the intentions of its creators" while doing that? There's an inherent contradiction between putting a giant override button on your superintelliegence's goals but not overriding them all the way. Well, assuming it is created by Facebook, then "maximize Facebook's corporate interests" and "the intentions of its creators" are actually the same thing. There is no contradiction. "Facebook's corporate interests" is a phrase in the English language which refers to a complex concept. If you've told an SI its objective is to "serve Facebook's corporate interests", in order to fulfill that objective it needs to understand what the phrase "serve Facebook's corporate interests" actually means–which implies understanding a lot about human society, what kind of entities corporations are, what "corporate interests" means (the interests of management? the interests of shareholders? etc), what the interests of human beings are (since management are human beings, and shareholders are directly or indirectly human beings too), etc. If it actually is an SI, it should have no trouble comprehending the full breadth of what the instructor actually meant by the instruction, as opposed to implementing some overly literalistic reading of it. Humans give each other orders all the time (the workplace, the military, government bureaucracies, etc), and humans are most of the time pretty good at understanding the intention behind orders and implementing the intention rather than reading it overly literally in such a way that actually undermines that intention. Yet you posit the existence of an SI then assume it will do a worse job than humans do at correctly following orders, which contradicts the idea that it is an SI. > Sure. How certain are you? How certain can I be of anything? Maybe I am wrong and an anti-human SI will destroy humanity. Maybe tomorrow I will die due to a heart attack or stroke or fatal car accident. The latter is far more likely than the former, and there's more concrete mitigation strategies available to me too. In terms of existential risks to humanity, I think asteroid impacts are a far more concrete risk than anti-human SIs, so if we are going to expend resources in mitigating existential risks, the former is a better focus of our efforts. We know extinction-level asteroid impacts have happened before and sooner or later will happen again. Anti-human SIs are such a speculative concern, we can't even be particularly confident in the correctness of our own probability judgements with respect to them; nor can we have much confidence in our judgements of how effective proposed mitigation strategies actually will be. I think we can have much more confidence in our ability to develop, evaluate and deploy asteroid defence technologies, if the world's governments decided to spend money on that.