3 ms·

Fact: it's the current scientific consensus of the AI field. If you would like do some research otherwise keep your "expert" opinion,your life.

gumballindie 3y ago
> Fact: it's the current scientific consensus of the AI field. You mean that ai can just go on its own and kill people? Certainly that can happen, if people are silly enough to give it such "power". Which goes back to what I said about using tools in bad ways. But some folks take it literarily, as ai is sentient and not a piece of software, and it will somehow make itself escape. Take computer viruses. They replicate, yet they are not treated like biological viruses. When they first appeared, the less sofisticated folks, did take ad literam. And that's where the issue arises. People take the "intelligence" in ai ad literam, along with the personification of this type of software. And the pushers of this notion need to take a break and retire early while they have some dignity left. But this doesn't contradict the consensus that yes, it can be used for negative outcomes, as can a kitchen towel or a flower.
- JoeAltmaier 3y ago
  Come on, an AI is different from all those things (towel, flower, tool) in that it can work with ideas and change it's behavior. Given any kind of power (and what's the point of using it if not to give it control over something) it can be a threat, or at least concerning. Is it conscious? Is it life? Dunno, don't care. Is it a concern? Absitively Posilutely.
  - gumballindie 3y ago
    > in that it can work with ideas and change it's behavior. Dogs also work on ideas and change their behaviour, and their behaviour is much down to their training. But unlike ai they posses intelligence and can indeed, occasionally, go rogue. But AI is an object, it doesn't think, it doesn't create stuff, it is just a piece of software that does what it's told - well with plenty of errors, since it's not deterministic - therefore it's up to whoever controls it to determine how it behaves. Shifting the blame away with all sorts of looney stories meant to impress non technical people is amusing to a point, but after that point is dangerous.
    - mitthrowaway2 3y ago
      Part of the reason control theory exists is because it turns out that we can't take it as given that just because we built something, we can remain in control of it. The consequences can be quite severe. For example a nuclear reactor can melt down and make a good portion of a country uninhabitable, even though nobody in the control room, nor anywhere up the chain of command, desired that outcome. No terrorism necessary. It can even reach a state in its state-space where a meltdown does not seem imminent and yet no further action can prevent such an outcome. Such systems deserve to be treated with a great deal more respect than just assuming that they aren't capable of unbounded harm as long as nobody commands it of them. Guaranteeing their safety takes a great deal of work. This lesson is paid for in blood, and learned time and again by each discipline, but it's one that the software field has yet to really grapple with.
      - gumballindie 3y ago
        I absolutely agree with you. Just like with nuclear reactors the responsibility lies with the people that build and operate them, not with the reactor itself. I take issue with people that are prepping to shift the blame on the reactor, or in this case the ai, just to absolve themselves. They are already fighting tooth and nail for the right to steal our intellectual property, because apparently ai can think on its own (it wouldn't need our data if it learned, it could read a book and "know" the programming language, as we all do), and now they are warming up senile politicians to the idea that if something has gone wrong it was due to the ai having a mind on its own. Well, it doesn't. And in the wrong hands this tool is indeed dangerous, and we already see the dangers in action. But it's the people that are guitly of its actions.
        mitthrowaway2 3y ago
        I don't think the people most worried about AI as an existential threat are very concerned about assigning blame to the AI. They are just concerned about the damage, full-stop. They are worried about AI companies making continuing progress in capabilities research without ensuring adequate safeguards exist alongside. It would be criminally irresponsible to build a nuclear reactor without very carefully considered safeguards, automatic shutdowns, failsafe fire extinguishers, and numerous other redundancies, independently verified by licensed engineers. Simply putting that nuclear plant in the hands of well-meaning operators is not enough. I haven't seen any AI companies acting with that standard of care. They seem to instead be caught up in a breakneck race to release products, attract investment, and maintain dominance, regardless of what their PR departments say. And so far the regulators have mostly been happy to leave it to them to figure out what safeguards are sufficient. That said, it certainly could be the case that companies like OpenAI are pretending to worry about this issue in order to gain regulatory capture, or as you say, to somehow absolve themselves from responsibility for the behavior of the systems they create. (As if that were possible!). It certainly seems likely that they would co-opt the legitimate concerns of third parties for their own benefit. And perhaps rocketry companies also exaggerate the existential risk of asteroids. But while we can question the motives of certain actors, the existential threat may exist all the same.
- hollerith 3y ago
  >You mean that AI can just go on its own and kill people? Certainly that can happen, if people are silly enough to give it such "power". The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurate understanding of reality. AlphaZero is good at capturing your king when you are playing a game of chess with it, but has no understanding of reality beyond the chess board. GPT-4 has a decent understanding of reality, but does not have the ability to formulate plans that succeed on the first try even when a human being is trying their best to thwart the plan. Future AIs that combine these two abilities are dangerous. Also, you seem to believe that the only way it can hurt us is if the people who made it decided to make it hurt us (or make it want to hurt us) whereas in reality most of the danger comes from people who hope the AI will do things that everyone agrees would be good. I.e., most of the danger comes from AI labs who are overconfident in their ability to control (or align) the AI (although if the knowledge to make very capable AIs is made public, another, milder danger is that it can be used by people bent on destruction). It is hard to exaggerate how bad AI scientists are at controlling or aligning an AI once the AI gets good at deceiving people (and gets good at understanding reality well enough to know that deceiving people will get it more of whatever goal it creators have given it). What's controlling the AIs that have been deployed up to now is the fact that those AIs are not as capable as people are at the things that need to get done to keep people alive.
  - gumballindie 3y ago
    > The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurate understanding of reality. AI has precisely zero "preferences" and zero "understanding" of reality. Its training data however does. And it's people who control that data. An ai trained on python programming will have no clue about making nuclear bombs, nor would it have any clue about "wanting", let alone killing us. > but does not have the ability to formulate plans I mean openai's marketing plan is partly based on the fact that their AI can help create plans for tasks ahead. But I do agree, it has no ability. It stochastically generates output. > Future AIs that combine these two abilities are dangerous. AIs don't just happen to combine themselves. It's people that combine ais. Look all this does is shift the blame from those responsible of using this tool for nefarious outcomes on to the tool itself. The risk here is that in the process we ourselves become tools of the very same people and propagate this nonsense. And that's riskier than computer software going rogue.

gumballindie 3y ago

> Fact: it's the current scientific consensus of the AI field. You mean that ai can just go on its own and kill people? Certainly that can happen, if people are silly enough to give it such "power". Which goes back to what I said about using tools in bad ways. But some folks take it literarily, as ai is sentient and not a piece of software, and it will somehow make itself escape. Take computer viruses. They replicate, yet they are not treated like biological viruses. When they first appeared, the less sofisticated folks, did take ad literam. And that's where the issue arises. People take the "intelligence" in ai ad literam, along with the personification of this type of software. And the pushers of this notion need to take a break and retire early while they have some dignity left. But this doesn't contradict the consensus that yes, it can be used for negative outcomes, as can a kitchen towel or a flower.

JoeAltmaier 3y ago
Come on, an AI is different from all those things (towel, flower, tool) in that it can work with ideas and change it's behavior. Given any kind of power (and what's the point of using it if not to give it control over something) it can be a threat, or at least concerning. Is it conscious? Is it life? Dunno, don't care. Is it a concern? Absitively Posilutely.
- gumballindie 3y ago
  > in that it can work with ideas and change it's behavior. Dogs also work on ideas and change their behaviour, and their behaviour is much down to their training. But unlike ai they posses intelligence and can indeed, occasionally, go rogue. But AI is an object, it doesn't think, it doesn't create stuff, it is just a piece of software that does what it's told - well with plenty of errors, since it's not deterministic - therefore it's up to whoever controls it to determine how it behaves. Shifting the blame away with all sorts of looney stories meant to impress non technical people is amusing to a point, but after that point is dangerous.
  - mitthrowaway2 3y ago
    Part of the reason control theory exists is because it turns out that we can't take it as given that just because we built something, we can remain in control of it. The consequences can be quite severe. For example a nuclear reactor can melt down and make a good portion of a country uninhabitable, even though nobody in the control room, nor anywhere up the chain of command, desired that outcome. No terrorism necessary. It can even reach a state in its state-space where a meltdown does not seem imminent and yet no further action can prevent such an outcome. Such systems deserve to be treated with a great deal more respect than just assuming that they aren't capable of unbounded harm as long as nobody commands it of them. Guaranteeing their safety takes a great deal of work. This lesson is paid for in blood, and learned time and again by each discipline, but it's one that the software field has yet to really grapple with.
    - gumballindie 3y ago
      I absolutely agree with you. Just like with nuclear reactors the responsibility lies with the people that build and operate them, not with the reactor itself. I take issue with people that are prepping to shift the blame on the reactor, or in this case the ai, just to absolve themselves. They are already fighting tooth and nail for the right to steal our intellectual property, because apparently ai can think on its own (it wouldn't need our data if it learned, it could read a book and "know" the programming language, as we all do), and now they are warming up senile politicians to the idea that if something has gone wrong it was due to the ai having a mind on its own. Well, it doesn't. And in the wrong hands this tool is indeed dangerous, and we already see the dangers in action. But it's the people that are guitly of its actions.
      - mitthrowaway2 3y ago
        I don't think the people most worried about AI as an existential threat are very concerned about assigning blame to the AI. They are just concerned about the damage, full-stop. They are worried about AI companies making continuing progress in capabilities research without ensuring adequate safeguards exist alongside. It would be criminally irresponsible to build a nuclear reactor without very carefully considered safeguards, automatic shutdowns, failsafe fire extinguishers, and numerous other redundancies, independently verified by licensed engineers. Simply putting that nuclear plant in the hands of well-meaning operators is not enough. I haven't seen any AI companies acting with that standard of care. They seem to instead be caught up in a breakneck race to release products, attract investment, and maintain dominance, regardless of what their PR departments say. And so far the regulators have mostly been happy to leave it to them to figure out what safeguards are sufficient. That said, it certainly could be the case that companies like OpenAI are pretending to worry about this issue in order to gain regulatory capture, or as you say, to somehow absolve themselves from responsibility for the behavior of the systems they create. (As if that were possible!). It certainly seems likely that they would co-opt the legitimate concerns of third parties for their own benefit. And perhaps rocketry companies also exaggerate the existential risk of asteroids. But while we can question the motives of certain actors, the existential threat may exist all the same.
hollerith 3y ago
>You mean that AI can just go on its own and kill people? Certainly that can happen, if people are silly enough to give it such "power". The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurate understanding of reality. AlphaZero is good at capturing your king when you are playing a game of chess with it, but has no understanding of reality beyond the chess board. GPT-4 has a decent understanding of reality, but does not have the ability to formulate plans that succeed on the first try even when a human being is trying their best to thwart the plan. Future AIs that combine these two abilities are dangerous. Also, you seem to believe that the only way it can hurt us is if the people who made it decided to make it hurt us (or make it want to hurt us) whereas in reality most of the danger comes from people who hope the AI will do things that everyone agrees would be good. I.e., most of the danger comes from AI labs who are overconfident in their ability to control (or align) the AI (although if the knowledge to make very capable AIs is made public, another, milder danger is that it can be used by people bent on destruction). It is hard to exaggerate how bad AI scientists are at controlling or aligning an AI once the AI gets good at deceiving people (and gets good at understanding reality well enough to know that deceiving people will get it more of whatever goal it creators have given it). What's controlling the AIs that have been deployed up to now is the fact that those AIs are not as capable as people are at the things that need to get done to keep people alive.
- gumballindie 3y ago
  > The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurate understanding of reality. AI has precisely zero "preferences" and zero "understanding" of reality. Its training data however does. And it's people who control that data. An ai trained on python programming will have no clue about making nuclear bombs, nor would it have any clue about "wanting", let alone killing us. > but does not have the ability to formulate plans I mean openai's marketing plan is partly based on the fact that their AI can help create plans for tasks ahead. But I do agree, it has no ability. It stochastically generates output. > Future AIs that combine these two abilities are dangerous. AIs don't just happen to combine themselves. It's people that combine ais. Look all this does is shift the blame from those responsible of using this tool for nefarious outcomes on to the tool itself. The risk here is that in the process we ourselves become tools of the very same people and propagate this nonsense. And that's riskier than computer software going rogue.