5 ms·

That's true and relevant, but doesn't prove much. What's the big plan, what's the next step in the plan? Maciej's larger point is that the AI faces tons of ver

That's true and relevant, but doesn't prove much. What's the big plan, what's the next step in the plan? Maciej's larger point is that the AI faces tons of very difficult problems in escaping its physical constraints. It's simplistic to wave hands and say "the AI is super duper smart and will have no difficulty hacking all computer systems, inventing and manufacturing swarms of unbeatable nanobots, etc. without being detected or resisted". It seems to me that the core conceit of AI doomerism is that sufficient intelligence can overcome all barriers with some plan that is so smart, people would never think of it. This is much less plausible than believers take it to be. In mathematics alone, it is very easy to come up with a problem that the collective mathematical ingenuity of the entire human race is helpless before for decades, centuries, or longer.

actionfromafar 3y ago
Can we detect companies doing bad things, today? Yes, and we still have a hard time containing them and their externalities. AI will just mean more of the same. It will make companies even more efficient at what they already do. It will be detected, it will be resisted, but will it help? That's the big question IMHO.
- civilized 3y ago
  I find this to be a much more plausible area of concern than standard horror-story AI doomerism. But the more we shift our focus from fantasy to reality, the more we have to acknowledge reality, as Maciej does in this talk. It isn't plausible to prevent all the great powers from developing AI. We don't have that level of cooperation in the world today. We are going to need to adapt to this by keeping up with our adversaries, whether they be hostile nations, corporations, or mad scientists.
  - DirkH 3y ago
    Standard rebuttal to this is if we banned human cloning we can ban AGI. Also China stands to gain from a ban on AGI. Anything that the CCP thinks it can't control it doesn't like. Also China gets the vast amount of its AI capabilities from the West because of tech's "move fast and break things" culture, which means security is not prioritized and tech leaks common. If the US bans AGI, the other powers don't have a choice. Their domestic capabilities are not up to the task without snooping on US research labs and AI tech startups. The biggest boon to foreign AGI research is the US. The argument to not regulate AGI used to be: "AGI is impossible or very far away, AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable" Then it became "AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable" Nowadays all I hear is simply: "AGI R&D cannot be stopped and is inevitable" Yes it can be stopped, same as human cloning. Coordinating to regulate its development isn't some magical unreachable goal.
    - civilized 3y ago
      The economic and strategic benefits of developing AGI are... not comparable to those for human cloning.
    - actionfromafar 3y ago
      I’m not even talking about AGI. We can be in deep trouble long before that. And then where do you draw the line? Human cloning is a distinct and clear line. Ever more clever AI agents is not.
      - DirkH 3y ago
        Very valid point and very valuable question. In this regard it is clearly different from human cloning and I should have thought of that. Obvious in hindsight. In my defense though, if we can find a distinct clear line for AGI, my point stands that we can coordinate and regulate it. But yea... Your point stands that by the time we draw the line it might be too late.
- Barrin92 3y ago
  >Yes, and we still have a hard times containing them Only because we're most of the time not properly motivated because there's no urgency. When a real threat, like a war or a virus say, comes around suddenly we can organize things pretty effectively. It resembles the Bill Clinton quote about controlling the internet being like nailing jell-O to a wall. Yet autocratic governments have done that, and not just that but turned it into quite scary systems of control, with little difficulty. Just like 90s cypherpunk fantasies about digital anarchy these AI scenarios are nerd revenge fantasies. The intelligent guy outsmarting the big guy, but in reality physical power always wins.
  - actionfromafar 3y ago
    Only… what? I don’t understand what you are arguing here. Climate change is already an existential threat to millions of people. Yet companies do what they do best - optimise for their paper clip production.
    - Barrin92 3y ago
      climate change is a risk to millions of people but it's not an existential risk. Just sadly a big one with the people worst affected having little say in anything. That kind of threat also for AI I think is worth worrying about. I fully expect autonomous weapons to be a concern, showing up in the worst and poorest warzones first probably. However this is different from the largely fantastical genuinely existential AI risk scenarios. The appropriate comparison here is nuclear weaponry and we literally have global controls with not a single company or even a rogue actor having ever used one.
      - shkkmo 3y ago
        Climate change is not an existential risk to humanity, it may be an existential risk to modern human societies, it is definitely an existential risk to millions of people. > The appropriate comparison here is nuclear weapons Then complete the comparison. Nuclear weapons require specific materials and technology that can be controlled and the testing of any developed weaponry is fairly easy to detect. Despite these advantages, there are several countries that have them. If nuclear weapon proliferation is your comparison, then it sure looks like we will fail to prevent or stamp out the development of superintellegent AI. It we did succeed, what would it look like? We'd need to impose strict controls on computing capacity and it's usage. We'd need inspection regimes for every server farm, etc. It would require massive effort and fundemental changes to our society.
iliane5 3y ago
100% agree. However, seeing how excited Palantir is with their war assistant LLM , the US testing autonomous fighter jets a few months ago, etc. I think there's a decent chance that AI won't even have to break out of its constraints. It's pretty much guaranteed people are going to do the obviously dumb thing and give it capabilities it shouldn't have or is not equipped to deal with safely.
natosaichek 3y ago
This is like saying "sure, that site is susceptible to a sql injection, but that doesn't mean the whole thing is insecure." If you have an intelligent adversary and the stakes of them succeeding are high, it is the defenders job to prove the system secure. Systems don't start off secure and become vulnerable - they start off vulnerable until proven secure. So yes, its okay to say "the things we're doing to 'contain' ais are almost certainly inadequate" until shown otherwise.
- civilized 3y ago
  This analogy assumes that there exists some relatively straightforward, secure, practical defense against superintelligence that I am arguing against, like protecting a website against SQL injection. I do not argue against such a defense if it exists - by all means research and present proposals. My comment is about the plausibility of AI domination. I don't think it is that plausible, so I have different views than others on how important it is that we restrict the development of AI.
senttoschool 3y ago
>Maciej's larger point is that the AI faces tons of very difficult problems in escaping its physical constraints. Why is it difficult? We are already putting AIs in every electronic device. Someone has probably already put an LLM in a robot somewhere. And you don't think Boston Dynamics is thinking about putting an LLM in one of their robots to test? And surely the military is building AI fighting robots. And then there's the thought that a super intelligent AI can easily hack its way into any machine it wants. Heck, the super intelligent AI doesn't even have to convince humans. In the future, it can just convince dumber AIs that are already in robots.

actionfromafar 3y ago

Can we detect companies doing bad things, today? Yes, and we still have a hard time containing them and their externalities. AI will just mean more of the same. It will make companies even more efficient at what they already do. It will be detected, it will be resisted, but will it help? That's the big question IMHO.

civilized 3y ago
I find this to be a much more plausible area of concern than standard horror-story AI doomerism. But the more we shift our focus from fantasy to reality, the more we have to acknowledge reality, as Maciej does in this talk. It isn't plausible to prevent all the great powers from developing AI. We don't have that level of cooperation in the world today. We are going to need to adapt to this by keeping up with our adversaries, whether they be hostile nations, corporations, or mad scientists.
- DirkH 3y ago
  Standard rebuttal to this is if we banned human cloning we can ban AGI. Also China stands to gain from a ban on AGI. Anything that the CCP thinks it can't control it doesn't like. Also China gets the vast amount of its AI capabilities from the West because of tech's "move fast and break things" culture, which means security is not prioritized and tech leaks common. If the US bans AGI, the other powers don't have a choice. Their domestic capabilities are not up to the task without snooping on US research labs and AI tech startups. The biggest boon to foreign AGI research is the US. The argument to not regulate AGI used to be: "AGI is impossible or very far away, AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable" Then it became "AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable" Nowadays all I hear is simply: "AGI R&D cannot be stopped and is inevitable" Yes it can be stopped, same as human cloning. Coordinating to regulate its development isn't some magical unreachable goal.
  - civilized 3y ago
    The economic and strategic benefits of developing AGI are... not comparable to those for human cloning.
  - actionfromafar 3y ago
    I’m not even talking about AGI. We can be in deep trouble long before that. And then where do you draw the line? Human cloning is a distinct and clear line. Ever more clever AI agents is not.
    - DirkH 3y ago
      Very valid point and very valuable question. In this regard it is clearly different from human cloning and I should have thought of that. Obvious in hindsight. In my defense though, if we can find a distinct clear line for AGI, my point stands that we can coordinate and regulate it. But yea... Your point stands that by the time we draw the line it might be too late.
Barrin92 3y ago
>Yes, and we still have a hard times containing them Only because we're most of the time not properly motivated because there's no urgency. When a real threat, like a war or a virus say, comes around suddenly we can organize things pretty effectively. It resembles the Bill Clinton quote about controlling the internet being like nailing jell-O to a wall. Yet autocratic governments have done that, and not just that but turned it into quite scary systems of control, with little difficulty. Just like 90s cypherpunk fantasies about digital anarchy these AI scenarios are nerd revenge fantasies. The intelligent guy outsmarting the big guy, but in reality physical power always wins.
- actionfromafar 3y ago
  Only… what? I don’t understand what you are arguing here. Climate change is already an existential threat to millions of people. Yet companies do what they do best - optimise for their paper clip production.
  - Barrin92 3y ago
    climate change is a risk to millions of people but it's not an existential risk. Just sadly a big one with the people worst affected having little say in anything. That kind of threat also for AI I think is worth worrying about. I fully expect autonomous weapons to be a concern, showing up in the worst and poorest warzones first probably. However this is different from the largely fantastical genuinely existential AI risk scenarios. The appropriate comparison here is nuclear weaponry and we literally have global controls with not a single company or even a rogue actor having ever used one.
    - shkkmo 3y ago
      Climate change is not an existential risk to humanity, it may be an existential risk to modern human societies, it is definitely an existential risk to millions of people. > The appropriate comparison here is nuclear weapons Then complete the comparison. Nuclear weapons require specific materials and technology that can be controlled and the testing of any developed weaponry is fairly easy to detect. Despite these advantages, there are several countries that have them. If nuclear weapon proliferation is your comparison, then it sure looks like we will fail to prevent or stamp out the development of superintellegent AI. It we did succeed, what would it look like? We'd need to impose strict controls on computing capacity and it's usage. We'd need inspection regimes for every server farm, etc. It would require massive effort and fundemental changes to our society.

iliane5 3y ago

100% agree. However, seeing how excited Palantir is with their war assistant LLM , the US testing autonomous fighter jets a few months ago, etc. I think there's a decent chance that AI won't even have to break out of its constraints. It's pretty much guaranteed people are going to do the obviously dumb thing and give it capabilities it shouldn't have or is not equipped to deal with safely.

natosaichek 3y ago

This is like saying "sure, that site is susceptible to a sql injection, but that doesn't mean the whole thing is insecure." If you have an intelligent adversary and the stakes of them succeeding are high, it is the defenders job to prove the system secure. Systems don't start off secure and become vulnerable - they start off vulnerable until proven secure. So yes, its okay to say "the things we're doing to 'contain' ais are almost certainly inadequate" until shown otherwise.

civilized 3y ago
This analogy assumes that there exists some relatively straightforward, secure, practical defense against superintelligence that I am arguing against, like protecting a website against SQL injection. I do not argue against such a defense if it exists - by all means research and present proposals. My comment is about the plausibility of AI domination. I don't think it is that plausible, so I have different views than others on how important it is that we restrict the development of AI.

senttoschool 3y ago

>Maciej's larger point is that the AI faces tons of very difficult problems in escaping its physical constraints. Why is it difficult? We are already putting AIs in every electronic device. Someone has probably already put an LLM in a robot somewhere. And you don't think Boston Dynamics is thinking about putting an LLM in one of their robots to test? And surely the military is building AI fighting robots. And then there's the thought that a super intelligent AI can easily hack its way into any machine it wants. Heck, the super intelligent AI doesn't even have to convince humans. In the future, it can just convince dumber AIs that are already in robots.