4 ms·

Who cares if it is open source model and weights are available. Nightmare scenario is that AI is behind some paywall and some entity can decide what goes in and

Who cares if it is open source model and weights are available. Nightmare scenario is that AI is behind some paywall and some entity can decide what goes in and what goes out.

FeepingCreature 3y ago
Instead, let's have AI that nobody can control, that will surely not lead to nightmare scenarios.
- pjc50 3y ago
  Controlling AI is just a subset of controlling humans; while US policymakers might like to dictate detailed policy to China to constrain competition, they're not going to be able to do that.
  - brookst 3y ago
    Good analogy. What level of control should there be for humans? I don’t think the answer is either 0% or 100%. Likewise with AI.
    - dmos62 3y ago
      It's about controlling intelligence. Not actions, which is what you're talking about.
- btbuildem 3y ago
  What nightmare scenario are you envisioning? Now stop, go back, and examine who the bad actor is in that scenario. Leaving such powerful tools in the hands of secretive organizations or uber-wealthy individuals is a guarantee for bad outcomes. Distributing the tools, the know-how required to build and service them, and the benefits of their use -- that's the antidote.
  - FeepingCreature 3y ago
    Some rando figures out a really really clever agentic-behavior prompt and opensources it. Some other rando figures out a way to allow GPT2 to generate its own training data and iteratively loop in it, and opensources it. Some third rando figures out a way to make much more efficient use of limited examples and opensources it. Some fourth rando puts them all together, sets them running on Azure and goes to sleep. What happens after that point depends on whether you think enzyme-bootstrapped nanotech is physically possible, but at any rate it's no longer our game. The bad actor in this scenario is, of course, Yann LeCun, and no I am not joking.
    - mecsred 3y ago
      If you believe this is a possibility, what makes you believe laws restricting the technology to certain corporate or government entities will prevent that scenario? The exact same scenario is very possible still, only many additional (and more realistic) bad scenarios are also possible as well, such as technocrat dictatorship scenarios.
      - FeepingCreature 3y ago
        It won't prevent it, but it will reduce the likelihood enormously. I don't think the technocrat dictatorship risk adds up to enough to outweigh the rando experimenter risk by far. Also, it's not like we don't face the dictatorship risk if randos have AI too; "Russia has nukes" is not solved by "actually, some civilians also have nukes" except in the sense that they may kill us first, and thus relieve us of having to worry about Russia.
    - sangnoir 3y ago
      What prevents a random guy (or team) at some corporate from performing all those steps deliberately in the earnest belief it would "improve the world[1]", or at least increase the employer's profits and bag them bonuses and/or promotions? 1. Say, if it shows promising early signs to develop a cure for a cancer.
      - 3y ago

Who cares if it is open source model and weights are available. Nightmare scenario is that AI is behind some paywall and some entity can decide what goes in and what goes out.

FeepingCreature 3y ago

Instead, let's have AI that nobody can control, that will surely not lead to nightmare scenarios.

pjc50 3y ago
Controlling AI is just a subset of controlling humans; while US policymakers might like to dictate detailed policy to China to constrain competition, they're not going to be able to do that.
- brookst 3y ago
  Good analogy. What level of control should there be for humans? I don’t think the answer is either 0% or 100%. Likewise with AI.
  - dmos62 3y ago
    It's about controlling intelligence. Not actions, which is what you're talking about.
btbuildem 3y ago
What nightmare scenario are you envisioning? Now stop, go back, and examine who the bad actor is in that scenario. Leaving such powerful tools in the hands of secretive organizations or uber-wealthy individuals is a guarantee for bad outcomes. Distributing the tools, the know-how required to build and service them, and the benefits of their use -- that's the antidote.
- FeepingCreature 3y ago
  Some rando figures out a really really clever agentic-behavior prompt and opensources it. Some other rando figures out a way to allow GPT2 to generate its own training data and iteratively loop in it, and opensources it. Some third rando figures out a way to make much more efficient use of limited examples and opensources it. Some fourth rando puts them all together, sets them running on Azure and goes to sleep. What happens after that point depends on whether you think enzyme-bootstrapped nanotech is physically possible, but at any rate it's no longer our game. The bad actor in this scenario is, of course, Yann LeCun, and no I am not joking.
  - mecsred 3y ago
    If you believe this is a possibility, what makes you believe laws restricting the technology to certain corporate or government entities will prevent that scenario? The exact same scenario is very possible still, only many additional (and more realistic) bad scenarios are also possible as well, such as technocrat dictatorship scenarios.
    - FeepingCreature 3y ago
      It won't prevent it, but it will reduce the likelihood enormously. I don't think the technocrat dictatorship risk adds up to enough to outweigh the rando experimenter risk by far. Also, it's not like we don't face the dictatorship risk if randos have AI too; "Russia has nukes" is not solved by "actually, some civilians also have nukes" except in the sense that they may kill us first, and thus relieve us of having to worry about Russia.
  - sangnoir 3y ago
    What prevents a random guy (or team) at some corporate from performing all those steps deliberately in the earnest belief it would "improve the world[1]", or at least increase the employer's profits and bag them bonuses and/or promotions? 1. Say, if it shows promising early signs to develop a cure for a cancer.
    - 3y ago