4 ms·

Some rando figures out a really really clever agentic-behavior prompt and opensources it. Some other rando figures out a way to allow GPT2 to generate its own t

Some rando figures out a really really clever agentic-behavior prompt and opensources it. Some other rando figures out a way to allow GPT2 to generate its own training data and iteratively loop in it, and opensources it. Some third rando figures out a way to make much more efficient use of limited examples and opensources it. Some fourth rando puts them all together, sets them running on Azure and goes to sleep. What happens after that point depends on whether you think enzyme-bootstrapped nanotech is physically possible, but at any rate it's no longer our game. The bad actor in this scenario is, of course, Yann LeCun, and no I am not joking.

mecsred 3y ago
If you believe this is a possibility, what makes you believe laws restricting the technology to certain corporate or government entities will prevent that scenario? The exact same scenario is very possible still, only many additional (and more realistic) bad scenarios are also possible as well, such as technocrat dictatorship scenarios.
- FeepingCreature 3y ago
  It won't prevent it, but it will reduce the likelihood enormously. I don't think the technocrat dictatorship risk adds up to enough to outweigh the rando experimenter risk by far. Also, it's not like we don't face the dictatorship risk if randos have AI too; "Russia has nukes" is not solved by "actually, some civilians also have nukes" except in the sense that they may kill us first, and thus relieve us of having to worry about Russia.
sangnoir 3y ago
What prevents a random guy (or team) at some corporate from performing all those steps deliberately in the earnest belief it would "improve the world[1]", or at least increase the employer's profits and bag them bonuses and/or promotions? 1. Say, if it shows promising early signs to develop a cure for a cancer.
- FeepingCreature 3y ago
  Nothing. But having to worry about corporations and randos is worse than just having to worry about corporations.

mecsred 3y ago

If you believe this is a possibility, what makes you believe laws restricting the technology to certain corporate or government entities will prevent that scenario? The exact same scenario is very possible still, only many additional (and more realistic) bad scenarios are also possible as well, such as technocrat dictatorship scenarios.

FeepingCreature 3y ago
It won't prevent it, but it will reduce the likelihood enormously. I don't think the technocrat dictatorship risk adds up to enough to outweigh the rando experimenter risk by far. Also, it's not like we don't face the dictatorship risk if randos have AI too; "Russia has nukes" is not solved by "actually, some civilians also have nukes" except in the sense that they may kill us first, and thus relieve us of having to worry about Russia.

sangnoir 3y ago

What prevents a random guy (or team) at some corporate from performing all those steps deliberately in the earnest belief it would "improve the world[1]", or at least increase the employer's profits and bag them bonuses and/or promotions? 1. Say, if it shows promising early signs to develop a cure for a cancer.

FeepingCreature 3y ago
Nothing. But having to worry about corporations and randos is worse than just having to worry about corporations.