5 ms·

The problem is that there is no “we” that can pump the brakes. There are a multitude of people and companies acting for a wide variety of motivations. The odds

The problem is that there is no “we” that can pump the brakes. There are a multitude of people and companies acting for a wide variety of motivations. The odds of getting a hundred thousand people to agree on anything, let alone with a profit motive involved, are zero. Also, you let the cat out of the bag on AI propaganda, so any halfway smart AI will now see through the ruse. Thanks a lot.

not_enoch_wise 4y ago
Are you calling me the end of humanity? I get that a lot.
didericis 4y ago
They’ll all be forced to pump the breaks when subtle errors compound through self reference and fill the internet/destroy the resource they’re extracting from through automation. It’s the same problem that caused the replication crisis. You start with something small, like a department for a new field, a random paper, etc. You explore and hit some interesting valuable knowledge. You share that, and it blows up and makes everyone’s lives vastly and unexpectedly better. So you pour a whole bunch of resources into getting more students, pushing more papers, creating more procedures, and building your little department into a giant machine. It works amazingly for a while, and then you think the machine is what’s generating the value and leave it on autopilot. Researchers that fit the machine remain, those that put more energy into actually exploring rather than fitting the machines criteria for explorer fade away. Knowledge slowly turns to nonsense through self reference as the true value, the researchers lucky enough to stumble on insight, get drowned out and leave.
goatlover 4y ago
That's what governments are supposed to do. However, we've seen with Covid, climate change, and the war in Ukraine that governments are not all going to agree, and political parties within governments which may take power will not all agree.
- brookst 4y ago
  Which is a good thing, right? Do we want every political party and government in the world to agree?
mLuby 4y ago
> There is no "we" that can pump the brakes.… The odds of getting a hundred thousand people to agree on anything, let alone with a profit motive involved, are zero. Laws and regulation can do exactly that. We could even use AI to write the legislation to outlaw itself.
- brookst 4y ago
  In every country? Is this UN action? Otherwise it’s a prisoner’s dilemma and countries that defect get all of the benefits and all of the risk, and countries that cooperate still get all of the risk, just none of the benefits.
  - mLuby 4y ago
    It's not so black or white. The International Criminal Court is one real-world example. The United States (to its shame, IMO) is not part of the ICC, yet most countries are and the institution is valuable despite not having 100% buy-in around the globe. Also the Prisoner's Dilemma over multiple rounds teaches different lessons than when it's a one-off situation.

not_enoch_wise 4y ago

Are you calling me the end of humanity? I get that a lot.

didericis 4y ago

They’ll all be forced to pump the breaks when subtle errors compound through self reference and fill the internet/destroy the resource they’re extracting from through automation. It’s the same problem that caused the replication crisis. You start with something small, like a department for a new field, a random paper, etc. You explore and hit some interesting valuable knowledge. You share that, and it blows up and makes everyone’s lives vastly and unexpectedly better. So you pour a whole bunch of resources into getting more students, pushing more papers, creating more procedures, and building your little department into a giant machine. It works amazingly for a while, and then you think the machine is what’s generating the value and leave it on autopilot. Researchers that fit the machine remain, those that put more energy into actually exploring rather than fitting the machines criteria for explorer fade away. Knowledge slowly turns to nonsense through self reference as the true value, the researchers lucky enough to stumble on insight, get drowned out and leave.

goatlover 4y ago

That's what governments are supposed to do. However, we've seen with Covid, climate change, and the war in Ukraine that governments are not all going to agree, and political parties within governments which may take power will not all agree.

brookst 4y ago
Which is a good thing, right? Do we want every political party and government in the world to agree?

mLuby 4y ago

> There is no "we" that can pump the brakes.… The odds of getting a hundred thousand people to agree on anything, let alone with a profit motive involved, are zero. Laws and regulation can do exactly that. We could even use AI to write the legislation to outlaw itself.

brookst 4y ago
In every country? Is this UN action? Otherwise it’s a prisoner’s dilemma and countries that defect get all of the benefits and all of the risk, and countries that cooperate still get all of the risk, just none of the benefits.
- mLuby 4y ago
  It's not so black or white. The International Criminal Court is one real-world example. The United States (to its shame, IMO) is not part of the ICC, yet most countries are and the institution is valuable despite not having 100% buy-in around the globe. Also the Prisoner's Dilemma over multiple rounds teaches different lessons than when it's a one-off situation.