3 ms·

How this argument reads to someone concerned about the trajectory of AI and the governance systems around it (current or future): There are various technical pr

How this argument reads to someone concerned about the trajectory of AI and the governance systems around it (current or future): There are various technical problems preventing these systems from being as scary as you imagine, but don't worry because we're working hard to solve them! Also "agency" isn't a requirement for almost any AI doom scenario I've heard of. Goals and the formation of instrumental goals is all it takes.

romafirst3 2y ago
But “goals” is doing an insane amount of heavy lifting here. In order to have goals you need to have a model of the world. LLMs do not have a model of the world. Your goal then is changing this world somehow so you need actuators. So you need a model and actuators. All we have is a stream of words appended in a probabilistic manner.
- llamaimperative 2y ago
  If only there were some examples of streams of words being used to enlist people to build (or become) actuators for evil purposes…
semi-extrinsic 2y ago
It's not that there are "technical problems" preventing anything. It's that these systems (LLMs) do not possess the fundamental properties that are necessary for being scary, and we have no clue at all how the systems would get those properties. Current "hyperintelligent AI" fears are exactly like the "grey goo" fear when nanotechnology was a buzzword in the early 2000s. We can all agree that if you take the fundamental vague concept "AI" or "nanotechnology" and pile on a couple of wheelbarrows full of hypotheticals, you get to something scary. That doesn't mean it has any relevance for the universe we currently exist in.
- llamaimperative 2y ago
  And yet you qualify your statement with “current.” Why? People anticipating future risks and discussing how and when to detect and mitigate them is actually exactly what we should be doing.

romafirst3 2y ago

But “goals” is doing an insane amount of heavy lifting here. In order to have goals you need to have a model of the world. LLMs do not have a model of the world. Your goal then is changing this world somehow so you need actuators. So you need a model and actuators. All we have is a stream of words appended in a probabilistic manner.

llamaimperative 2y ago
If only there were some examples of streams of words being used to enlist people to build (or become) actuators for evil purposes…

semi-extrinsic 2y ago

It's not that there are "technical problems" preventing anything. It's that these systems (LLMs) do not possess the fundamental properties that are necessary for being scary, and we have no clue at all how the systems would get those properties. Current "hyperintelligent AI" fears are exactly like the "grey goo" fear when nanotechnology was a buzzword in the early 2000s. We can all agree that if you take the fundamental vague concept "AI" or "nanotechnology" and pile on a couple of wheelbarrows full of hypotheticals, you get to something scary. That doesn't mean it has any relevance for the universe we currently exist in.

llamaimperative 2y ago
And yet you qualify your statement with “current.” Why? People anticipating future risks and discussing how and when to detect and mitigate them is actually exactly what we should be doing.