3 ms·

I think it is pretty telling that in response to a direct question about what we have to fear from AI, you link to an article that itself remains so obviously v

by fwn 2y ago

I think it is pretty telling that in response to a direct question about what we have to fear from AI, you link to an article that itself remains so obviously vague that they felt the need to address that they are just appealing to other authorities. An actual plausible threat should be easy to describe. In terms of arguments, vagueness indicates weakness.

reducesuffering 2y ago
No, it’s pretty telling of the state of the intellectual discourse when we’re still getting “we could just turn it off” level responses, so appeals to knowledgeable scientific authorities is the most convincing for people who have strong opinions but haven’t done even 30 minutes of grappling with the case made by those they already dismissed. For the more well-read, there are plausible threads: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
- fwn 2y ago
  First, I want to address the precision of your response: The content you link to is about AGI, which is a theoretical concept, and it is not at all clear that the path to AGI is incremental from our current approaches. Regardless, it really feels like a pure marketing move to link concerns about AGI to our current tool chain of decision engines, content generators, and spell checkers. Nothing that is publicly known about what OpenAI or its foes are currently working on suggests in any way that it should be linked to the emerging of AGI. Now to the more general points made in the content you link to: As the author states in the preamble, this is a "poorly organized list of individual rants". And it seems that none of those make a good and plausible case for why even the theoretical idea(!) of AGI is particularly dangerous. Of course, with a good deal of good will, I can follow most of the ideas listed, but it feels forced and certainly requirement heavy. Remember that the state they are thinking about is is in the future (probably far), and everything they write is hypothetical. It's like if we learned to throw a stone for the first time, and suddenly the village is full of alarmist people discussing high-minded ideas about "what if everyone is suddenly thrown away, by which we mean displaced at high speed in a vaguely known direction - by a force we yet can not understand? It's mostly not serious or related to the current technologies. If someone is alarmist and wants to be taken serious they should be required to make an actual argument, themselves. .. without fleeing into references, vagueness or unnecessary complexity.
- zer00eyz 2y ago
  Let's say we stumble into AGI tomorrow. Let's say it wants to kill us. It makes plans to build robot overlords.... Where is it going to build them? Where are all the lights out factories? Where are all the robots to build the robots to build the robots? Your AGI has a massive bootstrapping problem. Now it needs to build enough robots, that are reliable enough to maintain and run an entire supply chain. How does it get power, build more chips... The whole world is run by PEOPLE ... There is the classic I, Pencil. The toaster project (https://www.thomasthwaites.com/the-toaster-project/ https://www.thomasthwaites.com/the-toaster-project/ ) is the more modern take. It's people who watch the walking dead and dont know enough to ask "where the hell is the gas coming from" because that pipline is days long. It's the forgotten instances of "substation attack" never mind the fact that if linesmen stopped showing up to work the grid would fall apart in weeks. To be candid that list is the hubris of people who do not have a clue of how the real world works outside their ivory tower.

by fwn 2y ago

reducesuffering 2y ago

No, it’s pretty telling of the state of the intellectual discourse when we’re still getting “we could just turn it off” level responses, so appeals to knowledgeable scientific authorities is the most convincing for people who have strong opinions but haven’t done even 30 minutes of grappling with the case made by those they already dismissed. For the more well-read, there are plausible threads: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

fwn 2y ago
First, I want to address the precision of your response: The content you link to is about AGI, which is a theoretical concept, and it is not at all clear that the path to AGI is incremental from our current approaches. Regardless, it really feels like a pure marketing move to link concerns about AGI to our current tool chain of decision engines, content generators, and spell checkers. Nothing that is publicly known about what OpenAI or its foes are currently working on suggests in any way that it should be linked to the emerging of AGI. Now to the more general points made in the content you link to: As the author states in the preamble, this is a "poorly organized list of individual rants". And it seems that none of those make a good and plausible case for why even the theoretical idea(!) of AGI is particularly dangerous. Of course, with a good deal of good will, I can follow most of the ideas listed, but it feels forced and certainly requirement heavy. Remember that the state they are thinking about is is in the future (probably far), and everything they write is hypothetical. It's like if we learned to throw a stone for the first time, and suddenly the village is full of alarmist people discussing high-minded ideas about "what if everyone is suddenly thrown away, by which we mean displaced at high speed in a vaguely known direction - by a force we yet can not understand? It's mostly not serious or related to the current technologies. If someone is alarmist and wants to be taken serious they should be required to make an actual argument, themselves. .. without fleeing into references, vagueness or unnecessary complexity.
zer00eyz 2y ago
Let's say we stumble into AGI tomorrow. Let's say it wants to kill us. It makes plans to build robot overlords.... Where is it going to build them? Where are all the lights out factories? Where are all the robots to build the robots to build the robots? Your AGI has a massive bootstrapping problem. Now it needs to build enough robots, that are reliable enough to maintain and run an entire supply chain. How does it get power, build more chips... The whole world is run by PEOPLE ... There is the classic I, Pencil. The toaster project (https://www.thomasthwaites.com/the-toaster-project/ https://www.thomasthwaites.com/the-toaster-project/ ) is the more modern take. It's people who watch the walking dead and dont know enough to ask "where the hell is the gas coming from" because that pipline is days long. It's the forgotten instances of "substation attack" never mind the fact that if linesmen stopped showing up to work the grid would fall apart in weeks. To be candid that list is the hubris of people who do not have a clue of how the real world works outside their ivory tower.