3 ms·

Yes - but no one provided any argument for that. Instead, andreyf relied on it as a fact. I pointed out we can't assume that. As such: its reasonable to fear

by feral 11y ago

Yes - but no one provided any argument for that. Instead, andreyf relied on it as a fact. I pointed out we can't assume that. As such: its reasonable to fear AGI coming before we have that stuff sorted out. Hence, as such: there is a real risk here, which the safety folks are trying to mitigate; hence they aren't worried about a threat scenario easily dismissable by andreyf's argument. >It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. Ie. If you, or anyone had convincingly argued this, then it would be reasonable to dismiss (or at least greatly reduce) the concerns of the safety folk. But no one has, so its not reasonable to.

andreyf 11y ago
It is absolutely reasonable to fear how ML may be misused by a human, and those concerns fall mostly in the realm of political theory and economics. My claim is that I have seen no evidence that it is reasonable to fear a fictional self-aware autonomous AI (what I call HGI). The evidence people point to seems to have a very tenuous relationship with the reality of AI research. Once we have even an inkling of an idea of how one might approach creating an artificial HGI, we can discuss how we can prevent it from being evil and taking over the world. Until there's actually a proposal for how one might construct one, though, any such discussion is worse than useless. How can you talk about the risk of something when you literally have no idea what it is you are describing or how it's built, embodied with magical properties imagined by science fiction authors? To me, this makes about as much sense as talking about stopping aliens which could very well exist and could very well wipe out our civilization, i.e. great subject for wrapping up a party or going to the movies, but not exactly material for peer reviewed journals.
joe_the_user 11y ago
Me: It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. Feral: Yes - but no one provided any argument for that. Me: I should rephrase, I sort-of do above but still, negation of "it requires understanding" is "AGI could happen without understanding, at random, in some fashion". And I'd say that is the position that's no has provided to support for - again, akin to the worries that bring a giant supercollider online could destroy the earth. I mean, given that an AGI is an unknown, maybe bringing the supercollider online, another unknown, could create an AGI, which would then destroy the earth! Plus, human beings have generally not succeed in producing complex engineering achievements by accidents. The atomic bomb required a massive engineering effort, heavier-than-air flight required considerable effort, alchemists never synthesized CPUs by baking minerals at random, even largest pieces of modern software have not threatened to "achieve consciousness" even as they malfunction repeatedly. My argument is "consciousness by accident" is the extraordinary claim which requires extraordinary evidence. And AndreyF also adds - the dangers of ill-intentioned humans using AIs is here today, why worry about the hypothetical "getting out of control" problem when you humans who have historically inflicted vast amounts of misery on others of their species - oh, and consider several madmen fighting each other, that couldn't happen, not in Syria or whatever place one might name.

by feral 11y ago

andreyf 11y ago

It is absolutely reasonable to fear how ML may be misused by a human, and those concerns fall mostly in the realm of political theory and economics. My claim is that I have seen no evidence that it is reasonable to fear a fictional self-aware autonomous AI (what I call HGI). The evidence people point to seems to have a very tenuous relationship with the reality of AI research. Once we have even an inkling of an idea of how one might approach creating an artificial HGI, we can discuss how we can prevent it from being evil and taking over the world. Until there's actually a proposal for how one might construct one, though, any such discussion is worse than useless. How can you talk about the risk of something when you literally have no idea what it is you are describing or how it's built, embodied with magical properties imagined by science fiction authors? To me, this makes about as much sense as talking about stopping aliens which could very well exist and could very well wipe out our civilization, i.e. great subject for wrapping up a party or going to the movies, but not exactly material for peer reviewed journals.

joe_the_user 11y ago

Me: It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. Feral: Yes - but no one provided any argument for that. Me: I should rephrase, I sort-of do above but still, negation of "it requires understanding" is "AGI could happen without understanding, at random, in some fashion". And I'd say that is the position that's no has provided to support for - again, akin to the worries that bring a giant supercollider online could destroy the earth. I mean, given that an AGI is an unknown, maybe bringing the supercollider online, another unknown, could create an AGI, which would then destroy the earth! Plus, human beings have generally not succeed in producing complex engineering achievements by accidents. The atomic bomb required a massive engineering effort, heavier-than-air flight required considerable effort, alchemists never synthesized CPUs by baking minerals at random, even largest pieces of modern software have not threatened to "achieve consciousness" even as they malfunction repeatedly. My argument is "consciousness by accident" is the extraordinary claim which requires extraordinary evidence. And AndreyF also adds - the dangers of ill-intentioned humans using AIs is here today, why worry about the hypothetical "getting out of control" problem when you humans who have historically inflicted vast amounts of misery on others of their species - oh, and consider several madmen fighting each other, that couldn't happen, not in Syria or whatever place one might name.