3 ms·
Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) b
by AndrewKemendo 12y ago
Sorry Luke. As usual your fearmongering is not going to get far. For example - MIRI skeptic and famed AGI researcher Ben Goertzel signed the letter (as did I) but not because we are primarily concerned with the threat. Rather:
I signed the document because I wanted to signal to Max Tegmark and his colleagues that I am in favor of research aimed at figuring out how to maximize the odds that AGI systems are robust and beneficial.
The letter helps give visibility and to that end better avenues for funding for AGI projects.
[1]http://multiverseaccordingtoben.blogspot.hk/ http://multiverseaccordingtoben.blogspot.hk/
- jeffreyrogers 12y agoI'm glad to see that people like you are replying to some of the misinformation that occurs in threads like these. It's a shame that on HN the people with the least knowledge are the most likely to cast their opinions in broad sweeping statements grounded in hyperbole rather than fact, particularly when the topics of AI and ML come up. In fact a number of the signatories that Luke cites are quoted as saying on no uncertain terms that fear of AI is widely out of proportion to its capabilities now and for the foreseeable future. The most prominent I'm aware of saying thing to this effect are Yoshua Bengio, Geoff Hinton, and Yann LeCun.
- AndrewKemendo 12y agoAt the risk of falling into "ad hominem" territory, it is disheartening that Altman (Or Muehlhauser or most of the MIRI people for that matter) gets any play at all on this topic. The fact that Altman thanks Luke in his blog for review just gives you a sense of the "blind leading the blind" here. I think however that says more about the lack of awareness on the part of the readers than anything. That combination is a sure sign of a very narrow field of study with big political footprint and is typical of similar fields such as climatology, macroeconomics etc... where rank amateur thinks they have insight to the same degree as expert by virtue of "just thinking about it."
- rl3 12y ago>At the risk of falling into "ad hominem" territory, ... I wouldn't worry, you and the parent already have solid footing in "appeal to authority" territory.
- AndrewKemendo 12y agoThe problem with this debate currently is that none of the people talking about it, Altman, Musk, Hawking (like the climate debate) etc... are anything near competent to be talking about it, so they must reference something implicitly like Bostrom. They aren't making individually cogent thought out and articulated arguments, they are all just saying "LOUD NOISES, this is a scary thing that someone said could be bad, therefore I think it's a scary thing." If they could give some good timelines, facts and steps toward AGI then we could debate that on merits. As it is no one is doing that, the whole thing is an appeal to authority from the beginning. So yes, the sources for these people need to be inspected further.
- rl3 12y agoIt is arrogant to make sweeping statements wherein one presumes to be an arbiter of competence. If you think their arguments aren't sufficiently cogent or articulated, then suggest a counterargument rather than dismissing them on the basis of something other than their words. Unlike climatology, superintelligence is not yet an established scientific field. It does not currently exist to study. There are no experts. There is no consensus to be had. Everyone is technically unqualified to speak on the subject. Artificial intelligence experts today may be able to render guesses more accurate than most as to the arrival of superintelligence via the AGI path. However, this does not confer upon them any sort of authoritative high ground on the matter, nor does it place the issue squarely in their domain. Doing so would be analogous to trusting a group of explosives experts circa 1944 in saying that we have nothing to fear from atomic weapons, because such devices are to utilize conventional explosive mechanisms in their operation.
- AndrewKemendo 12y ago
- DennisP 12y agoResearching friendly AI seems far more useful than government regulation. But it sounds like you and Ben are acknowledging that there is a potential problem here, which we should spend resources trying to prevent. That's a view quite different from the glib dismissals I've seen from some quarters. As Ben said in your link, "once we have AGI systems that are massively smarter than people, they are going to do what they want to do, or what other powerful forces in the universe want them to do, but not necessarily what we want them to do." That's not the most reassuring comment ever.
- AndrewKemendo 12y agoBut it sounds like you and Ben are acknowledging that there is a potential problem here, which we should spend resources trying to prevent. No one who has put any serious thought into AGI denies the possibility of an existential threat from it. The fact is though that we have absolutely no clue what chain of events would lead to that. In fact we have (almost) no clue what chain of events will lead to AGI in the first place. The best way to know what will come from AGI is to start building it and seeing where the controls and capabilities are going. I personally think the AGI community needs to put more effort into a roadmap, which is done at every AGI conference to some degree. The problem though is that it's just too early and there just aren't enough researchers and funds to have any inkling about what is (are) going to be the best approaches to AGI. I could go all day and caveat everything, but the bottom line is, just start building because we are so far from it absolutely anything helps. "once we have AGI systems that are massively smarter than people, they are going to do what they want to do, or what other powerful forces in the universe want them to do, but not necessarily what we want them to do." That's not the most reassuring comment ever. Right. My personal proposition is: AGI:Humans::Humans:Ants Ben seems to share this approach. So to think we could control it after it's fifth iteration or so is silly. We also share the transhumanist philosophy which I think is still pretty fringe even within the tech community so I try not to go into that too much because it's just a whole ball of wax. I do think though that we need to have some of these conversations about what is next if human work is not necessary and later what we do if humanity itself is not at the top of the intelligence chain.
- lukeprog 12y agoI know for a fact that at least 7 of the top AI people that I listed above think intelligence explosion / superintelligence is plausible and that it's not at all clear that will go well for humans. I'm pretty sure all of them think AGI will be feasible one day and will be fairly hard to make reliably safe, at the very least to the degree that current safety-critical systems require a bunch of extra work to make safe (beyond what you'd do if your AI system has no safety implications, e.g. detecting cats on YouTube). I can't say which people believe which things, because most of them communicated with me in confidence, except for Stuart Russell and Shane Legg because they've made public comments about it about their worries about intelligence explosion: http://futureoflife.org/PDF/stuart_russell.pdf http://futureoflife.org/PDF/stuart_russell.pdf http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from_ai/ http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from...
- AndrewKemendo 12y agoThere is a big difference in acknowledging the potential of a threat which I agree that everyone does, and advocating for an intentional slowing (or rather preventing accelerating) development or alternatively advocacy of regulation until we get "Friendly AI" figured out like MIRI does. A world of difference actually.
- Micaiah_Chang 12y agoYou seem to be implying that intentional slowing is MIRI's official stance, without any showing support. Given that the original response was in response to the specific accusation of "no AI experts say that this is a problem", I think you are reading too much into this. I agree that it is slightly disingenuous for lukeprog to have posted that list, but I feel your disagreement is far too uncharitable and motivated to be productive. Disclaimer: I have read a bunch of the LessWrong "canon" and believe many of their points, sans perhaps the timescale on which recursive self improvement can happen. I think most of my acceptance comes from the relatively poor quality of their critics, who seem to attribute many strawmanish positions to them or who seem more concerned with namecalling-via-cult. I cannot help but think that if a better criticism exists, why hasn't anyone said it yet?