9 ms·
Eliezer Yudkowsky on the Singularity, Bayesian Brains and Closet Goblins
- jo6gwb 11y agoKnowing nothing about this field, reading the comments did serve to temper my enthusiasm for the article.
- saint_fiasco 11y agoI only see three comments, one by a guy who complains that Yudkowsky was mean to him and has no academic credentials, one from a guy who calls Yudkowsky overconfident (in what, his belief that we should be cautious?) and naive for being a libertarian and one from a guy who admits he also knows nothing about the field.
- retbull 11y agoFor all that I loved his hpmor book I can't stand to read him speaking the way he does. It feels like he tries too hard to sound smart and has built his own hill that he gets to be king of and ignore the rest of the world from the top of.
- rdbcasillas 11y agoWell, if you have read most of his lesswrong articles(like I have), its hard to not enjoy the way he speaks and writes about these things. But I totally get that it feels obtuse and unnecessarily complex to someone without good exposure to his http://lesswrong.com/ http://lesswrong.com/ writings.
- andrewprock 11y agoI think that's a big part of the disconnect. Unless you are a devotee of EY, you generally don't "get" him. I suspect that it's because he mostly operates in the echo chamber that is the "less wrong" community. In short, he generally does not get credit for what he knows and understands because he cannot communicate clearly with people who do not already know and understand his philosophy. Said another way, if you can't explain things to people unless they already understand you, then you can't explain things to people.
- noobiemcfoob 11y agoYes, yes and more yes. I love his ideas and thoughts on many topics, but it always comes across as the person who was too busy being "brilliant" to learn how to interface well with the broader society. The sad part is that he really is brilliant.
- armitron 11y agoNo he's not, he's just skilled at manipulating people's perceptions of him based on a narrow set of premises that he introduced. Go through his non-philosophical papers and point to one thing in there that's brilliant. His miniscule technical output also seems to agree with this. I weep for the world that considers people like Yudkowsky brilliant.
- noobiemcfoob 11y agoThere are many definitions for "brilliant", though most of them heavily involve philosophy from my perspective. The easiest example of his work that I'd describe as brilliant is HPMOR. It is a story that makes the Harry Potter world not just bearable but beautiful and lively. There are portions where the philosophy comes on so thick as to disrupt the story, but on the whole it is a beautiful work. Reading it is how I first discovered EY.
- p1esk 11y agoI haven't read HPMOR, and I haven't read Harry Potter books, but I watched the movies, and I think HP world is beautiful as is. What makes it unbearable for you?
- deleted 11y ago[deleted]
- rdbcasillas 11y agoPoint taken. But there are some complex ideas that just need some amount of effort from the reader in order to "get" them. I don't know what is the right balance but if you are being introduced to new ideas, introduction of those ideas will also bring some vocabulary that you will (at least initially) be uncomfortable with. I guess I want to see someone communicate the ideas on less wrong in a simpler way without dumbing down or ignoring the nuances.
- throw12122 11y agoIs this a cult? I enjoy the was Halmos and Feynman talk about things. I've never read anything interesting from Yudkowsky...
- Moshe_Silnorin 11y agoI don't think it was a cult. LessWrong was an interesting place where interesting people gathered for a time; though LessWrong is basically on life support now. Gwern, Hal Finny, Wei Dai, Hannu Rajaniemi, Aaron Swartz, and Scott Alexander are some of the more well-known (outside of LessWrong) former members. It really was an interesting community for a while. There were meet-ups and Yudkowsky did solicit for his nonprofit, but it never seemed culty to me. And none of the people I mentioned ever seemed wary of criticizing Yudkowsky when they thought he was wrong. See, for example, Gwern's epic reply to Yudkowsky's post on automation: http://lesswrong.com/lw/hh4/the_robots_ai_and_unemployment_antifaq/ http://lesswrong.com/lw/hh4/the_robots_ai_and_unemployment_a... This doesn't seem like the type of reply an entranced follower would give their guru. The fact that this very-critical comment got twice as many upvotes as the original post is telling of the attitude of the community as a whole.
- greenrd 11y agoWorth noting that the community has spread out into a kind of "diaspora": some have gone into effective altruism (many retaining their AI risk beliefs, especially the ones in California); some have been hired to work on existential risks full-time; Scott Alexander now has his own (impressive) blog and some LWers still follow him there and go to meetups that he attends... Of course, it's also probably true that some have since graduated, got jobs and simply have less time to read and post stuff on the internet any more.
- dTal 11y agoDepends what you mean by "cult". They are a defined group of people that display a substantial amount of in-group closeness (google "lesswrong cuddle puddle" if you have the stomach for it), possessing unusual views, organised around a central leader figure, that believes the leader's worldview is superior. Poisoned kool-aid, however, is unlikely. Make of it what you will.
- retbull 11y agoI've read a lot of his work, but I don't think it needs to be as complex as it is. I am smart, well educated, and well read and I often have to pull out a dictionary to read what he writes. Maybe I just need to read more but I don't feel like he is trying to communicate his ideas only listen to himself speak.
- Moshe_Silnorin 11y agoYudkowsky writes without modesty and doesn't mind seeming arrogant. This doesn't particularly bug me; in fact I kind of like it - I think he's a fascinating guy. But a lot of people find his writing style intolerable, and he has a disproportionate number of antifans.
- gaze 11y agoEnemies. Adversaries? Anti-fans? Really?
- Moshe_Silnorin 11y agoThey aren't enemies in the same way a fan isn't a friend. I would say "antifan" describes a distinct thing.
- dsjoerg 11y agoI disagree that he tries too hard to sound smart. I despise the tendency to unnecessarily use complex words when simple ones will do. Police officers, for example, are rightly famous for this tendency. In Eliezer's writing and in this Q&A, the complex words are there because they are the most concise and precise expression of what he's trying to say -- simpler words would either be inaccurate or would have to go on at length. For example: Horgan: Is college overrated? Yudkowsky: It'd be very surprising if college were underrated, given the social desirability bias of endorsing college. You'd need a few paragraphs to explain what Yudkowsky means to people who don't already understand social desirability bias.
- retbull 11y agoI actually really liked that line because it was beautifully concise I was talking more about the essay answers later in the interview.
- saint_fiasco 11y agoI think those are Yudkowsky desperately trying to explain things without using scientific technical terms and lesswrongian jargon. You can see he kind of drops character with the 'cyborg' question.
- snowwrestler 11y agoYudkowsky has a way of thinking that he believes everyone else should adopt. He is skilled at using phrasing to frame his statements in his way of thinking. For example, instead of saying something like, "a lot of people think college is good," he phrases it in terms of a bias. Why? Because if he can get the other person talking about the subject in terms of biases, then he has advanced his own favorite way of thinking. But is liking college really a bias? And if so, compared to what norm? He offers no explanation; he just asserts it.
- greenrd 11y agoI don't think he's saying that liking college is a bias. Or rather, he is, but I think he's saying something subtly different (which implies the former): that those who denigrate college, or are "meh" about college, take a "social desirability hit" - they are likely to be seen as either unintelligent manual labourers, or Philistines, or annoying self-taught outliers, or insulting people who have put in the hard work to go to college, or a combination. And therefore young people grow up in such an environment and many of them have a strong belief, in such an environment, that college is a necessity, and don't really consider the evidence dispassionately.
- leeleelee 11y agoWhen I read this, I actually came away with the impression that he was able to accurately and eloquently express certain ideas/beliefs/concepts that I agree with and can grasp in my own mind, but I would be unable to express these same ideas coherently to another human being.
- andrewprock 11y agoUnfortunately, the questions the interviewer asks are all rather trite. On the flip side, most of the answers are rambling and barely coherent. Not the finest presentation of the field as a whole. :(
- snowwrestler 11y agoI find Horgan very frustrating as a writer. He seems to prefer trite controversy over deep understanding.
- greenrd 11y agoThis is Scientific American, a general-interest publication, not AGI Risks Weekly. In other words, most of the people reading this article will have never come across these ideas, let alone considering the questions "obvious". I'd be overjoyed if scientific american was discussing transhumanism, the singularity and AGI risks on a regular basis, and I'd be happy to be proven wrong - but I don't think that's the case.
- AndrewKemendo 11y agoI am heavily biased as I was active on Overcoming Bias (Irony?) and then later the Less Wrong forums, but every interaction I have had over the years with Yudkowski or his followers has been insufferable. Beyond just the pathos of that, I take major issue with the lack of technical depth of MIRI and the rest of these "futurist" prognosticating semi-doomsayers with respect to current AI state of the art, or even proposals for actual roadmaps to AGI. Their Step 1: Create AGI; Step 2: ??; Step 3: Doomsday reasoning chain has led to what is in my mind the silly concept that AGI can be built to be "safe" to humanity. This leads to false hope and to things like the AI Safety Pledge aka "Research Priorities for Robust and Beneficial Artificial Intelligence"[1] - for which admittedly I am a signatory largely cause my research adviser did and it was the thing to do at the time. That said, I thought his response to how he differs from Kurzweil - moreso the conclusions he comes to - are pretty spot on. [1] http://futureoflife.org/ai-open-letter/ http://futureoflife.org/ai-open-letter/
- acqq 11y agoMy impression is, there's no real "hard" substance behind EY, and he doesn't even try to achieve that, being satisfied with being "influental" even as is.
- AndrewKemendo 11y agoHe's stated as much through the years - noting his lack of applicable development or project management skills. I don't think that's a hit on it's own, but without someone like a Ng/LeCun/Bengio the organization lacks a critical link with reality.
- greenrd 11y agoI'm not sure there's anything wrong with that - unlike one of my fellow RationalWiki editors who literally criticised effective altruists for (gasp) paying other people to do stuff for them, which is like the foundation of capitalism and the modern economy (it's also known as division of labour). Does the CEO of a software company have to know how to code? Does it matter? Maybe the CTO should - maybe - but the CEO?
- 11y ago
- saint_fiasco 11y ago>I'd try to do all the things smart economists have been yelling about for a while but that almost no country ever does. Yudkowsky then mentions NGDP level targeting, consumption taxes, land value tax, negative wage taxes, Singapore's healthcare and Estonia's e-government. Is there an economics textbook or article that explains in layman's terms what those things are and why they are good ideas?
- seiji 11y agoNGDP level targeting says during economic downturns, the government should fill the gap between the downturn and the previous economic output level (creating jobs, buying things, paying people to do things, etc). Basically, don't let an economic downturn interrupt people's lives—always give them something to do and opportunities to grow. land value tax comes from the idea nobody should be able to "own" land, you should rent land. When there's a more profitable use for your land than you are current exploiting, you must give the land up. The UK enjoys things like 99 year leases on land instead of in the US where you "buy" land and own it until the heat death of the universe. negative wage taxes is like basic income if you make the "negative wage limit" really high. The government pays you because you don't make enough (maybe for reasons outside your control, like all jobs you are qualified for are now done by robots). Singapore has free computerized centralized healthcare that doesn't cost a million dollars a person-year like in the US. Estonia lets you get something resembling a "mini passport" with no international recognition (perhaps some cross-EU identity recognition, but no residency benefits), but with legal ties to an Estonian "e-residency" so you can verify your identity online electronically (chipped smart cards verified by government records, etc).
- LesZedCB 11y ago> land value tax comes from the idea nobody should be able to "own" land, you should rent land. When there's a more profitable use for your land than you are current exploiting, you must give the land up. The UK enjoys things like 99 year leases on land instead of in the US where you "buy" land and own it until the heat death of the universe. That sounds like the scariest solution to the problem of land ownership. There are plenty of cases where the government decides that somebody must give their land up to expropriate more value from it with Eminient Domain laws. Who gets to decide when land is not extracting enough value? Politicians who are bought out by private corporations controlled by a board of directors, none of whom live anywhere near the land being expropriated? Land should be collectively owned by the people living on it and producing from it.
- mmagin 11y agoI'll just leave this here: http://rationalwiki.org/wiki/Roko's_basilisk http://rationalwiki.org/wiki/Roko's_basilisk
- maaku 11y agoDude that was years ago. Give it up.
- basch 11y ago>I don't think that humans and machines "merging" is a likely source for the first superhuman intelligences. I have to disagree with him a little bit. I think "merging" is already happening and will continue to accelerate. AI/Human feedback loops are easy to conceptualize. AI does its own thinking, when it is unsure it consults an array of humans for their opinion. Repeat ad infinitum. Maybe a second array of humans proofread AI decisions, watching for conclusions they disagree with. It might not be a matrix plug in the back of your head, but data centers and human-arrays stuck in a feedback-loop, (communicating bidirectionally with screens, cameras, and eye movements) probably offer a scenario that out performs either the machines or humans in isolation. Each augments the others limitations. Are machines faster by themselves, or with a human co-processor to consult at their discretion? Can a machine go "hey im not great at this task yet, but humans seem to accelerate at it"?
- pinouchon 11y agoWhen Ray Kurzweil talks about merging with the machine, he talks about it in the literal sense (as nanobots in your body, or uploading your mind to a computer). You seem to view the merging in a broader sense, which I agree is happening.
- basch 11y agohttp://abstrusegoose.com/171 http://abstrusegoose.com/171 http://blog.dilbert.com/post/102627914061/dilbert-pocket http://blog.dilbert.com/post/102627914061/dilbert-pocket (the philip k dick in me thinks, if we merge, the machines would absorb us, not us them into our blood. the borg > cyborgs)
- basch 11y agonot sure how i said accelerate and not excel. brain glitched.
- greenrd 11y agoIt's probably not a patch on what a dedicated AGI could achieve. Read Nick Bostrom's book Superintelligence for some in-depth arguments on this.