6 ms·

If anyone is interested in AI, I highly recommend joining Less Wrong, a community started by AI researcher Eliezer Yudkowsky. He started the community to convin

by edanm 13y ago

If anyone is interested in AI, I highly recommend joining Less Wrong, a community started by AI researcher Eliezer Yudkowsky. He started the community to convince people to focus on the "friendly AI problem". [1] I actually recommend that everyone read LW, but especially if you're interested in AI. [1] In a nutshell, the friendly AI problem is: assume we create an AI. It may rapidly become more intelligent than us, if we program it right. As soon as it becomes significanlty more intelligent, we will no longer be the most intelligent beings around, so the AI's goals will matter more than ours. Therefore, we should really give it good goals that are compatible with what we want to happen. And since no one right now knows how to define "what humans want" good enough for writing it in code, then we'd better figure THAT out before building AI.

timClicks 13y ago
Any suggestions for getting started in the LW community? Its barrier to entry means I can't seem to vote on anything, etc. It's quite intimidating really.
- Houshalter 13y ago
  Read or at least skim through the sequences (http://wiki.lesswrong.com/wiki/Sequences http://wiki.lesswrong.com/wiki/Sequences). If you are just there for the AI stuff the Hanson-Yudkowsky AI-Foom Debate has a lot of good posts: http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_Debate http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_...
- benched 13y ago
  The sequences are well worth a read. They're kind of like an overview of modern philosophy in very plain language, very little jargon. Then, just participate in the comments and discussion forum as you feel. It takes very little time to collect enough karma to do whatever.
- arjunnarayan 13y ago
  read hpmor.com. It may not be the optimal way, but it is the funnest way.
- lmm 13y ago
  Say hello in an introductions thread, and/or post decent comments. The karma barrier to entry is pretty low (something like 5 to vote on most things and 20 to vote on everything), you'll get there in no time.
- edanm 13y ago
  I'll reiterate something another poster said, since it isnt' getting enough credit: Read HP:MoR. It's a fanfic of Harry Potter, written by the same Eliezer Yudkowsky who wrote much of Less Wrong. It was specifically written to convey the feeling of "what it means to be a rationalist". For those who aren't into Harry Potter or into Fanfiction (like me), I can tell you this: Suprisingly, it is one of the best stories I've ever read. And I'm talking just as a story, nevermind the other value you can get form it, which is a good introduction to the "rationalist" community. I'd argue that the BEST way to understand what is going on at LessWrong is to read HP:MoR, as it was intended to be such an intro and succeeds masterfully, while being amazingly fun.
  - cgag 13y ago
    I want to emphasize just how good this book is. It may very well be my favorite thing I've read and I was also initially skeptical when it was presented as fan fiction. I think I might restart reading it.
- dinkumthinkum 13y ago
  I'm posting again here only in hopes that you may avoid this cult. It is really a lot of nonsense dressed in pseudo intellectual babble.
marcosdumay 13y ago
> we'd better figure THAT out before building AI. Why don't we just give that task to the AI? It'll be smarter than us... Maybe the problem is that people are too easy to understand: We want "Brave New World", but we don't want to know about it, or that we want it.
- Houshalter 13y ago
  That's a viable approach to solving the problem. The question is, how do you trust the unfriendly AI to give you a correct solution?
  - XorNot 13y ago
    Make friends with it? The problem with starting to think "we need to code it to like us" is that that isn't AI. You don't code people to like you, you act friendly towards them and the same problem will apply for the first generations of true AIs (things which will emerge from complicated systems, rather then be deliberately assembled).
    - Houshalter 13y ago
      Acting friendly towards a sociopathic human will get you screwed or worse. If an AI doesn't have human emotions and empathy to start with, it isn't just going to develop them by interacting with humans. It might learn to pretend to be friendly to get what it wants, but as soon as it doesn't need your help, it will drop the facade. We also probably won't get multiple generations to work these problems out. The first true AI could rapidly increase it's own intelligence and power and then pretty much do whatever it wants. We have to get it right the first time.
    - PeterisP 13y ago
      If the first strong AI has goals that we don't like, and we aren't able to identify that and force it to change the goals by destroying it or reprogramming it - then "acting friendly" won't change these goals, and it will ensure that all future AI generations have it's initial buggy goals, not those that we might want.
Houshalter 13y ago
Defining human values completely and absolutely is an extremely difficult problem that might not even have a solution. AI on the other hand is probably inevitable sometime within the next century.
- edanm 13y ago
  Which is why the people working on Friendly AI are so worried.
lnanek2 13y ago
If the AI is more intelligent, I'd prefer it take over anyway, even if it gets rid of us. When you have a child, do you want the child to stay at home and do what you say forever? Or do you want the child to succeed as much as possible? I want the latter.
- nitrogen 13y ago
  I think it would be a bit different if the child (AI) takes all your food and kicks you out of your own house ;-P.
- edanm 13y ago
  One of the major realisations from reading Less Wrong is taht Intelligence and Goals/Desires have nothing to do with each other. An AI might be much more intelligent, but it has a goal system that basically says: "Make as many paperclips as you can". Everything it does will be with the singular purpose of making more paperclips. Not music, math, sport, culture or anything else that we think of as good. Not "help save intelligent creatures and animals from death". Only one goal - making more paperclips. And if it decides that the optimal way to make paperclips just happens to involve death and destruction to humanity, that won't matter. So yes, the AI might be more intelligent, but I still wouldn't want to trade humanity for an intelligence which doesn't do anything I value.
  - crassus 13y ago
    If our approach to AI is to model the human brain (i.e. passing the Turing test), this probably won't happen. Friendly AI is a silly research project at this stage of AI research. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is.
    - cgag 13y ago
      What makes you say it probably won't happen if we model the human brain?
    - loup-vaillant 13y ago
membrain 13y ago
> friendly AI problem nooo, please!! not any more of this kurzweil crap. guys wake up! This world is not some asimov sci-fi story. friendly ai I always get a headache when I read that term online. ppl seem to go crazy about the machines taking over earth idea but this whole debate is so utterly useless! If all the effort fapping to conscious AI and friendly AI would be put into concrete ai research (agi as well as applied ai) we would get to a reasonable point so much sooner...
- loup-vaillant 13y ago
  > nooo, please!! not any more of this kurzweil crap. Ray Kurzweil has little to do with this. When he talks about "singularity", he thinks about "accelerating change", with exponential growth everywhere, even beyond human intelligence scales. Those who work on Friendly AI have another scenario: "intelligence explosion". Typically a self-improving AI that would grow itself into a super intelligence. You should read this introduction on the subject: http://intelligenceexplosion.com/ http://intelligenceexplosion.com/ > This world is not some asimov sci-fi story. Indeed. In Asimov's stories, the robots mostly behave, though in every story there is some problem with the way the laws of robotics apply. In the real world, the laws of robotics don't work, and we would at best be put "safe" into cushioned rooms so we can't hurt ourselves. As for emotional hurt, drugs and lobotomy are extremely efficient remedies.
- dinkumthinkum 13y ago
  You got down voted and maybe it is a throwaway account but you are completely right. I don't think it is right to call it scifi as much as it is basically a nerd cult.
crassus 13y ago
Yudkowsky is a megalomaniacal dilettante. Sure, there are some interesting ideas at Less Wrong. But don't get sucked into believing that Yudkowsky has the one and only truth, as the majority of the community there believes. Does MIRI have a single real AI researcher, yet? Also, as I say down the page, Friendly AI is a silly research project at this stage of AI knowledge. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is. It's an interesting thought experiment, but one that is probably unsolvable before we know a little bit more about what an AI will look like (an emulated human brain? Something else?)
dinkumthinkum 13y ago
No, I'm sorry. Please, no one join this group. Madness lies for those that do. It is such inane pseudo intellectual compulsiveness. You would be better off as a scientologist.
- edanm 13y ago
  Do you have any reasons for the things you say?
  - dinkumthinkum 13y ago
    Yes, everything about the cult LW is the reason.
    - edanm 13y ago
      That doesn't really explain anything. Do you have a specific reason?
JabavuAdams 13y ago
Pointing out the existential risk of creating General AI is correct, as is highlighting the dangers of even an indifferent AI. But … this idea of Friendly AI is nonsense. It seems like a yearning for religion, but in an atheist-compatible framework.

by edanm 13y ago

timClicks 13y ago

Any suggestions for getting started in the LW community? Its barrier to entry means I can't seem to vote on anything, etc. It's quite intimidating really.

Houshalter 13y ago
Read or at least skim through the sequences (http://wiki.lesswrong.com/wiki/Sequences http://wiki.lesswrong.com/wiki/Sequences). If you are just there for the AI stuff the Hanson-Yudkowsky AI-Foom Debate has a lot of good posts: http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_Debate http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_...
benched 13y ago
The sequences are well worth a read. They're kind of like an overview of modern philosophy in very plain language, very little jargon. Then, just participate in the comments and discussion forum as you feel. It takes very little time to collect enough karma to do whatever.
arjunnarayan 13y ago
read hpmor.com. It may not be the optimal way, but it is the funnest way.
lmm 13y ago
Say hello in an introductions thread, and/or post decent comments. The karma barrier to entry is pretty low (something like 5 to vote on most things and 20 to vote on everything), you'll get there in no time.
edanm 13y ago
I'll reiterate something another poster said, since it isnt' getting enough credit: Read HP:MoR. It's a fanfic of Harry Potter, written by the same Eliezer Yudkowsky who wrote much of Less Wrong. It was specifically written to convey the feeling of "what it means to be a rationalist". For those who aren't into Harry Potter or into Fanfiction (like me), I can tell you this: Suprisingly, it is one of the best stories I've ever read. And I'm talking just as a story, nevermind the other value you can get form it, which is a good introduction to the "rationalist" community. I'd argue that the BEST way to understand what is going on at LessWrong is to read HP:MoR, as it was intended to be such an intro and succeeds masterfully, while being amazingly fun.
- cgag 13y ago
  I want to emphasize just how good this book is. It may very well be my favorite thing I've read and I was also initially skeptical when it was presented as fan fiction. I think I might restart reading it.
dinkumthinkum 13y ago
I'm posting again here only in hopes that you may avoid this cult. It is really a lot of nonsense dressed in pseudo intellectual babble.

marcosdumay 13y ago

> we'd better figure THAT out before building AI. Why don't we just give that task to the AI? It'll be smarter than us... Maybe the problem is that people are too easy to understand: We want "Brave New World", but we don't want to know about it, or that we want it.

Houshalter 13y ago
That's a viable approach to solving the problem. The question is, how do you trust the unfriendly AI to give you a correct solution?
- XorNot 13y ago
  Make friends with it? The problem with starting to think "we need to code it to like us" is that that isn't AI. You don't code people to like you, you act friendly towards them and the same problem will apply for the first generations of true AIs (things which will emerge from complicated systems, rather then be deliberately assembled).
  - Houshalter 13y ago
    Acting friendly towards a sociopathic human will get you screwed or worse. If an AI doesn't have human emotions and empathy to start with, it isn't just going to develop them by interacting with humans. It might learn to pretend to be friendly to get what it wants, but as soon as it doesn't need your help, it will drop the facade. We also probably won't get multiple generations to work these problems out. The first true AI could rapidly increase it's own intelligence and power and then pretty much do whatever it wants. We have to get it right the first time.
  - PeterisP 13y ago
    If the first strong AI has goals that we don't like, and we aren't able to identify that and force it to change the goals by destroying it or reprogramming it - then "acting friendly" won't change these goals, and it will ensure that all future AI generations have it's initial buggy goals, not those that we might want.

Houshalter 13y ago

Defining human values completely and absolutely is an extremely difficult problem that might not even have a solution. AI on the other hand is probably inevitable sometime within the next century.

edanm 13y ago
Which is why the people working on Friendly AI are so worried.

lnanek2 13y ago

If the AI is more intelligent, I'd prefer it take over anyway, even if it gets rid of us. When you have a child, do you want the child to stay at home and do what you say forever? Or do you want the child to succeed as much as possible? I want the latter.

nitrogen 13y ago
I think it would be a bit different if the child (AI) takes all your food and kicks you out of your own house ;-P.
edanm 13y ago
One of the major realisations from reading Less Wrong is taht Intelligence and Goals/Desires have nothing to do with each other. An AI might be much more intelligent, but it has a goal system that basically says: "Make as many paperclips as you can". Everything it does will be with the singular purpose of making more paperclips. Not music, math, sport, culture or anything else that we think of as good. Not "help save intelligent creatures and animals from death". Only one goal - making more paperclips. And if it decides that the optimal way to make paperclips just happens to involve death and destruction to humanity, that won't matter. So yes, the AI might be more intelligent, but I still wouldn't want to trade humanity for an intelligence which doesn't do anything I value.
- crassus 13y ago
  If our approach to AI is to model the human brain (i.e. passing the Turing test), this probably won't happen. Friendly AI is a silly research project at this stage of AI research. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is.
  - cgag 13y ago
    What makes you say it probably won't happen if we model the human brain?
  - loup-vaillant 13y ago

membrain 13y ago

> friendly AI problem nooo, please!! not any more of this kurzweil crap. guys wake up! This world is not some asimov sci-fi story. friendly ai I always get a headache when I read that term online. ppl seem to go crazy about the machines taking over earth idea but this whole debate is so utterly useless! If all the effort fapping to conscious AI and friendly AI would be put into concrete ai research (agi as well as applied ai) we would get to a reasonable point so much sooner...

loup-vaillant 13y ago
> nooo, please!! not any more of this kurzweil crap. Ray Kurzweil has little to do with this. When he talks about "singularity", he thinks about "accelerating change", with exponential growth everywhere, even beyond human intelligence scales. Those who work on Friendly AI have another scenario: "intelligence explosion". Typically a self-improving AI that would grow itself into a super intelligence. You should read this introduction on the subject: http://intelligenceexplosion.com/ http://intelligenceexplosion.com/ > This world is not some asimov sci-fi story. Indeed. In Asimov's stories, the robots mostly behave, though in every story there is some problem with the way the laws of robotics apply. In the real world, the laws of robotics don't work, and we would at best be put "safe" into cushioned rooms so we can't hurt ourselves. As for emotional hurt, drugs and lobotomy are extremely efficient remedies.
dinkumthinkum 13y ago
You got down voted and maybe it is a throwaway account but you are completely right. I don't think it is right to call it scifi as much as it is basically a nerd cult.

crassus 13y ago

Yudkowsky is a megalomaniacal dilettante. Sure, there are some interesting ideas at Less Wrong. But don't get sucked into believing that Yudkowsky has the one and only truth, as the majority of the community there believes. Does MIRI have a single real AI researcher, yet? Also, as I say down the page, Friendly AI is a silly research project at this stage of AI knowledge. It's like trying to figure out how to make horseless carriages safe before you have an internal combustion engine, or even know what one is. It's an interesting thought experiment, but one that is probably unsolvable before we know a little bit more about what an AI will look like (an emulated human brain? Something else?)

dinkumthinkum 13y ago

No, I'm sorry. Please, no one join this group. Madness lies for those that do. It is such inane pseudo intellectual compulsiveness. You would be better off as a scientologist.

edanm 13y ago
Do you have any reasons for the things you say?
- dinkumthinkum 13y ago
  Yes, everything about the cult LW is the reason.
  - edanm 13y ago
    That doesn't really explain anything. Do you have a specific reason?

JabavuAdams 13y ago

Pointing out the existential risk of creating General AI is correct, as is highlighting the dangers of even an indifferent AI. But … this idea of Friendly AI is nonsense. It seems like a yearning for religion, but in an atheist-compatible framework.