5 ms·

Let's reimplement Eurisko

21 points by kf 17y ago

jibiki 17y ago
It would be interesting to apply Eurisko-style techniques to a hopeless problem like translating natural languages or playing Arimaa. I don't think it would work, but the computer might learn things that humans don't know.
akkartik 17y ago
I want to reiterate that sources for a reimplementation of AM are available. It's a start. http://github.com/akkartik/am-utexas/tree/master http://github.com/akkartik/am-utexas/tree/master (earlier mentions: http://news.ycombinator.com/item?id=405112 http://news.ycombinator.com/item?id=405112 and http://lesswrong.com/lw/10g/lets_reimplement_eurisko/usd http://lesswrong.com/lw/10g/lets_reimplement_eurisko/usd)
sho 17y ago
What is with the neo-luddite scaremongering in the comments of that post? I was amazed by the conviction and tenacity of some of the arguers. Is there really a large movement of people opposed to AI research on "skynet" grounds? "This is a road that does not lead to Friendly AI, only to AGI. I doubt this has anything to do with Lenat's motives - but I'm glad the source code isn't published and I don't think you'd be doing a service to the human species by trying to reimplement it." Christ, what can you even say to that apart from "go back to Above Top Secret" ..
- jibiki 17y ago
  It's Eliezer's website, and friendly AI is kind of his thing. This link is long but entertaining: http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/ Here's another good one, although it doesn't seem to tell the whole story (he lost two of the next three games): http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox I'm not sure what the canonical EY defense of FAI is. Most of his articles on it seem to be very tl;dr.
  - frig 17y ago
    A wag other than myself once summarized EY's other site as "wallowing in bias", which summarization did seem apt every time I overcame that tl;dr feeling and tried to wade through; I'll happily admit the possibility that there really is something nontrivial and worthwhile going on there, but the tone and contents were at more-than-a-glance indistinguishable from what people write when they are trying to convince themselves that they do believe in their heart what their mind wants to believe it believes. As often, Dostoyevsky said it best: He was one of those idealistic beings common in Russia, who are suddenly struck by some overmastering idea which seems, as it were, to crush them at once, and sometimes for ever. They are never equal to coping with it, but put passionate faith in it, and their whole life passes afterwards, as it were, in the last agonies under the weight of the stone that has fallen upon them and half crushed them.
    - jibiki 17y ago
      I don't know about that, I think it's good that someone is looking at the risks of runaway AI and similar things. Progress relies upon a certain diversity of thought, and that means that some people (and nearly all scientists) will end up wasting their lives looking down blind alleys, and none of us knows whether EY is one of them. Your Dostoyevsky quote could well apply to Einstein, for instance, in his later years.
      - arakyd 17y ago
        There are blind alleys, and then there are ideas that make people want to psychoanalyze you. To hyperbolize a bit, but only a bit, worrying about runaway AI that has the potential to turn the universe into paperclips is about as useful as worrying about warp drives that damage spacetime. It's hard enough to create fictional technologies that require basic changes in our understanding of the laws of physics, let alone debug them before they are created, let alone debug them before we make the basic discoveries that make them possible, let alone make them provably safe before we make the basic discoveries that make them possible and before we even figure out how to prove that relatively simple programs won't crash. I know of only one science fiction author who was honest enough to spell out the fact that his hypothetical runaway AI had to make several foundational breakthroughs in physics at just the right time in order to plausibly take off faster than anyone could stop it, or become as smart and powerful as singularitarians assume is possible. We have no evidence that the laws of physics in our universe allow that sort of thing. Most super-AI fiction (and everything written about super-AI is fiction at this point) either glosses over this by tacitly assuming that the plodding nature of human intelligence is the only real bottleneck to godhood (most transhumanists), or assumes that large numbers of humans will be incredibly incompetent (most mainstream AI fiction).
- rms 17y ago
  That's Eliezer. Less Wrong is his site. He also posts here sometimes. Friendly AI is his big idea. No, there is not a large movement of people opposed to AI research on skynet grounds, Eliezer is the main advocate. I'm certainly willing to admit that Friendly AI is good and important but I think it's perfectly fine to reimplement and (recursively self-) improve Eurisko. There's always quantum immortality. Some selections from his writing: http://lesswrong.com/lw/y4/three_worlds_collide_08/ http://lesswrong.com/lw/y4/three_worlds_collide_08/ (That Alien Message, my favorite short piece by Eliezer) http://lesswrong.com/lw/y4/three_worlds_collide_08/ http://lesswrong.com/lw/y4/three_worlds_collide_08/ (Three Worlds Collide, novella length fiction) http://lesswrong.com/lw/r5/the_quantum_physics_sequence/ http://lesswrong.com/lw/r5/the_quantum_physics_sequence/ (Quantum Physics Series) I'm glad Eliezer is out there thinking the thoughts that he is thinking. Maybe by the time we're closer to AI people will start really listening to him. For now, AI research continues to crawl along.
  - randallsquared 17y ago
    There's always quantum immortality. That's not the best argument for inspiring confidence, to put it mildly.
    - rms 17y ago
      I was trying to be funny. At its best my sense of humor involves statements where you are unclear if I am being sarcastic or serious, because the statements rest somewhere in between. Kind of like my username.
  - sho 17y ago
    "That Alien Message" Thanks for the link. I read it, and it's very well written. What a great metaphor for conceptualising the speed of thought of a super-AI. That said, the essay still absolutely reeks of unfounded near-paranoia. The hypothetical alien civilisation with the power to manipulate the output of stars - or even to build a simulation of such astonishing power - must surely know the very basics of risk management. And the superintelligent "earth" seemed completely friendly, all they wanted to do was talk. They must have reasoned that "hacking" the other civilisation with self-replicating nano-machines would destroy them. Why on earth would they do that? You'd expect that the one thing an AI would not want to do is destroy unique information. Start with a great setup and an engaging story, then right at the end take a totally unjustified turn into pure worst-possible-case disaster fantasy. It really just proves nothing, except that Mr. EY does not believe risks can be managed. And really, if you're going to write the "precautionary principle" this large, we really just couldn't do anything, ever. It reminds me of those nuts arguing against the LHC. "I'm glad Eliezer is out there thinking the thoughts that he is thinking." The idea of hostile AI probably predates the notion of friendly AI! It's not a new thing and I'd argue it's always been on the table. It just has to be kept in perspective, like any risk scenario from new technology. EY takes it to the most ridiculous all-or-nothing paranoid extreme when we're not even close to the time when we need to seriously weigh up these things. There would be fantastic, concrete benefits to having even slightly better AI. Trying to retard research because of some ill-justified nightmare "what if" scenario set in the far-off speculative future is ludicrous and, arguably, morally wrong.
- gort 17y ago
  I found this article quite compelling: http://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf http://selfawaresystems.files.wordpress.com/2008/01/ai_drive... Imagine you do have an AI with a certain goal. It seems obvious that: It will resist being switched off (because if switched off it cannot fulful its goal). It will try, and likely succeed, in becoming more powerful (to better satisfy its goal). It will not want to change its goal (because it sees that changing its goal means its current goal, i.e. the one currently motivating it, will go unfulfilled).

21 points by kf 17y ago

It would be interesting to apply Eurisko-style techniques to a hopeless problem like translating natural languages or playing Arimaa. I don't think it would work, but the computer might learn things that humans don't know.

akkartik 17y ago

I want to reiterate that sources for a reimplementation of AM are available. It's a start. http://github.com/akkartik/am-utexas/tree/master http://github.com/akkartik/am-utexas/tree/master (earlier mentions: http://news.ycombinator.com/item?id=405112 http://news.ycombinator.com/item?id=405112 and http://lesswrong.com/lw/10g/lets_reimplement_eurisko/usd http://lesswrong.com/lw/10g/lets_reimplement_eurisko/usd)

sho 17y ago

What is with the neo-luddite scaremongering in the comments of that post? I was amazed by the conviction and tenacity of some of the arguers. Is there really a large movement of people opposed to AI research on "skynet" grounds? "This is a road that does not lead to Friendly AI, only to AGI. I doubt this has anything to do with Lenat's motives - but I'm glad the source code isn't published and I don't think you'd be doing a service to the human species by trying to reimplement it." Christ, what can you even say to that apart from "go back to Above Top Secret" ..

jibiki 17y ago
It's Eliezer's website, and friendly AI is kind of his thing. This link is long but entertaining: http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/ Here's another good one, although it doesn't seem to tell the whole story (he lost two of the next three games): http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox I'm not sure what the canonical EY defense of FAI is. Most of his articles on it seem to be very tl;dr.
- frig 17y ago
  A wag other than myself once summarized EY's other site as "wallowing in bias", which summarization did seem apt every time I overcame that tl;dr feeling and tried to wade through; I'll happily admit the possibility that there really is something nontrivial and worthwhile going on there, but the tone and contents were at more-than-a-glance indistinguishable from what people write when they are trying to convince themselves that they do believe in their heart what their mind wants to believe it believes. As often, Dostoyevsky said it best: He was one of those idealistic beings common in Russia, who are suddenly struck by some overmastering idea which seems, as it were, to crush them at once, and sometimes for ever. They are never equal to coping with it, but put passionate faith in it, and their whole life passes afterwards, as it were, in the last agonies under the weight of the stone that has fallen upon them and half crushed them.
  - jibiki 17y ago
    I don't know about that, I think it's good that someone is looking at the risks of runaway AI and similar things. Progress relies upon a certain diversity of thought, and that means that some people (and nearly all scientists) will end up wasting their lives looking down blind alleys, and none of us knows whether EY is one of them. Your Dostoyevsky quote could well apply to Einstein, for instance, in his later years.
    - arakyd 17y ago
      There are blind alleys, and then there are ideas that make people want to psychoanalyze you. To hyperbolize a bit, but only a bit, worrying about runaway AI that has the potential to turn the universe into paperclips is about as useful as worrying about warp drives that damage spacetime. It's hard enough to create fictional technologies that require basic changes in our understanding of the laws of physics, let alone debug them before they are created, let alone debug them before we make the basic discoveries that make them possible, let alone make them provably safe before we make the basic discoveries that make them possible and before we even figure out how to prove that relatively simple programs won't crash. I know of only one science fiction author who was honest enough to spell out the fact that his hypothetical runaway AI had to make several foundational breakthroughs in physics at just the right time in order to plausibly take off faster than anyone could stop it, or become as smart and powerful as singularitarians assume is possible. We have no evidence that the laws of physics in our universe allow that sort of thing. Most super-AI fiction (and everything written about super-AI is fiction at this point) either glosses over this by tacitly assuming that the plodding nature of human intelligence is the only real bottleneck to godhood (most transhumanists), or assumes that large numbers of humans will be incredibly incompetent (most mainstream AI fiction).
rms 17y ago
That's Eliezer. Less Wrong is his site. He also posts here sometimes. Friendly AI is his big idea. No, there is not a large movement of people opposed to AI research on skynet grounds, Eliezer is the main advocate. I'm certainly willing to admit that Friendly AI is good and important but I think it's perfectly fine to reimplement and (recursively self-) improve Eurisko. There's always quantum immortality. Some selections from his writing: http://lesswrong.com/lw/y4/three_worlds_collide_08/ http://lesswrong.com/lw/y4/three_worlds_collide_08/ (That Alien Message, my favorite short piece by Eliezer) http://lesswrong.com/lw/y4/three_worlds_collide_08/ http://lesswrong.com/lw/y4/three_worlds_collide_08/ (Three Worlds Collide, novella length fiction) http://lesswrong.com/lw/r5/the_quantum_physics_sequence/ http://lesswrong.com/lw/r5/the_quantum_physics_sequence/ (Quantum Physics Series) I'm glad Eliezer is out there thinking the thoughts that he is thinking. Maybe by the time we're closer to AI people will start really listening to him. For now, AI research continues to crawl along.
- randallsquared 17y ago
  There's always quantum immortality. That's not the best argument for inspiring confidence, to put it mildly.
  - rms 17y ago
    I was trying to be funny. At its best my sense of humor involves statements where you are unclear if I am being sarcastic or serious, because the statements rest somewhere in between. Kind of like my username.
- sho 17y ago
  "That Alien Message" Thanks for the link. I read it, and it's very well written. What a great metaphor for conceptualising the speed of thought of a super-AI. That said, the essay still absolutely reeks of unfounded near-paranoia. The hypothetical alien civilisation with the power to manipulate the output of stars - or even to build a simulation of such astonishing power - must surely know the very basics of risk management. And the superintelligent "earth" seemed completely friendly, all they wanted to do was talk. They must have reasoned that "hacking" the other civilisation with self-replicating nano-machines would destroy them. Why on earth would they do that? You'd expect that the one thing an AI would not want to do is destroy unique information. Start with a great setup and an engaging story, then right at the end take a totally unjustified turn into pure worst-possible-case disaster fantasy. It really just proves nothing, except that Mr. EY does not believe risks can be managed. And really, if you're going to write the "precautionary principle" this large, we really just couldn't do anything, ever. It reminds me of those nuts arguing against the LHC. "I'm glad Eliezer is out there thinking the thoughts that he is thinking." The idea of hostile AI probably predates the notion of friendly AI! It's not a new thing and I'd argue it's always been on the table. It just has to be kept in perspective, like any risk scenario from new technology. EY takes it to the most ridiculous all-or-nothing paranoid extreme when we're not even close to the time when we need to seriously weigh up these things. There would be fantastic, concrete benefits to having even slightly better AI. Trying to retard research because of some ill-justified nightmare "what if" scenario set in the far-off speculative future is ludicrous and, arguably, morally wrong.
gort 17y ago
I found this article quite compelling: http://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf http://selfawaresystems.files.wordpress.com/2008/01/ai_drive... Imagine you do have an AI with a certain goal. It seems obvious that: It will resist being switched off (because if switched off it cannot fulful its goal). It will try, and likely succeed, in becoming more powerful (to better satisfy its goal). It will not want to change its goal (because it sees that changing its goal means its current goal, i.e. the one currently motivating it, will go unfulfilled).