8 ms·

Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. Back in the day on the SL4 and singularitarian mailing lists, it was spoken o

by thom 2y ago

Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. Back in the day on the SL4 and singularitarian mailing lists, it was spoken of in reverent tones, and I’m sure I remember a much younger Eliezer Yudkowsky cautioning that Doug Lenat should have perceived a non-zero chance of hard takeoff at the moment of its birth. I suspect its achievements were slightly overblown and heavily guided by a human hand, but it’s still fascinating and definitely worthy of study. Genetic programming hasn’t yielded many interesting results since, and the unreasonably effectiveness of differentiable programming and backpropagation has sucked up much of the oxygen in the room. But not everything is differentiable, the combination of the two still seems worth investigating, and EURISKO goes to show the power of heuristic approaches to some problems.

cabalamat 2y ago
> Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. I agree. > I suspect its achievements were slightly overblown and heavily guided by a human hand So do I. We'll find out how much of its performance was real, and how much bullshit. > the unreasonably effectiveness of differentiable programming and backpropagation has sucked up much of the oxygen in the room The Bitter Lesson -- http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Animats 2y ago
Not really. Read [1], which references "Why AM and Eurisko appear to work". There's a reason that line of development did not continue. [1] https://news.ycombinator.com/item?id=28343118 https://news.ycombinator.com/item?id=28343118
radomir_cernoch 2y ago
> Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. I'm really baffled by such statement and genuinely curious. How come that studying GOFAI as undergraduate and graduate at many European universities, doing a PhD. and working in the field for several years _never_ exposed me to EURISKO up until last week (thanks to HN)? I heard about Cyc, many formalism and algorithms that related to EURISKO, but never heard of its name. Is EURISKO famous in US only?
- radomir_cernoch 2y ago
  For that reason, a comparison between GPT 2 and EURISKO seems funny to me. I discussed ChatGPT with my yoga teacher recently, but I bet not even my IT colleagues would have a clue about EURISKO. :-)
  - Phiwise_ 2y ago
    So? There's a real possibility DART has still saved its customers more money over its lifetime than GPT has, and odds are basically 100% that your yoga teacher and IT colleagues haven't heard a thing about it either. The general public has all sorts of wrong impressions and unknown unknowns of facts that I don't see why they should ever be used as a technology industry benchmark by anyone not working in the UI department of a smartphone vendor.
- rjsw 2y ago
  > Is EURISKO famous in US only? It was featured in a BBC radio series on AI made by Colin Blakemore [1] around 1980, the papers on AM and EURISKO were in the library of the UK university that I attended. [1] https://en.wikipedia.org/wiki/Colin_Blakemore#Public_engagement_and_public_service https://en.wikipedia.org/wiki/Colin_Blakemore#Public_engagem...
craigus 2y ago
"... I’m sure I remember a much younger Eliezer Yudkowsky cautioning that Doug Lenat should have perceived a non-zero chance of hard takeoff at the moment of its birth." https://www.lesswrong.com/posts/rJLviHqJMTy8WQkow/recursion-magic https://www.lesswrong.com/posts/rJLviHqJMTy8WQkow/recursion-...
- cabalamat 2y ago
  Also, in 2009 someone suggested re-implementing Eurisko[1], and Yudkowsky cautioned against it: > This is a road that does not lead to Friendly AI, only to AGI. I doubt this has anything to do with Lenat's motives - but I'm glad the source code isn't published and I don't think you'd be doing a service to the human species by trying to reimplement it. To my mind -- and maybe this is just the benefit of hindsight -- this seems way too overcautious on Yudkowsky's part. [1]: https://www.lesswrong.com/posts/t47TeAbBYxYgqDGQT/let-s-reimplement-eurisko https://www.lesswrong.com/posts/t47TeAbBYxYgqDGQT/let-s-reim...
  - randallsquared 2y ago
    Machinery can be a lot simpler than biology. Birds are incredibly complex systems: wing structure, musculature, feathers, etc. An airplane can be a vaguely wing-shaped piece of metal and a pulse jet. It doesn’t seem super implausible that there is some algorithm that is to human consciousness what a pulse jet with wings is to a bird. Maybe LLMs are that, but maybe they’re far more than is really needed because we don’t yet know what we are doing. I would bet against it being possible to implement consciousness on a PDP, but I wouldn’t be very confident about it.
api 2y ago
> a much younger Eliezer Yudkowsky cautioning that Doug Lenat should have perceived a non-zero chance of hard takeoff at the moment of its birth Why is Yudkowsky taken seriously? This stuff is comparable to the "LHC micro black holes will destroy Earth" hysteria. There are actual concerns around AI like deep fakes, a deluge of un-filterable spam, mass manipulation via industrial scale propaganda, mass unemployment created by widespread automation leading to civil unrest, opaque AIs making judgements that can't be evaluated properly, AI as a means of mass appropriation of work and copyright violation, concentration of power in large AI companies, etc. The crackpot "hard takeoff" hysteria only distracts from reasonable discourse about these risks and how to mitigate them.
- TeMPOraL 2y ago
  > Why is Yudkowsky taken seriously? Trivialities Annoyances Immediate harm X-Risk |------------------------------------------------------| \----stuff you mention-------/ \---stuff Eliezer------/ wrote about > The crackpot "hard takeoff" hysteria only distracts from reasonable discourse about these risks and how to mitigate them. IDK, I feel endless hand-wringing about copyright and deepfakes distract from risks of actual, significant harm at scale, some of which you also mentioned.
  - deleted 2y ago
    [deleted]
- thom 2y ago
  Perhaps we can disagree on the shape of the curve, but it seems likely that ever more capable AI will enable ever more serious harms. Absolutely true that we should counter those harms in the present and not fixate on a theoretical future, but the medicine is much the same either way.
- adw 2y ago
  > Why is Yudkowsky taken seriously? People like religion, particularly if it doesn't affect how they live their life _today_ too much. You get all of the emotional benefits of feeling like you're doing something virtuous without the effort of actually performing good works.
lisper 2y ago
> the combination of the two still seems worth investigating This. Back in the late 1980's and early 90's the debate-du-jour was between deliberative and reactive control systems for robots. I got my Ph.D. for simply saying that the entire debate was based on the false premise that it had to be one or the other, that each approach had its strengths and weaknesses, and that if you just put the two together the whole would be greater than the sum of its parts. (Well, it was a little more than that. I had to actually show that it worked, which was more work that simply advancing the hypothesis, but in retrospect it seems kinda obvious, doesn't it?) If I were still in the game today, combining generative-AI and old-school symbolic reasoning (which has also advanced a lot in 30 years) would be the first thing I would focus my attention (!) on.
- adw 2y ago
  People have advanced that argument a lot, and it's often worked for a short while; then the statistical models get better. Chess was a game for humans. It was very briefly a game for humans and machines (Kasparov had a go at getting "Advanced Chess" off the ground as a competitive sport), but soon enough having a human in the team made the program worse. But at least the evaluation functions were designed by humans, right? That lasted a remarkably long time; first Stockfish became the strongest engine in the world by using distributed hyperparameter search to tweak its piece-square tables, then AlphaZero came along and used a policy network + MCTS instead of alpha-beta search, then (with an assist from the Shogi community) Stockfish struck back with a completely learned evaluation function via NNUE. So the last frontier of human expertise in chess is search heuristics, and that's going to fall too: https://arxiv.org/abs/2402.04494 https://arxiv.org/abs/2402.04494. The common theme with all of this is that the stuff which we used before are, fundamentally, hacks to get around _not having enough compute_, but which make the system worse once you don't have to make those tradeoffs around inductive biases. Empirical evidence suggests that raw scaling has a long way to run yet.
  - YeGoblynQueenne 2y ago
    That's the "bitter lesson", right? Which is really a sour lesson- as in sour grapes. See, Rich Sutton's point with his Bitter Lesson is that encoding expert knowledge only improves performance temporarily, which is eventually surpassed by more data and compute. There are only two problems with this: One, statistical machine learning systems have an extremely limited ability to encode expert knowledge. The language of continuous functions is alien to most humans and it's very difficult to encode one's intuitive, common sense knowledge into a system using that language [1]. That's what I mean when I say "sour grapes". Statistical machine learning folks can't use expert knowledge very well, so they pretend it's not needed. Two, all the loud successes of statistical machine learning in the last couple of decades are closely tied to minutely specialised neural net architectures: CNNs for image classification, LSTMs for translation, Transformers for language, Difussion models and Ganns for image generation. If that's not encoding knowledge of a domain, what is? Three, because of course three, despite point number two, performance keeps increasing only as data and compute increases. That's because the minutely specialised architectures in point number two are inefficient as all hell; the result of not having a good way to encode expert knowledge. Statistical machine learning folk make a virtue out of necessity and pretend that only being able to increase performance by increasing resources is some kind of achievement, whereas it's exactly the opposite: it is a clear demonstration that the capabilities of systems are not improving [2]. If capabilities were improving, we should see the number of examples required to train a state-of-the-art system either staying the same, or going down. Well, it ain't. Of course the neural net [community] will complain that their systems have reached heights never before seen in classical AI, but that's an argument that can only be sustained by the ignorance of the continued progress in all the classical AI subjects such as planning and scheduling, SAT solving, verification, automated theorem proving and so on. For example, and since planning is high on my priorities these days, see this video where the latest achievements in planning are discussed (from 2017). https://youtu.be/g3lc8BxTPiU?si=LjoFITSI5sfRFjZI https://youtu.be/g3lc8BxTPiU?si=LjoFITSI5sfRFjZI See particularly around this point where he starts talking about the Rollout IW(1) symbolic planning algorithm that plays Atari from screen pixels with performance comparable to Deep-RL; except it does so online (i.e. no training, just reasoning on the fly): https://youtu.be/g3lc8BxTPiU?si=33XSM6yK9hOlZJnf&t=1387 https://youtu.be/g3lc8BxTPiU?si=33XSM6yK9hOlZJnf&t=1387 Bitter lesson my sweet little ass. ____________ [1] Gotta find where this paper was but none other than Vladimir Vapnik basically demonstrated this by trying the maddest experiment I've ever seen in machine learning: using poetry to improve a vision classifier. It didn't work. He's spent the last 20 years trying to find a good way to encode human knowledge into continuous functions. It doesn't work. [2] In particular their capability for inductive generalisation which remains absolutely crap.

by thom 2y ago

cabalamat 2y ago

> Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. I agree. > I suspect its achievements were slightly overblown and heavily guided by a human hand So do I. We'll find out how much of its performance was real, and how much bullshit. > the unreasonably effectiveness of differentiable programming and backpropagation has sucked up much of the oxygen in the room The Bitter Lesson -- http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Animats 2y ago

Not really. Read [1], which references "Why AM and Eurisko appear to work". There's a reason that line of development did not continue. [1] https://news.ycombinator.com/item?id=28343118 https://news.ycombinator.com/item?id=28343118

radomir_cernoch 2y ago

> Up until about GPT 2, EURISKO was arguably the most interesting achievement in AI. I'm really baffled by such statement and genuinely curious. How come that studying GOFAI as undergraduate and graduate at many European universities, doing a PhD. and working in the field for several years _never_ exposed me to EURISKO up until last week (thanks to HN)? I heard about Cyc, many formalism and algorithms that related to EURISKO, but never heard of its name. Is EURISKO famous in US only?

radomir_cernoch 2y ago
For that reason, a comparison between GPT 2 and EURISKO seems funny to me. I discussed ChatGPT with my yoga teacher recently, but I bet not even my IT colleagues would have a clue about EURISKO. :-)
- Phiwise_ 2y ago
  So? There's a real possibility DART has still saved its customers more money over its lifetime than GPT has, and odds are basically 100% that your yoga teacher and IT colleagues haven't heard a thing about it either. The general public has all sorts of wrong impressions and unknown unknowns of facts that I don't see why they should ever be used as a technology industry benchmark by anyone not working in the UI department of a smartphone vendor.
rjsw 2y ago
> Is EURISKO famous in US only? It was featured in a BBC radio series on AI made by Colin Blakemore [1] around 1980, the papers on AM and EURISKO were in the library of the UK university that I attended. [1] https://en.wikipedia.org/wiki/Colin_Blakemore#Public_engagement_and_public_service https://en.wikipedia.org/wiki/Colin_Blakemore#Public_engagem...

craigus 2y ago

"... I’m sure I remember a much younger Eliezer Yudkowsky cautioning that Doug Lenat should have perceived a non-zero chance of hard takeoff at the moment of its birth." https://www.lesswrong.com/posts/rJLviHqJMTy8WQkow/recursion-magic https://www.lesswrong.com/posts/rJLviHqJMTy8WQkow/recursion-...

cabalamat 2y ago
Also, in 2009 someone suggested re-implementing Eurisko[1], and Yudkowsky cautioned against it: > This is a road that does not lead to Friendly AI, only to AGI. I doubt this has anything to do with Lenat's motives - but I'm glad the source code isn't published and I don't think you'd be doing a service to the human species by trying to reimplement it. To my mind -- and maybe this is just the benefit of hindsight -- this seems way too overcautious on Yudkowsky's part. [1]: https://www.lesswrong.com/posts/t47TeAbBYxYgqDGQT/let-s-reimplement-eurisko https://www.lesswrong.com/posts/t47TeAbBYxYgqDGQT/let-s-reim...
- randallsquared 2y ago
  Machinery can be a lot simpler than biology. Birds are incredibly complex systems: wing structure, musculature, feathers, etc. An airplane can be a vaguely wing-shaped piece of metal and a pulse jet. It doesn’t seem super implausible that there is some algorithm that is to human consciousness what a pulse jet with wings is to a bird. Maybe LLMs are that, but maybe they’re far more than is really needed because we don’t yet know what we are doing. I would bet against it being possible to implement consciousness on a PDP, but I wouldn’t be very confident about it.

api 2y ago

> a much younger Eliezer Yudkowsky cautioning that Doug Lenat should have perceived a non-zero chance of hard takeoff at the moment of its birth Why is Yudkowsky taken seriously? This stuff is comparable to the "LHC micro black holes will destroy Earth" hysteria. There are actual concerns around AI like deep fakes, a deluge of un-filterable spam, mass manipulation via industrial scale propaganda, mass unemployment created by widespread automation leading to civil unrest, opaque AIs making judgements that can't be evaluated properly, AI as a means of mass appropriation of work and copyright violation, concentration of power in large AI companies, etc. The crackpot "hard takeoff" hysteria only distracts from reasonable discourse about these risks and how to mitigate them.

TeMPOraL 2y ago
> Why is Yudkowsky taken seriously? Trivialities Annoyances Immediate harm X-Risk |------------------------------------------------------| \----stuff you mention-------/ \---stuff Eliezer------/ wrote about > The crackpot "hard takeoff" hysteria only distracts from reasonable discourse about these risks and how to mitigate them. IDK, I feel endless hand-wringing about copyright and deepfakes distract from risks of actual, significant harm at scale, some of which you also mentioned.
- deleted 2y ago
  [deleted]
thom 2y ago
Perhaps we can disagree on the shape of the curve, but it seems likely that ever more capable AI will enable ever more serious harms. Absolutely true that we should counter those harms in the present and not fixate on a theoretical future, but the medicine is much the same either way.
adw 2y ago
> Why is Yudkowsky taken seriously? People like religion, particularly if it doesn't affect how they live their life _today_ too much. You get all of the emotional benefits of feeling like you're doing something virtuous without the effort of actually performing good works.

lisper 2y ago

> the combination of the two still seems worth investigating This. Back in the late 1980's and early 90's the debate-du-jour was between deliberative and reactive control systems for robots. I got my Ph.D. for simply saying that the entire debate was based on the false premise that it had to be one or the other, that each approach had its strengths and weaknesses, and that if you just put the two together the whole would be greater than the sum of its parts. (Well, it was a little more than that. I had to actually show that it worked, which was more work that simply advancing the hypothesis, but in retrospect it seems kinda obvious, doesn't it?) If I were still in the game today, combining generative-AI and old-school symbolic reasoning (which has also advanced a lot in 30 years) would be the first thing I would focus my attention (!) on.

adw 2y ago
People have advanced that argument a lot, and it's often worked for a short while; then the statistical models get better. Chess was a game for humans. It was very briefly a game for humans and machines (Kasparov had a go at getting "Advanced Chess" off the ground as a competitive sport), but soon enough having a human in the team made the program worse. But at least the evaluation functions were designed by humans, right? That lasted a remarkably long time; first Stockfish became the strongest engine in the world by using distributed hyperparameter search to tweak its piece-square tables, then AlphaZero came along and used a policy network + MCTS instead of alpha-beta search, then (with an assist from the Shogi community) Stockfish struck back with a completely learned evaluation function via NNUE. So the last frontier of human expertise in chess is search heuristics, and that's going to fall too: https://arxiv.org/abs/2402.04494 https://arxiv.org/abs/2402.04494. The common theme with all of this is that the stuff which we used before are, fundamentally, hacks to get around _not having enough compute_, but which make the system worse once you don't have to make those tradeoffs around inductive biases. Empirical evidence suggests that raw scaling has a long way to run yet.
- YeGoblynQueenne 2y ago
  That's the "bitter lesson", right? Which is really a sour lesson- as in sour grapes. See, Rich Sutton's point with his Bitter Lesson is that encoding expert knowledge only improves performance temporarily, which is eventually surpassed by more data and compute. There are only two problems with this: One, statistical machine learning systems have an extremely limited ability to encode expert knowledge. The language of continuous functions is alien to most humans and it's very difficult to encode one's intuitive, common sense knowledge into a system using that language [1]. That's what I mean when I say "sour grapes". Statistical machine learning folks can't use expert knowledge very well, so they pretend it's not needed. Two, all the loud successes of statistical machine learning in the last couple of decades are closely tied to minutely specialised neural net architectures: CNNs for image classification, LSTMs for translation, Transformers for language, Difussion models and Ganns for image generation. If that's not encoding knowledge of a domain, what is? Three, because of course three, despite point number two, performance keeps increasing only as data and compute increases. That's because the minutely specialised architectures in point number two are inefficient as all hell; the result of not having a good way to encode expert knowledge. Statistical machine learning folk make a virtue out of necessity and pretend that only being able to increase performance by increasing resources is some kind of achievement, whereas it's exactly the opposite: it is a clear demonstration that the capabilities of systems are not improving [2]. If capabilities were improving, we should see the number of examples required to train a state-of-the-art system either staying the same, or going down. Well, it ain't. Of course the neural net [community] will complain that their systems have reached heights never before seen in classical AI, but that's an argument that can only be sustained by the ignorance of the continued progress in all the classical AI subjects such as planning and scheduling, SAT solving, verification, automated theorem proving and so on. For example, and since planning is high on my priorities these days, see this video where the latest achievements in planning are discussed (from 2017). https://youtu.be/g3lc8BxTPiU?si=LjoFITSI5sfRFjZI https://youtu.be/g3lc8BxTPiU?si=LjoFITSI5sfRFjZI See particularly around this point where he starts talking about the Rollout IW(1) symbolic planning algorithm that plays Atari from screen pixels with performance comparable to Deep-RL; except it does so online (i.e. no training, just reasoning on the fly): https://youtu.be/g3lc8BxTPiU?si=33XSM6yK9hOlZJnf&t=1387 https://youtu.be/g3lc8BxTPiU?si=33XSM6yK9hOlZJnf&t=1387 Bitter lesson my sweet little ass. ____________ [1] Gotta find where this paper was but none other than Vladimir Vapnik basically demonstrated this by trying the maddest experiment I've ever seen in machine learning: using poetry to improve a vision classifier. It didn't work. He's spent the last 20 years trying to find a good way to encode human knowledge into continuous functions. It doesn't work. [2] In particular their capability for inductive generalisation which remains absolutely crap.