6 ms·

A Retrospective on Paradigms of AI Programming (2002)

45 points by 0xpgm 4y ago

hislaziness 4y ago
If anyone is interested PAIP is downloadable at https://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp
inconfident2021 4y ago
I hate to say this but symbolic reasoning and planning has aged like milk. The use case is so limited and the only people using these are the people full of nostalgia. One paradigm shift into deep learning in 2012 and 10 years later we have something as general as GPT-4. Gatekeeping in academia is a real thing. Sad but true.
- zelphirkalt 4y ago
  Symbolic reasoning is for things we truly know how to do. It can be used as supplemental implementation for known special cases as well.
- jmeister 4y ago
  There was no gate-keeping. Yann LeCun was doing neural networks in the late 80s.
  - visarga 4y ago
    Late 80s was the second AI winter, it wasn't all peaches. Why do you think just a handful of people were working on neural nets?
    - YeGoblynQueenne 4y ago
      Because before backpropagation (mid-80's) nobody could train multi-layer neural nets. Although to be fair, there wasn't "just a handful of people". Just look at Geoff Hinton's citations count throughout the '80s.
- bmitc 4y ago
  Machine learning has been around for decades. The real breakthrough has been the massive increase in compute power, the ubiquity of distributed systems, and the amount of data available.
- vkazanov 4y ago
  Symbolic reasoning and related fields are just areas to were important to be researched back in the day. There are some very interesting results that came of that family of research, especially on the intersection of traditional math and CS, i.e. logic stuff, proof assistants, various solvers, etc. Machine learning and its flavours were always there. What is new in deep learning is the scale of everything: amount of compute made possible by clusters of GPU-centric machines and amount of data made available by the Internet. The shocking thing here how almost boring the core of recent advances is: do more of the same, much, much more, and quantity will turn into new qualities yet again.
- FrustratedMonky 4y ago
  Its amazing that everything is some conspiracy of 'they' just wouldn't let us advance. Most times these sentiments are from those that are just learning a field, and haven't realized that the 'establishment' wasn't stopping advancement, it was also advancing. Just because there is a big break through, doesn't mean that everything that came before wasn't needed to reach it.
bitwize 4y ago
Funny to see this now, when AI is done by using Python to orchestrate numeric kernels written in C++ or Fortran. GOFAI lost, big time, and the key to success in AI is the ability to perform matrix multiplications as fast as machinely possible -- which means the central data structure you should be using is the array or vector, to ensure data locality so you don't blow up your cache. Lisp is, almost, the opposite of a language you would choose to do AI in today. It may be a more powerful language for other applications, but that will hardly matter because the productivity gains from using AI will dwarf those from programming language choice.
- mhh__ 4y ago
  Python at scale is really just build a graph, doesn't really matter what language you do that in other than history
- varjag 4y ago
  Common Lisp has certainly better array and vector support than Python so this can't be it.
  - bitwize 4y ago
    That's why the numeric kernels, where all the speed-sensitive array and vector processing is done, are written in C++ or Fortran.
  - gcr 4y ago
    on the minimal chance you're not being sarcastic, could you say more about this?
    - varjag 4y ago
      Common Lisp has has native support for vectors and arrays. They can be adjustable or fixed size, displaced to other arrays (kind of a "view" with a different geometry), specialized on elements of particular type or generic. For adjustable vectors you can specify their allocation growth increments for push operations. Or pre-allocate adjustable arrays but specify a lower initial size. Arrays can be accessed in row-major order mapping them into vectors etc etc. Foreign function interface of different implementations typically provide a way to access C allocated memory areas as Lisp arrays.
lincpa 4y ago
[dead]
randcraw 4y ago
Has anyone tried to update Norvig's Lisp popularity metrics? (Though counting Usenet articles ain't what it used to be.) I'd love to see how it's aged after another 20 years.
wbeckler 4y ago
I think a good question might be, what language is better for gpt to program. If you want to write applications that accomplish intelligent things via symbolic manipulation, is lisp a good language to ask an AIto write it in? Is PAIP in the learning set for GPT4?
- profile53 4y ago
  I’d be willing to bet Java, Python, or C are the best because of how much training data was in those languages. They aren’t the best at a theoretical level but at a practical level, almost definitely.
mark_l_watson 4y ago
Peter wrote this about the same time I first met him at the Lisp Users Venders conference in San Diego. I had just had my Common Lisp book published by Springer Verlag. Fast forward a few years and he helped me a lot my reading through my Java AI book manuscript, offering a lot of good advice. A year or two after that when I had lunch with him at Google I was really surprised that he had moved on to Python. It is ironic to me, that I followed him to mostly using Python, but 20 years later! In the end, it is all about getting valuable code written and deployed, not using a particular language. That said, I still enjoy using a variety of languages, even though Python is the most practical and productive language for what I work on now.
jonjacky 4y ago
The most helpful content here is the "list of the 52 most important lessons in PAIP" near the end. Despite the title of the book, many of these lessons are not just about AI, or even particularly about symbolic programming or Lisp. They are generally useful advice about programming.

45 points by 0xpgm 4y ago

If anyone is interested PAIP is downloadable at https://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp

inconfident2021 4y ago

I hate to say this but symbolic reasoning and planning has aged like milk. The use case is so limited and the only people using these are the people full of nostalgia. One paradigm shift into deep learning in 2012 and 10 years later we have something as general as GPT-4. Gatekeeping in academia is a real thing. Sad but true.

zelphirkalt 4y ago
Symbolic reasoning is for things we truly know how to do. It can be used as supplemental implementation for known special cases as well.
jmeister 4y ago
There was no gate-keeping. Yann LeCun was doing neural networks in the late 80s.
- visarga 4y ago
  Late 80s was the second AI winter, it wasn't all peaches. Why do you think just a handful of people were working on neural nets?
  - YeGoblynQueenne 4y ago
    Because before backpropagation (mid-80's) nobody could train multi-layer neural nets. Although to be fair, there wasn't "just a handful of people". Just look at Geoff Hinton's citations count throughout the '80s.
bmitc 4y ago
Machine learning has been around for decades. The real breakthrough has been the massive increase in compute power, the ubiquity of distributed systems, and the amount of data available.
vkazanov 4y ago
Symbolic reasoning and related fields are just areas to were important to be researched back in the day. There are some very interesting results that came of that family of research, especially on the intersection of traditional math and CS, i.e. logic stuff, proof assistants, various solvers, etc. Machine learning and its flavours were always there. What is new in deep learning is the scale of everything: amount of compute made possible by clusters of GPU-centric machines and amount of data made available by the Internet. The shocking thing here how almost boring the core of recent advances is: do more of the same, much, much more, and quantity will turn into new qualities yet again.
FrustratedMonky 4y ago
Its amazing that everything is some conspiracy of 'they' just wouldn't let us advance. Most times these sentiments are from those that are just learning a field, and haven't realized that the 'establishment' wasn't stopping advancement, it was also advancing. Just because there is a big break through, doesn't mean that everything that came before wasn't needed to reach it.

bitwize 4y ago

Funny to see this now, when AI is done by using Python to orchestrate numeric kernels written in C++ or Fortran. GOFAI lost, big time, and the key to success in AI is the ability to perform matrix multiplications as fast as machinely possible -- which means the central data structure you should be using is the array or vector, to ensure data locality so you don't blow up your cache. Lisp is, almost, the opposite of a language you would choose to do AI in today. It may be a more powerful language for other applications, but that will hardly matter because the productivity gains from using AI will dwarf those from programming language choice.

mhh__ 4y ago
Python at scale is really just build a graph, doesn't really matter what language you do that in other than history
varjag 4y ago
Common Lisp has certainly better array and vector support than Python so this can't be it.
- bitwize 4y ago
  That's why the numeric kernels, where all the speed-sensitive array and vector processing is done, are written in C++ or Fortran.
- gcr 4y ago
  on the minimal chance you're not being sarcastic, could you say more about this?
  - varjag 4y ago
    Common Lisp has has native support for vectors and arrays. They can be adjustable or fixed size, displaced to other arrays (kind of a "view" with a different geometry), specialized on elements of particular type or generic. For adjustable vectors you can specify their allocation growth increments for push operations. Or pre-allocate adjustable arrays but specify a lower initial size. Arrays can be accessed in row-major order mapping them into vectors etc etc. Foreign function interface of different implementations typically provide a way to access C allocated memory areas as Lisp arrays.

lincpa 4y ago

[dead]

randcraw 4y ago

Has anyone tried to update Norvig's Lisp popularity metrics? (Though counting Usenet articles ain't what it used to be.) I'd love to see how it's aged after another 20 years.

wbeckler 4y ago

I think a good question might be, what language is better for gpt to program. If you want to write applications that accomplish intelligent things via symbolic manipulation, is lisp a good language to ask an AIto write it in? Is PAIP in the learning set for GPT4?

profile53 4y ago
I’d be willing to bet Java, Python, or C are the best because of how much training data was in those languages. They aren’t the best at a theoretical level but at a practical level, almost definitely.

mark_l_watson 4y ago

Peter wrote this about the same time I first met him at the Lisp Users Venders conference in San Diego. I had just had my Common Lisp book published by Springer Verlag. Fast forward a few years and he helped me a lot my reading through my Java AI book manuscript, offering a lot of good advice. A year or two after that when I had lunch with him at Google I was really surprised that he had moved on to Python. It is ironic to me, that I followed him to mostly using Python, but 20 years later! In the end, it is all about getting valuable code written and deployed, not using a particular language. That said, I still enjoy using a variety of languages, even though Python is the most practical and productive language for what I work on now.

jonjacky 4y ago

The most helpful content here is the "list of the 52 most important lessons in PAIP" near the end. Despite the title of the book, many of these lessons are not just about AI, or even particularly about symbolic programming or Lisp. They are generally useful advice about programming.