7 ms·

I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue t

I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former? Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods.

Muller20 3y ago
Depends on your application. AFAIK they are still used in games because they are easier to control and people want to play against simple AI agents.
danwee 3y ago
> Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods. Same for me. > I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former? I think it still matters. Plenty of examples in the tech industry where "old" tech/paradigms became the new "hype". They say it's all a cycle.
kqr 3y ago
The company I work for makes a killing on applying a 1980s-era ML technique to a really tough modern business problem -- the resulting product is probably the best in the world at what it's doing. Old techniques have several things going for them, with one of the more important ones for us being explainability. A random person off the street could hypothetically, with an hour or two of training, diagnose problems just by looking at the structure of the model. That's very helpful for adapting to market needs. Generality is another big plus. Since the model encodes intuitive ideas there's a lot of room for using it in innovative ways. Older techniques also tend to produce better results with less data, because big data wasn't as much of a thing back then. Unless you have a crazy amount of resources, I think it's far better to be bleeding edge in as few things as possible. Solving a new business problem? Perhaps don't spend too much time on also solving all the childhood diseases of a new technology.
- npsomaratna 3y ago
  Cost is also important. LLMs are expensive to run. I think that going forward, we'll see a mix of "normal" programming, LLMs, and simpler machine learning techniques all combined together, because of economic reasons.
  - kqr 3y ago
    Hm. I'm not sure about cost. I haven't done any sort of analysis but I would guess with 75 % certainty that the techniques we use need more watts per byte of data than modern techniques. Modern techniques are very efficient in terms of their operation -- it's just that they achieve performance by needing to do so much operation!
    - bmitc 3y ago
      I was assuming they were referring to the data collection, storage, and processing and model training.
- i_don_t_know 3y ago
  What 1980s-era ML technique are you using and why is it better suited for your application than something modern? Is it „just“ the ability to explain solutions, or is there something else like guarantees for XYZ?
  - kqr 3y ago
    The core part of it is doing dimensionality reduction with shallow neural networks, but essentially all user-facing functionality we get by controlling the training methods and hooking into the neural network and querying it all sorts of ways. We switched from more modern techniques primarily because they needed too much data to work well, but the other things I mentioned are the benefits we noted along the way. I don't know if that answers your question.
kazinator 3y ago
You can run it in an air-gapped embedded system with (by today's standards) low RAM, and a slow CPU. Being able to trace how an answer is derived is also worth something.
jstx1 3y ago
If it's for work or for building something useful, the book is a waste of time. If it's for your personal interest, you can read and learn anything you want. Personally I avoid books like this one (similar to how I avoid very esoteric languages) because I want to spend my time on things are interesting and useful instead of only interesting.
- deleted 3y ago
  [deleted]
hiepph 3y ago
With modern and trendy music, should we listen to old and classical music. Should we forget the history? I think not. My favourite genres are Jazz and classical. Sorry if you don't get the metaphor, but it's like so.
- jstx1 3y ago
  The metaphor only works if you assume that software has only aesthetic utility.
- cyberbanjo 3y ago
  Jazz is a contemporary pop music, so this doesn't even make sense
dimatura 3y ago
This book is a good intro to Lisp itself, which is worth knowing in any case- and symbolic AI is probably one of the best domains for it! As for GOFAI in the age of DL/LLM, yes, you should know it, for a couple of reasons. A lot of these techniques aren't really considered "AI" anymore, they're just regular CS algorithms everybody should know: graph search, backtracking, optimization, parsing, etc. The other is that a lot of newer DL/LLM is actually going back to these old problems, but bringing all the new techniques to deal with limitations of the classical algorithms.
oumua_don17 3y ago
But what is GOFAI? https://en.wikipedia.org/wiki/GOFAI https://en.wikipedia.org/wiki/GOFAI
bionhoward 3y ago
it all loops around, pretty soon we’ll all be back to good old fashioned philosophy
abecedarius 3y ago
Think of this as a "learn to program better" book, even if you don't care about GOFAI. It has a lot of very nice code, and commentary without blather. I've said this about as long as I knew of it, so this isn't just revisionism.
blihp 3y ago
It depends on what you were expecting from GOFAI. The future state is likely to be a combination of approaches where each makes sense rather than a single one. We're still in the honeymoon phase with deep learning and disillusionment with its ultimate limitations is likely still to come.

Muller20 3y ago

Depends on your application. AFAIK they are still used in games because they are easier to control and people want to play against simple AI agents.

danwee 3y ago

> Personally, those "old" methods in the 80s make a lot more sense to me than recent statistical methods. Same for me. > I've been keeping my eye on the so called "GOFAI" for a long time but with recent advances in ANN methods (DL, LLM), does it even make sense to further pursue the former? I think it still matters. Plenty of examples in the tech industry where "old" tech/paradigms became the new "hype". They say it's all a cycle.

kqr 3y ago

The company I work for makes a killing on applying a 1980s-era ML technique to a really tough modern business problem -- the resulting product is probably the best in the world at what it's doing. Old techniques have several things going for them, with one of the more important ones for us being explainability. A random person off the street could hypothetically, with an hour or two of training, diagnose problems just by looking at the structure of the model. That's very helpful for adapting to market needs. Generality is another big plus. Since the model encodes intuitive ideas there's a lot of room for using it in innovative ways. Older techniques also tend to produce better results with less data, because big data wasn't as much of a thing back then. Unless you have a crazy amount of resources, I think it's far better to be bleeding edge in as few things as possible. Solving a new business problem? Perhaps don't spend too much time on also solving all the childhood diseases of a new technology.

npsomaratna 3y ago
Cost is also important. LLMs are expensive to run. I think that going forward, we'll see a mix of "normal" programming, LLMs, and simpler machine learning techniques all combined together, because of economic reasons.
- kqr 3y ago
  Hm. I'm not sure about cost. I haven't done any sort of analysis but I would guess with 75 % certainty that the techniques we use need more watts per byte of data than modern techniques. Modern techniques are very efficient in terms of their operation -- it's just that they achieve performance by needing to do so much operation!
  - bmitc 3y ago
    I was assuming they were referring to the data collection, storage, and processing and model training.
i_don_t_know 3y ago
What 1980s-era ML technique are you using and why is it better suited for your application than something modern? Is it „just“ the ability to explain solutions, or is there something else like guarantees for XYZ?
- kqr 3y ago
  The core part of it is doing dimensionality reduction with shallow neural networks, but essentially all user-facing functionality we get by controlling the training methods and hooking into the neural network and querying it all sorts of ways. We switched from more modern techniques primarily because they needed too much data to work well, but the other things I mentioned are the benefits we noted along the way. I don't know if that answers your question.

kazinator 3y ago

You can run it in an air-gapped embedded system with (by today's standards) low RAM, and a slow CPU. Being able to trace how an answer is derived is also worth something.

jstx1 3y ago

If it's for work or for building something useful, the book is a waste of time. If it's for your personal interest, you can read and learn anything you want. Personally I avoid books like this one (similar to how I avoid very esoteric languages) because I want to spend my time on things are interesting and useful instead of only interesting.

deleted 3y ago
[deleted]

hiepph 3y ago

With modern and trendy music, should we listen to old and classical music. Should we forget the history? I think not. My favourite genres are Jazz and classical. Sorry if you don't get the metaphor, but it's like so.

jstx1 3y ago
The metaphor only works if you assume that software has only aesthetic utility.
cyberbanjo 3y ago
Jazz is a contemporary pop music, so this doesn't even make sense

dimatura 3y ago

This book is a good intro to Lisp itself, which is worth knowing in any case- and symbolic AI is probably one of the best domains for it! As for GOFAI in the age of DL/LLM, yes, you should know it, for a couple of reasons. A lot of these techniques aren't really considered "AI" anymore, they're just regular CS algorithms everybody should know: graph search, backtracking, optimization, parsing, etc. The other is that a lot of newer DL/LLM is actually going back to these old problems, but bringing all the new techniques to deal with limitations of the classical algorithms.

oumua_don17 3y ago

But what is GOFAI? https://en.wikipedia.org/wiki/GOFAI https://en.wikipedia.org/wiki/GOFAI

bionhoward 3y ago

it all loops around, pretty soon we’ll all be back to good old fashioned philosophy

abecedarius 3y ago

Think of this as a "learn to program better" book, even if you don't care about GOFAI. It has a lot of very nice code, and commentary without blather. I've said this about as long as I knew of it, so this isn't just revisionism.

blihp 3y ago

It depends on what you were expecting from GOFAI. The future state is likely to be a combination of approaches where each makes sense rather than a single one. We're still in the honeymoon phase with deep learning and disillusionment with its ultimate limitations is likely still to come.