5 ms·

Other than Minsky, I don’t think others (who are nevertheless scientists in their respective fields) are considered to have made significant contributions to mo

by aborsy 3y ago

Other than Minsky, I don’t think others (who are nevertheless scientists in their respective fields) are considered to have made significant contributions to modern machine learning or AI. McCarthy’s work around this topic culminated in LISP, leading to Emacs, a text editor! From that period, Rosenblatt’s work was instrumental to modern AI.

abecedarius 3y ago
Solomonoff's https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... is about as basic to the theory of intelligent agents as anything gets. (He's in the pic and I'd guess this article was by a relative.)
- astrange 3y ago
  If I was an intelligent agent, I would prefer to be based on a theory that was computable without time travel, which this one isn't.
  - deleted 3y ago
    [deleted]
  - taneq 3y ago
    Ah, but time travel (or rather, prediction, but I’m being whimsical here) is the essence of intelligence. Working off your current state and inputs your mind peers forward in time to imagine the ghost of the future, and echoes of this future ripple back to drive your actions.
daveguy 3y ago
Claiming that the creator of LISP did not have a significant impact on AI is not a defensible position.
- JamilD 3y ago
  People forget for how long Lisp had an impact on AI, even outside GOFAI techniques; LeCun's early neural networks were written in Lisp: https://leon.bottou.org/publications/pdf/sn-1988.pdf https://leon.bottou.org/publications/pdf/sn-1988.pdf
  - jahewson 3y ago
    I don’t know - there’s real impact and then there’s inconsequential path dependency. This feels like the latter. The networks turned out to be valuable but LISP did not.
- aborsy 3y ago
  The story goes as, John McCarthy was applying for an assistant professorship position at MIT. MIT told him, but we have here Norbert Wiener who was a renowned mathematician at the time and had published cybernetics some time ago, in which he talks about agents interacting with the environment and feedback control, sort of modern computation-based AI. McCarthy changed the name from cybernetics to AI, and focused on symbolic systems and logic. The approach was generally not successful. Some people consider that the logic-based approach to AI pioneered in this conference contributed to an (what we now call) AI winter. People like John Pierce of Bell Labs, a very influential figure in government, defunded research in computation-based AI such as for speech recognition (he wrote articles, saying, basically, researchers pursuing these techniques are charlatans). There is no major algorithm or idea in undergrad machine learning textbooks named after these people. There are other people from that era.
  - dr_dshiv 3y ago
    Makes sense. I heard that some of Wiener’s anti-war sentiment (specifically anti-military-work-during-peacetime) may have contributed… cybernetics really collapsed hard as a discipline, even though I find it very helpful from a systems design perspective. AI has always bothered me as a term because, from a design perspective, the goal should be creating intelligent systems—not necessarily entirely artificial ones. >There is no major algorithm or idea in undergrad machine learning textbooks named after these people. Maybe the pandemonium idea from Selfridge?
fipar 3y ago
Emacs is so much more than a text editor! But I need to stay on topic... I believe your assessment of LISP (and therefore of MacArthy)'s impact on AI to be unfair. Just a few days ago https://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp was discussed on this site, for example.
dwroberts 3y ago
Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)
- aborsy 3y ago
  If I recall, Shannon returned a proposal to McCarthy for research during his stay on the cellular automata (perhaps inspired by von Neumann), not machines playing games. Shannon did foundational research in information theory, communications, cryptography, digital relay circuit design, and gambling. Crediting him for impact on AI and machine learning is a bit of a stretch (even though the IEEE Spectrum and Bell Labs published articles claiming credit, indicating that Shannon played a role). He was an avid chess player, and demonstrated machines doing various things: playing, finding their ways in a maze, juggling etc. It wasn’t based on a foundational theory of computers playing games (and the approach is different from the reinforcement learning or optimization of the models).
  - copenja 3y ago
    Shannon published a paper on Chess AI, just give it a quick google.
  - dwroberts 3y ago
    > finding their ways in a maze, Another beginning building block of the field of AI. Are you even thinking about what you’re writing here or is this the output of an LLM?
currymj 3y ago
Shannon was, I believe, the first person to define an n-gram probabilistic language model of English text and then sample from it!

by aborsy 3y ago

abecedarius 3y ago

Solomonoff's https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_induc... is about as basic to the theory of intelligent agents as anything gets. (He's in the pic and I'd guess this article was by a relative.)

astrange 3y ago
If I was an intelligent agent, I would prefer to be based on a theory that was computable without time travel, which this one isn't.
- deleted 3y ago
  [deleted]
- taneq 3y ago
  Ah, but time travel (or rather, prediction, but I’m being whimsical here) is the essence of intelligence. Working off your current state and inputs your mind peers forward in time to imagine the ghost of the future, and echoes of this future ripple back to drive your actions.

daveguy 3y ago

Claiming that the creator of LISP did not have a significant impact on AI is not a defensible position.

JamilD 3y ago
People forget for how long Lisp had an impact on AI, even outside GOFAI techniques; LeCun's early neural networks were written in Lisp: https://leon.bottou.org/publications/pdf/sn-1988.pdf https://leon.bottou.org/publications/pdf/sn-1988.pdf
- jahewson 3y ago
  I don’t know - there’s real impact and then there’s inconsequential path dependency. This feels like the latter. The networks turned out to be valuable but LISP did not.
aborsy 3y ago
The story goes as, John McCarthy was applying for an assistant professorship position at MIT. MIT told him, but we have here Norbert Wiener who was a renowned mathematician at the time and had published cybernetics some time ago, in which he talks about agents interacting with the environment and feedback control, sort of modern computation-based AI. McCarthy changed the name from cybernetics to AI, and focused on symbolic systems and logic. The approach was generally not successful. Some people consider that the logic-based approach to AI pioneered in this conference contributed to an (what we now call) AI winter. People like John Pierce of Bell Labs, a very influential figure in government, defunded research in computation-based AI such as for speech recognition (he wrote articles, saying, basically, researchers pursuing these techniques are charlatans). There is no major algorithm or idea in undergrad machine learning textbooks named after these people. There are other people from that era.
- dr_dshiv 3y ago
  Makes sense. I heard that some of Wiener’s anti-war sentiment (specifically anti-military-work-during-peacetime) may have contributed… cybernetics really collapsed hard as a discipline, even though I find it very helpful from a systems design perspective. AI has always bothered me as a term because, from a design perspective, the goal should be creating intelligent systems—not necessarily entirely artificial ones. >There is no major algorithm or idea in undergrad machine learning textbooks named after these people. Maybe the pandemonium idea from Selfridge?

fipar 3y ago

Emacs is so much more than a text editor! But I need to stay on topic... I believe your assessment of LISP (and therefore of MacArthy)'s impact on AI to be unfair. Just a few days ago https://github.com/norvig/paip-lisp https://github.com/norvig/paip-lisp was discussed on this site, for example.

dwroberts 3y ago

Shannon did foundational work on the theory of computers being able to play chess. That stuff might seem ‘obvious’ but you have to remember that would have seemed futuristic if not impossible when it was first proposed. That work in itself is fundamental to a lot of subsequent AI research (esp just the concept of ‘game playing’ as a model for testing approaches)

aborsy 3y ago
If I recall, Shannon returned a proposal to McCarthy for research during his stay on the cellular automata (perhaps inspired by von Neumann), not machines playing games. Shannon did foundational research in information theory, communications, cryptography, digital relay circuit design, and gambling. Crediting him for impact on AI and machine learning is a bit of a stretch (even though the IEEE Spectrum and Bell Labs published articles claiming credit, indicating that Shannon played a role). He was an avid chess player, and demonstrated machines doing various things: playing, finding their ways in a maze, juggling etc. It wasn’t based on a foundational theory of computers playing games (and the approach is different from the reinforcement learning or optimization of the models).
- copenja 3y ago
  Shannon published a paper on Chess AI, just give it a quick google.
- dwroberts 3y ago
  > finding their ways in a maze, Another beginning building block of the field of AI. Are you even thinking about what you’re writing here or is this the output of an LLM?

currymj 3y ago

Shannon was, I believe, the first person to define an n-gram probabilistic language model of English text and then sample from it!