4 ms·

Lenat was able to produce superhuman performing AI in the early 1980s [1]. [1] https://voidfarer.livejournal.com/623.html https://voidfarer.livejournal.com/623

by thesz 3y ago

Lenat was able to produce superhuman performing AI in the early 1980s [1]. [1] https://voidfarer.livejournal.com/623.html https://voidfarer.livejournal.com/623.html You can label it "bad idea" but you can't bring LLMs back in time.

goatlover 3y ago
Why didn't it ever have the impact that LLMs are having now? Or that DeepMind has had? Cyc didn't pass the Turing Test or become superhuman chess and go players. Yet it's had much more time to become successful.
- eru 3y ago
  I'm very skeptical of Cyc and other symbolic approaches. However I think they have a good excuse for 'Why didn't it ever have the impact that LLMs are having now?': lack of data and lack of compute. And it's the same excuse that neural networks themselves have: back in those days, we just didn't have enough data, and we didn't have enough compute, even if we had the data. (Of course, we learned in the meantime that neural networks benefit a lot from extra data and extra compute. Whether that can be brought to bear on Cyc-style symbolic approaches is another question.)
  - thesz 3y ago
    Usually, LLM's output gets passed through beam search [1] which is as symbolic as one can get. [1] https://www.width.ai/post/what-is-beam-search https://www.width.ai/post/what-is-beam-search It is possible to even have 3-gram model to output better text predictions if you combine it with the beam search.
    - eru 3y ago
      See https://news.ycombinator.com/item?id=40073039 https://news.ycombinator.com/item?id=40073039 for a discussion.
- thesz 3y ago
  I guess it is funding. Compare funding of Google+Meta to what was/is available to Cyc. Cyc was able to produce an impact, I keep pointing to MathCraft [1] which, at 2017, did not have a rival in the neural AI. [1] https://en.wikipedia.org/wiki/Cyc#MathCraft https://en.wikipedia.org/wiki/Cyc#MathCraft
- Barrin92 3y ago
  Because the goal of Cyc was always to build a genuinely generally intelligent common sense reasoning system and that is a non-sexy long term research project. Being great at chess isn't a measure that's relevant for a project like this, Stockfish is superhuman at chess.
- riku_iki 3y ago
  > Why didn't it ever have the impact that LLMs are having now? Google has its own Knowledge Graph, with billions of daily views, which is wider but more shallow version of Cyc. It is unclear if LLM user facing impact surpassed that project.

by thesz 3y ago

goatlover 3y ago

Why didn't it ever have the impact that LLMs are having now? Or that DeepMind has had? Cyc didn't pass the Turing Test or become superhuman chess and go players. Yet it's had much more time to become successful.

eru 3y ago
I'm very skeptical of Cyc and other symbolic approaches. However I think they have a good excuse for 'Why didn't it ever have the impact that LLMs are having now?': lack of data and lack of compute. And it's the same excuse that neural networks themselves have: back in those days, we just didn't have enough data, and we didn't have enough compute, even if we had the data. (Of course, we learned in the meantime that neural networks benefit a lot from extra data and extra compute. Whether that can be brought to bear on Cyc-style symbolic approaches is another question.)
- thesz 3y ago
  Usually, LLM's output gets passed through beam search [1] which is as symbolic as one can get. [1] https://www.width.ai/post/what-is-beam-search https://www.width.ai/post/what-is-beam-search It is possible to even have 3-gram model to output better text predictions if you combine it with the beam search.
  - eru 3y ago
    See https://news.ycombinator.com/item?id=40073039 https://news.ycombinator.com/item?id=40073039 for a discussion.
thesz 3y ago
I guess it is funding. Compare funding of Google+Meta to what was/is available to Cyc. Cyc was able to produce an impact, I keep pointing to MathCraft [1] which, at 2017, did not have a rival in the neural AI. [1] https://en.wikipedia.org/wiki/Cyc#MathCraft https://en.wikipedia.org/wiki/Cyc#MathCraft
Barrin92 3y ago
Because the goal of Cyc was always to build a genuinely generally intelligent common sense reasoning system and that is a non-sexy long term research project. Being great at chess isn't a measure that's relevant for a project like this, Stockfish is superhuman at chess.
riku_iki 3y ago
> Why didn't it ever have the impact that LLMs are having now? Google has its own Knowledge Graph, with billions of daily views, which is wider but more shallow version of Cyc. It is unclear if LLM user facing impact surpassed that project.