14 ms·

> I loved GEB when I first read it in high school, and when I first reread it years later, but I don’t think its fundamental view of the relation between minds

by OskarS 6y ago

> I loved GEB when I first read it in high school, and when I first reread it years later, but I don’t think its fundamental view of the relation between minds and machines has stood the test of time. I still see this as an open question. You’re certainly correct that this kind of AI research is seriously out of vogue, but it seems to me that while “modern” brute force compute AI puts up impressive results and is a hugely useful technique, it has made exactly zero progress on anything that could be conceived of as “general intelligence”. It just doesn’t seem to me to be a thing AI researchers are even interested in any more. Like, the Twitter program that uses AI to crop images based on a dataset of a gazillion cropped images is pretty far from Turing’s thinking machines. I don’t know the way there, but it always seemed to me that the old-style AI research in the GEB style is still a rich vein we haven’t come close to mining out.

strangegecko 6y ago
> this kind of AI research is seriously out of vogue I would think this is a consequence of it being a very hard problem. ML gets all the industry funding and publicity because you get results that are immediately useful. You can work on General AI for decades and come up empty handed since there doesn't seem to be an incremental approach where intermediate results are useful. So it's closer to esoteric mathematics or philosophy in terms of "being en vogue". So I see this mostly as a reflection of the academic landscape in general. Funding is more focused on application and less on theoretical / fundamental research.
joseluis 6y ago
I very much agree. The recent outstanding advances in AI since at least the 90s seems to be in the spectrum of recognizing patterns bruteforcing through an insurmountable amount of data, arriving at black boxes that provide narrow solutions, with its inner structure remaining inescrutable to human understanding. I'm currently reading Surfaces and Essences from the same author, and it's so far been most illuminating. He very convincingly presents the thesis of the analogy being the foundation of each nd every concept and human thought. If someone could manage to apply and translate those insights into real algorithms and data structures to play with, that would be IMHO a big step towards general AI, or the very least a much more human like AI approach with its particular applications.
- heuroci 6y ago
  I am reading the same book. I think analogy is interesting but it doesn’t make sense that it would be the foundation. analogy is a heuristic to reason and communicate. It necessarily comes after intelligence.
  - galaxyLogic 6y ago
    I think visual analogues guide abstract analysis of the analogues. In our brain we recognize shapes as similar whether that be visual shares or shapes of conceptual analysis. Category theory has an appealing visual base of links/arrows between things.
    - heuroci 6y ago
      I share your view on category theory and perhaps lattice theory as well. What I was trying to say is that before you can draw that analogy, regardless of what "shape" means to you , you have to construct the base shape. Analogy to me comes after you have stabilized "shapes" where you can draw comparison between a new input a set of stable patterns at which point you can point out the analogy. Syllogism may be closely related/involved. I am not disputing that being able to draw parallels and employ analogies are related to intelligence. I am just not so sure that this is how you can synthesize intelligence. It is all very complex of course, and there is a school of thought that cognition is very closely related to sensory and embodiment (i.e.: embodied cognition ). I think Aubrey de grey switched from AI to longevity because he figured he will need a long life to understand intelligence :) Personally, I am still working on it but had adjusted the scope a bit.
  - hyperpallium2 6y ago
    Like shadows creating artificial shadows with paint.
redisman 6y ago
I know its kind of a unpopular opinion but to me ML is likely a local maxima of applied statistics and not one that leads to any kind of "real" AI.
- asdff 6y ago
  I don't think that's an unpopular opinion since that's a fact, ML is literally statistics. The central question is "given my training set of y, how likely is that my input x is also y," which is probability.
  - deleted 6y ago
    [deleted]
  - The_rationalist 6y ago
    Machine learning isn't necessarily probabilistic. E.g a rule based (causal) parser can learn the structure of a document.
    - dlkf 6y ago
      No. In machine learning, the word "learn" means that the function that maps inputs to outputs is learned from data. In the case of a rule-based parser, this function is crafted by humans rather than learned from data, so it's not an example of machine learning. If we start using words this way, you could say that a deterministic fibonacci function "learns" the value of fib(5) by computing it. "Machine learning" becomes synonymous with "computer science."
      - The_rationalist 6y ago
        No and yes, we are both wrong so let's refine our thinking in order to gain accuracy and reach agreement. Machine learning isn't necessarily probabilistic. This statement definitely is true and I'll prove it. However my original example E.g a rule based (causal) parser can learn the structure of a document. alone is unsufficent to qualify it of machine learning! Indeed, an HTML parser alone isn't enough to say it's learning, more accurately it is only memorizing the structure of a page. ML means that the function that maps inputs to outputs is learned from data. This definition is overly restrictive (but match e.g the behavior of neural networks). Wikipedia has a more inclusive and useful definition of what qualifies as ML: Machine learning (ML) is the study of computer algorithms that improve automatically through experience and by the use of data.[1] So a ML algorithm allow to automatically improve throughput at a tasks through learning the representation of the data it is fed. Considering this definition, we can realize that the most used ML algorithm in the world is Pagerank (for improving the ranking of Google search results). And surprisingly, this algorithm is non-probabilistic. It attribute higher weight proportionally to URLs that are the most linked to by other sites. I.e it basically learn the structure of the graph that is the web. And its performance is data driven. So it's not just about causally memorizing the structure of a graph but also about reusing its past memories for newer queries, then we can effectively talk about machine learning. Pagerank diagram -> https://en.m.wikipedia.org/wiki/PageRank#/media/File:PageRank-hi-res.png https://en.m.wikipedia.org/wiki/PageRank#/media/File:PageRan... I'll give you an example of a non statistical machine learning algorithm that I am currently developing: Semantic parsing is the task of encoding semantic meaning in a graph from a natural language text input. This graph can then be used for semantic question answering. It is rule based and it will learn the semantic structure of the text. The more data you fed it the more knowledge it can encode. Hence it's performance at question answering increase with experience. It is causal and yet it fit the useful definition of machine learning cited above.
quotemstr 6y ago
> exactly zero progress on anything that could be conceived of as “general intelligence” GPT-3 is hilarious. If that's not a sign of our inching towards general AI, nothing is. From https://www.gwern.net/GPT-3#devils-dictionary-of-science https://www.gwern.net/GPT-3#devils-dictionary-of-science (The "Navy Seal Copypasta" section is even funnier, but it's not really HN-appropriate.) -- “A role for…” [phrase] A frequent phrase found in submitted and published papers; it often indicates that the authors have nothing to say about the topic of their paper. In its more emphatic form, “A role for…” usually indicates a struggle by the authors to take a side on an issue, after a lengthy attempt to be both non-committal and a supporting party to all sides, as often happens in “molecular and cellular” or “basic and translational” research. “Reviewer” [noun] A participant in the review of a grant, paper, or grant proposal. In spite of being in a poor position to assess the merits of a proposal, reviewer tends to demand that authors submit their data for statistical analysis and back their results with it, which the reviewer usually does not. Reviewer usually requires that the author cite his or her own work to prove that he or she is worth reviewing. It is also assumed that the reviewer can detect the slightest amount of bias in any paper, which the reviewer also assumes has not been corrected for." “Rigor” Something for scientists to aspire to, a state of mind that would not be required if scientists could be trusted to do their job. “Science” A complex web of data, opinions, lies, and errors, now considered the most important (because most expensive) technology in the modern society. To remind you of this, you will frequently see scientists and editors use the word, claim to do something for the sake of science, or see it used as an adjective. “The topic of the paper” A wide-ranging category of things or ideas that may not have been relevant when the paper was written, but which the authors believe the paper should be about. Often, the topic is too broad or a non-topic, but is occasionally useful in order to generate support for yet another set of related papers, conferences, seminars, webinars, and so forth, which in turn are used to generate more data for “new findings”, which, after they are manipulated enough, may end up being published and generating yet more data to support a “re-review” of the original paper or other things. “Validation step” Another name for a random setting of a parameter of a model, simulation, or algorithm. “Writeup” A form of scientific communication in which the author states the information he or she wanted the readers to extract from the paper while making it as difficult as possible for them to find it. “Writer’s block” A common affliction among students, arising from various causes, such as: their desire to sell their ideas for a profit, their inability to realize this desire, the fact that their ideas are not selling and will not be bought, and the delusion that most of the wealth and fame in the world would be theirs if they would spend enough years doing science.
maest 6y ago
I believe it was Russell and Norvig who said about current AI techniques that they are like a man trying to reach the moon by climbing a tree: “One can report steady progress, all the way to the top of the tree.”
- bagels 6y ago
  And then we eventually build the space elevator.
- sn41 6y ago
  Jitendra Malik, Computer Vision fame, calls it "the fallacy of the successful first step". Look ma, I can leap! It's only a matter of time till we get to the moon.
  - beambot 6y ago
    To be fair: if you measure the cosmological time from the first human step to man walking on the moon, it was barely the blink of an eye...
    - coldtea 6y ago
      Sure. But why should we compare it to the cosmological time, and not to e.g. civilization time? Or homo sapiens time? In which case it took 100s to 1000s of lifetimes between the step and the moon...
- earthboundkid 6y ago
  The other criticism is that the sheer amount of data in use shows we’re doing something wrong. A child can internalize the grammar of a language with only a few years of exposure. Modern AI corpuses consists of terabytes of data. Why are the results so lacking despite using vastly more input?
  - mpoteat 6y ago
    Why would you assume that years of exposure isn't terabytes of data? At least we can train language models in hours or days rather than years. If you view it that way we've made quite a bit of progress!
justicezyx 6y ago
> general intelligence You might be interested in Jean Piaget's work on the development of intelligence, and his observation on children development and various stages of intelligence. BTW what is general intelligence in your opinion?

by OskarS 6y ago

strangegecko 6y ago

> this kind of AI research is seriously out of vogue I would think this is a consequence of it being a very hard problem. ML gets all the industry funding and publicity because you get results that are immediately useful. You can work on General AI for decades and come up empty handed since there doesn't seem to be an incremental approach where intermediate results are useful. So it's closer to esoteric mathematics or philosophy in terms of "being en vogue". So I see this mostly as a reflection of the academic landscape in general. Funding is more focused on application and less on theoretical / fundamental research.

joseluis 6y ago

I very much agree. The recent outstanding advances in AI since at least the 90s seems to be in the spectrum of recognizing patterns bruteforcing through an insurmountable amount of data, arriving at black boxes that provide narrow solutions, with its inner structure remaining inescrutable to human understanding. I'm currently reading Surfaces and Essences from the same author, and it's so far been most illuminating. He very convincingly presents the thesis of the analogy being the foundation of each nd every concept and human thought. If someone could manage to apply and translate those insights into real algorithms and data structures to play with, that would be IMHO a big step towards general AI, or the very least a much more human like AI approach with its particular applications.

heuroci 6y ago
I am reading the same book. I think analogy is interesting but it doesn’t make sense that it would be the foundation. analogy is a heuristic to reason and communicate. It necessarily comes after intelligence.
- galaxyLogic 6y ago
  I think visual analogues guide abstract analysis of the analogues. In our brain we recognize shapes as similar whether that be visual shares or shapes of conceptual analysis. Category theory has an appealing visual base of links/arrows between things.
  - heuroci 6y ago
    I share your view on category theory and perhaps lattice theory as well. What I was trying to say is that before you can draw that analogy, regardless of what "shape" means to you , you have to construct the base shape. Analogy to me comes after you have stabilized "shapes" where you can draw comparison between a new input a set of stable patterns at which point you can point out the analogy. Syllogism may be closely related/involved. I am not disputing that being able to draw parallels and employ analogies are related to intelligence. I am just not so sure that this is how you can synthesize intelligence. It is all very complex of course, and there is a school of thought that cognition is very closely related to sensory and embodiment (i.e.: embodied cognition ). I think Aubrey de grey switched from AI to longevity because he figured he will need a long life to understand intelligence :) Personally, I am still working on it but had adjusted the scope a bit.
- hyperpallium2 6y ago
  Like shadows creating artificial shadows with paint.

redisman 6y ago

I know its kind of a unpopular opinion but to me ML is likely a local maxima of applied statistics and not one that leads to any kind of "real" AI.

asdff 6y ago
I don't think that's an unpopular opinion since that's a fact, ML is literally statistics. The central question is "given my training set of y, how likely is that my input x is also y," which is probability.
- deleted 6y ago
  [deleted]
- The_rationalist 6y ago
  Machine learning isn't necessarily probabilistic. E.g a rule based (causal) parser can learn the structure of a document.
  - dlkf 6y ago
    No. In machine learning, the word "learn" means that the function that maps inputs to outputs is learned from data. In the case of a rule-based parser, this function is crafted by humans rather than learned from data, so it's not an example of machine learning. If we start using words this way, you could say that a deterministic fibonacci function "learns" the value of fib(5) by computing it. "Machine learning" becomes synonymous with "computer science."
    - The_rationalist 6y ago
      No and yes, we are both wrong so let's refine our thinking in order to gain accuracy and reach agreement. Machine learning isn't necessarily probabilistic. This statement definitely is true and I'll prove it. However my original example E.g a rule based (causal) parser can learn the structure of a document. alone is unsufficent to qualify it of machine learning! Indeed, an HTML parser alone isn't enough to say it's learning, more accurately it is only memorizing the structure of a page. ML means that the function that maps inputs to outputs is learned from data. This definition is overly restrictive (but match e.g the behavior of neural networks). Wikipedia has a more inclusive and useful definition of what qualifies as ML: Machine learning (ML) is the study of computer algorithms that improve automatically through experience and by the use of data.[1] So a ML algorithm allow to automatically improve throughput at a tasks through learning the representation of the data it is fed. Considering this definition, we can realize that the most used ML algorithm in the world is Pagerank (for improving the ranking of Google search results). And surprisingly, this algorithm is non-probabilistic. It attribute higher weight proportionally to URLs that are the most linked to by other sites. I.e it basically learn the structure of the graph that is the web. And its performance is data driven. So it's not just about causally memorizing the structure of a graph but also about reusing its past memories for newer queries, then we can effectively talk about machine learning. Pagerank diagram -> https://en.m.wikipedia.org/wiki/PageRank#/media/File:PageRank-hi-res.png https://en.m.wikipedia.org/wiki/PageRank#/media/File:PageRan... I'll give you an example of a non statistical machine learning algorithm that I am currently developing: Semantic parsing is the task of encoding semantic meaning in a graph from a natural language text input. This graph can then be used for semantic question answering. It is rule based and it will learn the semantic structure of the text. The more data you fed it the more knowledge it can encode. Hence it's performance at question answering increase with experience. It is causal and yet it fit the useful definition of machine learning cited above.

quotemstr 6y ago

> exactly zero progress on anything that could be conceived of as “general intelligence” GPT-3 is hilarious. If that's not a sign of our inching towards general AI, nothing is. From https://www.gwern.net/GPT-3#devils-dictionary-of-science https://www.gwern.net/GPT-3#devils-dictionary-of-science (The "Navy Seal Copypasta" section is even funnier, but it's not really HN-appropriate.) -- “A role for…” [phrase] A frequent phrase found in submitted and published papers; it often indicates that the authors have nothing to say about the topic of their paper. In its more emphatic form, “A role for…” usually indicates a struggle by the authors to take a side on an issue, after a lengthy attempt to be both non-committal and a supporting party to all sides, as often happens in “molecular and cellular” or “basic and translational” research. “Reviewer” [noun] A participant in the review of a grant, paper, or grant proposal. In spite of being in a poor position to assess the merits of a proposal, reviewer tends to demand that authors submit their data for statistical analysis and back their results with it, which the reviewer usually does not. Reviewer usually requires that the author cite his or her own work to prove that he or she is worth reviewing. It is also assumed that the reviewer can detect the slightest amount of bias in any paper, which the reviewer also assumes has not been corrected for." “Rigor” Something for scientists to aspire to, a state of mind that would not be required if scientists could be trusted to do their job. “Science” A complex web of data, opinions, lies, and errors, now considered the most important (because most expensive) technology in the modern society. To remind you of this, you will frequently see scientists and editors use the word, claim to do something for the sake of science, or see it used as an adjective. “The topic of the paper” A wide-ranging category of things or ideas that may not have been relevant when the paper was written, but which the authors believe the paper should be about. Often, the topic is too broad or a non-topic, but is occasionally useful in order to generate support for yet another set of related papers, conferences, seminars, webinars, and so forth, which in turn are used to generate more data for “new findings”, which, after they are manipulated enough, may end up being published and generating yet more data to support a “re-review” of the original paper or other things. “Validation step” Another name for a random setting of a parameter of a model, simulation, or algorithm. “Writeup” A form of scientific communication in which the author states the information he or she wanted the readers to extract from the paper while making it as difficult as possible for them to find it. “Writer’s block” A common affliction among students, arising from various causes, such as: their desire to sell their ideas for a profit, their inability to realize this desire, the fact that their ideas are not selling and will not be bought, and the delusion that most of the wealth and fame in the world would be theirs if they would spend enough years doing science.

maest 6y ago

I believe it was Russell and Norvig who said about current AI techniques that they are like a man trying to reach the moon by climbing a tree: “One can report steady progress, all the way to the top of the tree.”

bagels 6y ago
And then we eventually build the space elevator.
sn41 6y ago
Jitendra Malik, Computer Vision fame, calls it "the fallacy of the successful first step". Look ma, I can leap! It's only a matter of time till we get to the moon.
- beambot 6y ago
  To be fair: if you measure the cosmological time from the first human step to man walking on the moon, it was barely the blink of an eye...
  - coldtea 6y ago
    Sure. But why should we compare it to the cosmological time, and not to e.g. civilization time? Or homo sapiens time? In which case it took 100s to 1000s of lifetimes between the step and the moon...
earthboundkid 6y ago
The other criticism is that the sheer amount of data in use shows we’re doing something wrong. A child can internalize the grammar of a language with only a few years of exposure. Modern AI corpuses consists of terabytes of data. Why are the results so lacking despite using vastly more input?
- mpoteat 6y ago
  Why would you assume that years of exposure isn't terabytes of data? At least we can train language models in hours or days rather than years. If you view it that way we've made quite a bit of progress!

justicezyx 6y ago

> general intelligence You might be interested in Jean Piaget's work on the development of intelligence, and his observation on children development and various stages of intelligence. BTW what is general intelligence in your opinion?