3 ms·

I think you should be more precise and avoid anthropomorphism when talking about gen AI, as anthropomorphism leads to a lot of shaky epistemological assumptions

by viccis 1y ago

I think you should be more precise and avoid anthropomorphism when talking about gen AI, as anthropomorphism leads to a lot of shaky epistemological assumptions. Your car example didn't imply intelligence, but we're talking about a technology that people misguidedly treat as though it is real intelligence.

exe34 1y ago
What does "real intelligence" mean? I fear that any discussion that starts with the assumption such a thing exists will only end up as "oh only carbon based humans (or animals if you happen to be generous) have it".
- viccis 1y ago
  Any intelligence that can synthesize knowledge with or without direct experience.
  - exe34 1y ago
    So ChatGPT? Or maybe that can't "really synthesize"?
    - viccis 1y ago
      How would ChatGPT come up with something truly novel, not related to anything it's ever seen before?
      - exe34 1y ago
        Has a human ever done that?
        viccis 1y ago
        We obviously can, otherwise where do our myriad of complex concepts, many of which aren't empirical, come from? How could we have modern mathematics unless some thinker had devised the various ways of conceptualizing and manipulating numbers? This is a very old question [1] with a number of good answers as to how a human can [2]. 1: https://plato.stanford.edu/entries/hume/#CopyPrin https://plato.stanford.edu/entries/hume/#CopyPrin 2: https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_distinction#The_ease_of_knowing_analytic_propositions https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_dis...
        ben_w 1y ago
        As you link to The Copy Principle: it, or at least that summary of it, appears to be very much what AI do. As a priori knowledge is all based on axioms, I do not accept that it is an example of "something truly novel, not related to anything it's ever seen before". Knowledge, yes, but not of the kind you describe. And this would still be the case even if LLMs couldn't approximate logical theorem provers, which they can: https://chatgpt.com/share/685528af-4270-8011-ba75-e601211a02c7 https://chatgpt.com/share/685528af-4270-8011-ba75-e601211a02...
        exe34 1y ago
        You'd have to pick something that fits: > come up with something truly novel, not related to anything it's ever seen before? I've never heard of a human coming up with something that's not related to anything they've ever seen before. There is no concept in science that I know of that just popped into existence in somebody's head. Everyone credits those who came before.
  - ben_w 1y ago
    Here you say: > with or without But in the other reply, you're asking for: > something truly novel, not related to anything it's ever seen before So, assuming the former was a typo, you only believe in a priori knowledge, e.g. maths and logic? https://en.wikipedia.org/wiki/A_priori_and_a_posteriori https://en.wikipedia.org/wiki/A_priori_and_a_posteriori I mean, LLMs can and do help with this even though it's not their strength; that's more of a Lean-type-problem: https://en.wikipedia.org/wiki/Lean_(proof_assistant) https://en.wikipedia.org/wiki/Lean_(proof_assistant)
    - viccis 1y ago
      Yeah I was specifically asking for synthetic a priori knowledge, which AI by definition can't provide. It can only estimate the joint distribution over tokens, so anything generated from it is by definition a posteriori. It can generate novel statements, but I don't think there's any compelling definition of "knowledge" (including the common JTB one) that could apply to what it actually is (it's just the highest probability semiotic result). And in fact, going by the JTB definition of knowledge, AI models making correct novel statements would just be an elaborate example of a Gettier problem. I think LLMs as a symbolic layer (effective, as a "sense organ") with some kind of logical reasoning engine like everyone loved decades ago could accomplish something closer to "intelligence" or "thinking", which I assume is what you were implying with Lean.
      - ben_w 1y ago
        My example with Lean is that it's specifically a thing that does a priori knowledge: given "A implies B" and "A", therefore "B". Or all of maths from the chosen axioms. So, just to be clear, you were asked: > What does "real intelligence" mean? And your answer is that it must be a priori knowledge, and are fine with Lean being one. But you don't accept that LLMs can weakly approximate theorem provers? FWIW, I agree that the "Justified True Belief" definition of knowledge leads to such conclusions as you draw, but I would say that this is also the case with humans — if you do this, then the Gettier problems show that even humans only have belief, not knowledge: when you "see a sheep in a field", you may be later embarrassed to learn that what you saw was a white coated Puli and there was a real sheep hiding behind a bush, but in the moment the subjective experience of your state of "knowledge" is exactly the same as if you had, in fact, seen a sheep. Just, be careful with what is meant by the word "belief", there's more than one way I can also contradict Wittgenstein's quote on belief: > If there were a verb meaning "to believe falsely," it would not have any significant first person, present indicative. Depending on what I mean by "believe", and indeed "I" given that different parts of my mind can disagree with each other (which is why motion sickness happens).

by viccis 1y ago

exe34 1y ago

What does "real intelligence" mean? I fear that any discussion that starts with the assumption such a thing exists will only end up as "oh only carbon based humans (or animals if you happen to be generous) have it".

viccis 1y ago
Any intelligence that can synthesize knowledge with or without direct experience.
- exe34 1y ago
  So ChatGPT? Or maybe that can't "really synthesize"?
  - viccis 1y ago
    How would ChatGPT come up with something truly novel, not related to anything it's ever seen before?
    - exe34 1y ago
      Has a human ever done that?
      - viccis 1y ago
        We obviously can, otherwise where do our myriad of complex concepts, many of which aren't empirical, come from? How could we have modern mathematics unless some thinker had devised the various ways of conceptualizing and manipulating numbers? This is a very old question [1] with a number of good answers as to how a human can [2]. 1: https://plato.stanford.edu/entries/hume/#CopyPrin https://plato.stanford.edu/entries/hume/#CopyPrin 2: https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_distinction#The_ease_of_knowing_analytic_propositions https://en.wikipedia.org/wiki/Analytic%E2%80%93synthetic_dis...
        ben_w 1y ago
        As you link to The Copy Principle: it, or at least that summary of it, appears to be very much what AI do. As a priori knowledge is all based on axioms, I do not accept that it is an example of "something truly novel, not related to anything it's ever seen before". Knowledge, yes, but not of the kind you describe. And this would still be the case even if LLMs couldn't approximate logical theorem provers, which they can: https://chatgpt.com/share/685528af-4270-8011-ba75-e601211a02c7 https://chatgpt.com/share/685528af-4270-8011-ba75-e601211a02...
        exe34 1y ago
        You'd have to pick something that fits: > come up with something truly novel, not related to anything it's ever seen before? I've never heard of a human coming up with something that's not related to anything they've ever seen before. There is no concept in science that I know of that just popped into existence in somebody's head. Everyone credits those who came before.
- ben_w 1y ago
  Here you say: > with or without But in the other reply, you're asking for: > something truly novel, not related to anything it's ever seen before So, assuming the former was a typo, you only believe in a priori knowledge, e.g. maths and logic? https://en.wikipedia.org/wiki/A_priori_and_a_posteriori https://en.wikipedia.org/wiki/A_priori_and_a_posteriori I mean, LLMs can and do help with this even though it's not their strength; that's more of a Lean-type-problem: https://en.wikipedia.org/wiki/Lean_(proof_assistant) https://en.wikipedia.org/wiki/Lean_(proof_assistant)
  - viccis 1y ago
    Yeah I was specifically asking for synthetic a priori knowledge, which AI by definition can't provide. It can only estimate the joint distribution over tokens, so anything generated from it is by definition a posteriori. It can generate novel statements, but I don't think there's any compelling definition of "knowledge" (including the common JTB one) that could apply to what it actually is (it's just the highest probability semiotic result). And in fact, going by the JTB definition of knowledge, AI models making correct novel statements would just be an elaborate example of a Gettier problem. I think LLMs as a symbolic layer (effective, as a "sense organ") with some kind of logical reasoning engine like everyone loved decades ago could accomplish something closer to "intelligence" or "thinking", which I assume is what you were implying with Lean.
    - ben_w 1y ago
      My example with Lean is that it's specifically a thing that does a priori knowledge: given "A implies B" and "A", therefore "B". Or all of maths from the chosen axioms. So, just to be clear, you were asked: > What does "real intelligence" mean? And your answer is that it must be a priori knowledge, and are fine with Lean being one. But you don't accept that LLMs can weakly approximate theorem provers? FWIW, I agree that the "Justified True Belief" definition of knowledge leads to such conclusions as you draw, but I would say that this is also the case with humans — if you do this, then the Gettier problems show that even humans only have belief, not knowledge: when you "see a sheep in a field", you may be later embarrassed to learn that what you saw was a white coated Puli and there was a real sheep hiding behind a bush, but in the moment the subjective experience of your state of "knowledge" is exactly the same as if you had, in fact, seen a sheep. Just, be careful with what is meant by the word "belief", there's more than one way I can also contradict Wittgenstein's quote on belief: > If there were a verb meaning "to believe falsely," it would not have any significant first person, present indicative. Depending on what I mean by "believe", and indeed "I" given that different parts of my mind can disagree with each other (which is why motion sickness happens).