4 ms·

Top-down style of AI capability denial. You got an abstract idea, like "next token prediction" and that's all you need to know. Doesn't matter what the AI does,

Top-down style of AI capability denial. You got an abstract idea, like "next token prediction" and that's all you need to know. Doesn't matter what the AI does, it has no chance to prove it reasons. Why not look at what the model does, maybe in all those tensors there are reasoning principles encoded.

beepbooptheory 3y ago
What is a "reasoning principle" and how might it be "encoded"? Also are we really calling it "denial" now? Because it's a little funny to make the move of simply psychologizing away the criticism, when your actual AI argument, presumably, is that inner consciousness/rationality is black boxes all the way down anyway. Like, how can you presume to look inside the mind of your critic so confidently, to assert they are in denial, but in the same breath say that such a thing is in principal impossible? Don't you think it maybe takes away the force of your argument? Or at least goes against its spirit?
mjburgess 3y ago
I've spent the last decade in the most technical parts of this industry; it is my job to expunge this credulousness. It is indeed trivial to show that P(A|B) is a poor model of "B => A" (and B causes A, and many other relata). Software engineers, philosophers and experimental scientists seem pretty good at seeing this --- people who "convert into" engineering are totally dumbfounded by it. P(A|B) becomes an increasingly 'useful' model of arbitrary relations as the implicit model P_m(A|B) grows to include all instances of A,B. That's what digitising all of human history and storing it in the weights of an LLM does. This all follows from basic stats you'd be taught in an applied statistics course; one never taken by most in the ML industry. (Note its still a broken model because there's an infinite number of novel instances of (A,B) pairs in most cases that cannot be modelled with this sort of inductive learning). Engineering at its heart, is a kind of pseudoscience (or, if you prefer: a magic trick). You find some heuristic which behaves as-if its the target under fragile but engineering-stable conditions. The problem with engineers who only have magic tricks in their toolkit is this credulousness. Homeopathy worked: you put people in beds and give them water and they recover (indeed, better than leaching them).
- antonvs 3y ago
  > people who "convert into" engineering are totally dumbfounded by it. How does that human failure to reason affect the evaluation of GPT's reasoning capabilities? You're being very dogmatic that anyone who has a different opinion than you are "desperate to not thinking clearly" or "gullible", but you're exhibiting errors in reasoning that are rather ironic given the context. Edit: You don't know how human reasoning works - no-one does. People tend to assume that our post-hoc conscious ability to "understand" the reasoning process must somehow relate to the actual operations the brain performs when reasoning. But that's not necessarily the case. Note that I'm not claiming that LLMs are equivalent to human in their reasoning ability: after all, in some cases they're functionally demonstrably superior, especially compared to the average human. But in other significant cases, they're certainly worse. The point is we shouldn't impose too many assumptions on how reasoning "should" work, and there seems to be a strong tendency to do that, which you're exhibiting.
  - mjburgess 3y ago
    Maybe this is a clearer way of thinking about the problem: Imagine you call a friend for help on an essay (on any topic of your choice). But they have access to google and libgen and they're quick at looking. For awhile, on the call, you think your friend is a genius but then you get suspicious. How would you tell if they knew what they were talking about? NB. whether they do or not has nothing to do with whatever qs you come up with -- they do or they dont. And, indeed, people are easy to fool. You might ask, "why care if they know?" but then you have serious questions on things which matter, not just essays. And googling isnt good enough. Being able to weight information by expertise, engage in reasoning in which contradictions are impermissible, engage in causal inference, in abduction -- in imagining possibilities which have never occurred exactly before ------ these suddenly become vital. And your friend who is not a PhD in anything, let alone everything, suddenly becomes vastly more dangerous and insidious.
    - antonvs 3y ago
      > they do or they dont Only a Sith... Seriously, the idea that "reasoning" could be a binary property seems very disconnected from the reality of the situation. > reasoning in which contradictions are impermissible Every human on the planet would fail that test, including you. "Fooling apes" is much easier when the ape you need to fool is yourself.

beepbooptheory 3y ago

What is a "reasoning principle" and how might it be "encoded"? Also are we really calling it "denial" now? Because it's a little funny to make the move of simply psychologizing away the criticism, when your actual AI argument, presumably, is that inner consciousness/rationality is black boxes all the way down anyway. Like, how can you presume to look inside the mind of your critic so confidently, to assert they are in denial, but in the same breath say that such a thing is in principal impossible? Don't you think it maybe takes away the force of your argument? Or at least goes against its spirit?

mjburgess 3y ago

I've spent the last decade in the most technical parts of this industry; it is my job to expunge this credulousness. It is indeed trivial to show that P(A|B) is a poor model of "B => A" (and B causes A, and many other relata). Software engineers, philosophers and experimental scientists seem pretty good at seeing this --- people who "convert into" engineering are totally dumbfounded by it. P(A|B) becomes an increasingly 'useful' model of arbitrary relations as the implicit model P_m(A|B) grows to include all instances of A,B. That's what digitising all of human history and storing it in the weights of an LLM does. This all follows from basic stats you'd be taught in an applied statistics course; one never taken by most in the ML industry. (Note its still a broken model because there's an infinite number of novel instances of (A,B) pairs in most cases that cannot be modelled with this sort of inductive learning). Engineering at its heart, is a kind of pseudoscience (or, if you prefer: a magic trick). You find some heuristic which behaves as-if its the target under fragile but engineering-stable conditions. The problem with engineers who only have magic tricks in their toolkit is this credulousness. Homeopathy worked: you put people in beds and give them water and they recover (indeed, better than leaching them).

antonvs 3y ago
> people who "convert into" engineering are totally dumbfounded by it. How does that human failure to reason affect the evaluation of GPT's reasoning capabilities? You're being very dogmatic that anyone who has a different opinion than you are "desperate to not thinking clearly" or "gullible", but you're exhibiting errors in reasoning that are rather ironic given the context. Edit: You don't know how human reasoning works - no-one does. People tend to assume that our post-hoc conscious ability to "understand" the reasoning process must somehow relate to the actual operations the brain performs when reasoning. But that's not necessarily the case. Note that I'm not claiming that LLMs are equivalent to human in their reasoning ability: after all, in some cases they're functionally demonstrably superior, especially compared to the average human. But in other significant cases, they're certainly worse. The point is we shouldn't impose too many assumptions on how reasoning "should" work, and there seems to be a strong tendency to do that, which you're exhibiting.
- mjburgess 3y ago
  Maybe this is a clearer way of thinking about the problem: Imagine you call a friend for help on an essay (on any topic of your choice). But they have access to google and libgen and they're quick at looking. For awhile, on the call, you think your friend is a genius but then you get suspicious. How would you tell if they knew what they were talking about? NB. whether they do or not has nothing to do with whatever qs you come up with -- they do or they dont. And, indeed, people are easy to fool. You might ask, "why care if they know?" but then you have serious questions on things which matter, not just essays. And googling isnt good enough. Being able to weight information by expertise, engage in reasoning in which contradictions are impermissible, engage in causal inference, in abduction -- in imagining possibilities which have never occurred exactly before ------ these suddenly become vital. And your friend who is not a PhD in anything, let alone everything, suddenly becomes vastly more dangerous and insidious.
  - antonvs 3y ago
    > they do or they dont Only a Sith... Seriously, the idea that "reasoning" could be a binary property seems very disconnected from the reality of the situation. > reasoning in which contradictions are impermissible Every human on the planet would fail that test, including you. "Fooling apes" is much easier when the ape you need to fool is yourself.