4 ms·

Thanks for sharing. I also tend to view learning in terms of manifolds. It's a powerful representation. > I'm personally pretty convinced that, in a high enoug

Thanks for sharing. I also tend to view learning in terms of manifolds. It's a powerful representation. > I'm personally pretty convinced that, in a high enough dimensional space, this is indistinguishable from reasoning I actually have journaled extensively about this and even written some on Hacker News about it with respect to what I've been calling probabilistic reasoning manifolds: > This manifold is constructed via learning a decontextualized pattern space on a given set of inputs. Given the inherent probabilistic nature of sampling, true reasoning is expressed in terms of probabilities, not axioms. It may be possible to discover axioms by locating fixed points or attractors on the manifold, but ultimately you're looking at a probabilistic manifold constructed from your input set. > But I don't think you can untie this "reasoning" from your input data. It's possible you will find "meta-reasoning", or similar structures found in any sufficiently advanced reasoning manifold, but these highly decontextualized structures might be entirely useless without proper recontextualization, necessitating that a reasoning manifold is trained on input whose patterns follow learnable underlying rules, if the manifold is to be useful for processing input of that kind. > Decontextualization is learning, decomposing aspects of an input into context-agnostic relationships. But recontextualization is the other half of that, knowing how to take highly abstract, sometimes inexpressible, context-agnostic relationships and transform them into useful analysis in novel domains Full comment: https://news.ycombinator.com/item?id=42871894 https://news.ycombinator.com/item?id=42871894

mjburgess 1y ago
Are you talking about reasoning in general, reasoning qua that mental process which operates on (representations of) propositions? In which case, I cannot understand " true reasoning is expressed in terms of probabilities, not axioms " One of the features of reasoning is that it does not operate in this way. It's highly implausible animals would have been endowed with no ability to operate non-probabilistically on propositions represented by them, since this is essential for correct reasoning -- and a relatively trivial capability to provide. Eg., "if the spider is in boxA, then it is not everywhere else" and so on
- jvanderbot 1y ago
  I suspect, as a layperson who watches people make decisions all the time, that somewhere in our mind is a "certainty checker". We don't do logic itself, we just create logic from certainty as part of verbal reasoning. It's our messy internal inference of likelihoods that causes us to pause and think, or dash forward with confidence, and convincing others is the only place we need things like "theorems". This is the only way I can square things like intuition, writing to formalize thoughts, verbal argument, etc, with the fact that people are just so mushy all the time.
  - mjburgess 1y ago
    People are only mushy in their verbalised reasoning, because its the nature of such reasoning to handle hard cases. Animal cognition, at its basic levels, is incredibly refined and makes necessary use of logic, flawlessly, frequently. This naive cynicism about our mental capacities is a product of this credulity about statistical AI. If one beings with an earnest study of animal intelligence, in order to describe it, it disappears. It's exactly and only a project of the child playing with his lego, certain that great engineering projects have little use for any more than stacking bricks.
    - jvanderbot 1y ago
      Well, we disagree fundamentally. And, I applaud the heavy handed use of condescension. Logical propositions ("2+2=4 regardless of my certainty about it") seem a long way from necessary or sufficient to survival for animals. A fuzzy heatmap of "where is prey going" or "How many prey over there" is much closer to necessary and sufficient. The fact that measurements or senses can update those estimates is a long way from a logical deduction. Something more like probability factor graph can do it, without the pernicious use of "concepts" or too much need for implication, which is sticky and overly rigorous. That's all I have to say, and I doubt we'll find middle ground.
      - mjburgess 1y ago
        You can enumerate, all you wish, all the fuzzy judgements we need to make. This confirms a capacity for uncertain reasoning. It says nothing about the trivial and innumerable ways concepts compose both in content (imagine that A and-also B) and in logical relation (eg., imagine that not A and B). The point of my "condescension" was to point out that people of your position are arguing from ignorance, with confirmation bias -- ie., having no study of animal intelligence, and only ever repeating what they know about their own study of irrelevant systems. Your reply evidences this exactly. Zero engagement with any facts on the ground about actual animal intelligence. Are you really actually trying to account for animal intelligence, or as I have claimed twice now, are you only really wishing to maintain your ignorance of it, dismiss any analysis of it, and instead "confirm" that whatever you are aware of "must, presumably, apply". Imagine being faced with such bad faith over and over and over again. It is like arguing with people who insist the world is flat, and when challenged, point to euclidean geometry and the flatness of the pavement under their feet. If I begin by anticipating such behaviour, you can see why.

mjburgess 1y ago

Are you talking about reasoning in general, reasoning qua that mental process which operates on (representations of) propositions? In which case, I cannot understand " true reasoning is expressed in terms of probabilities, not axioms " One of the features of reasoning is that it does not operate in this way. It's highly implausible animals would have been endowed with no ability to operate non-probabilistically on propositions represented by them, since this is essential for correct reasoning -- and a relatively trivial capability to provide. Eg., "if the spider is in boxA, then it is not everywhere else" and so on

jvanderbot 1y ago
I suspect, as a layperson who watches people make decisions all the time, that somewhere in our mind is a "certainty checker". We don't do logic itself, we just create logic from certainty as part of verbal reasoning. It's our messy internal inference of likelihoods that causes us to pause and think, or dash forward with confidence, and convincing others is the only place we need things like "theorems". This is the only way I can square things like intuition, writing to formalize thoughts, verbal argument, etc, with the fact that people are just so mushy all the time.
- mjburgess 1y ago
  People are only mushy in their verbalised reasoning, because its the nature of such reasoning to handle hard cases. Animal cognition, at its basic levels, is incredibly refined and makes necessary use of logic, flawlessly, frequently. This naive cynicism about our mental capacities is a product of this credulity about statistical AI. If one beings with an earnest study of animal intelligence, in order to describe it, it disappears. It's exactly and only a project of the child playing with his lego, certain that great engineering projects have little use for any more than stacking bricks.
  - jvanderbot 1y ago
    Well, we disagree fundamentally. And, I applaud the heavy handed use of condescension. Logical propositions ("2+2=4 regardless of my certainty about it") seem a long way from necessary or sufficient to survival for animals. A fuzzy heatmap of "where is prey going" or "How many prey over there" is much closer to necessary and sufficient. The fact that measurements or senses can update those estimates is a long way from a logical deduction. Something more like probability factor graph can do it, without the pernicious use of "concepts" or too much need for implication, which is sticky and overly rigorous. That's all I have to say, and I doubt we'll find middle ground.
    - mjburgess 1y ago
      You can enumerate, all you wish, all the fuzzy judgements we need to make. This confirms a capacity for uncertain reasoning. It says nothing about the trivial and innumerable ways concepts compose both in content (imagine that A and-also B) and in logical relation (eg., imagine that not A and B). The point of my "condescension" was to point out that people of your position are arguing from ignorance, with confirmation bias -- ie., having no study of animal intelligence, and only ever repeating what they know about their own study of irrelevant systems. Your reply evidences this exactly. Zero engagement with any facts on the ground about actual animal intelligence. Are you really actually trying to account for animal intelligence, or as I have claimed twice now, are you only really wishing to maintain your ignorance of it, dismiss any analysis of it, and instead "confirm" that whatever you are aware of "must, presumably, apply". Imagine being faced with such bad faith over and over and over again. It is like arguing with people who insist the world is flat, and when challenged, point to euclidean geometry and the flatness of the pavement under their feet. If I begin by anticipating such behaviour, you can see why.