8 ms·

>> And this should be obvious given this simple sketch of the argument: classical logic is just Bayesian inference with all probabilities pinned to 0 and 1, and

>> And this should be obvious given this simple sketch of the argument: classical logic is just Bayesian inference with all probabilities pinned to 0 and 1, and a model of a system in classical logic is a model of how it operates. Therefore you just need to run Bayes' rule long enough for the probabilities to converge. I've heard this before and it's an argument that perhaps makes sense from a Bayesian point of view, but makes no sense from a logic point of view. To clarify, here's some relevant background on classical logic and probabilities. I will take First Order Logic (FOL) as an example of a classical logic, but the same observations hold true for e.g. propositional logic. In FOL semantics, an interpretation assigns truth values in {true, false} to the atomic formulae ("atoms") of a predicate. In Bayesian inference, a distribution assigns probabilities to the outcomes of an event, such that the sum of probabilities of all outcomes of an event is 1. Now, for "classical logic [to be] just Bayesian inference with all probabilities pinned to 0 and 1", it must mean that, given an interpretation I, all atoms true under I are assigned the probability "1" and all atoms false under I are assigned the probability "0". This means that the sum of probabilities of all atoms is higher than 1 (it is n times 1, where n is the cardinality of the set of atoms true under I). This makes for a non-Bayesian distribution [*]. So your simple argument sketch doesn't work. Find a better one. ______________ [*] As an aside, all this means that we assume outcomes to be analogous to atoms, events to predicates and distributions to interpretations. There may be a better mapping between concepts and some have been proposed, e.g. that assign probabilities to interpretations rather than atoms, but under FOL semantics the only thing that takes a truth value is atoms, and formulae formed of atoms, and so whatever framework comes out of such transformations has nothing to do with classical logic anymore. Btw, you can also try to assign different probabilities to atoms than 0 and 1, but then you're outside classical logic again. The fact of the matter is that logic and probabilities are incompatible and any attempt to reconcile the two is doomed to fail. You can speak the language of certainty, or the language of uncertainty, but not both at once.

naasking 3y ago
I'm going to consolidate replies to all three of your comments here and only going to address specific salient points that I think are the crux of disagreements: Re: Bayesian inference and logic To clarify, I was informally referring to the standard correspondence with propositional logic [1], but generalizations of this to include quantifiers also seem possible [2]. That said, I agree that my simple sketch is not a good justification for Solomonoff Induction, so a poor attempt at a summary on my part. Re: A "system" is not a function. I don't know a formal definition of a system, but if you're talking about whatever the human brain is doing that's likely to be more easily modelled as a Turing machine. If you agree that a system can be modelled as a function, then there's a formal equivalence. That's what "is" means to me. Certainly you can insist on a definition for "is" that's more like a small-step operational semantics to more faithfully model internals, but for the purposes of this discussion, I don't think the internal details matter if the observable inputs and outputs are the same. If a human brain "is" equivalent to some Turing machine, then I can also say that it "is" equivalent to some term in the lambda calculus. Re: If, by any chance, you think of the brain as a universal Turing machine (UTM) The brain can be described by a finite state automaton due to the Bekenstein Bound. So it's not a universal Turing machine, strictly speaking. Certainly it would be interesting to know how the internals of the brain work in minute detail, but that's ambitious. Re: They have consistently failed because the pieces of the puzzle don't fit. I'm curious specifically what doesn't fit. I understand there's tension between logic and probability, and learning and human-like reasoning are obviously still burgeoning fields, but the rest seem fairly well explored with plenty of correspondences. Re: I think in modern machine learning people have simply given up on trying to make any coherent image out of the puzzle pieces and just bashed them until they yielded and are now wedged together in incongruous patterns that sort of show some kind of image, there. Which is kinda what you'd expect from evolution by natural selection. [1] https://en.wikipedia.org/wiki/Bayes%27_theorem#Correspondence_to_other_mathematical_frameworks https://en.wikipedia.org/wiki/Bayes%27_theorem#Correspondenc... [2] https://www.lesswrong.com/posts/W8YscokXMiDnLKJ96/bayesian-inference-on-1st-order-logic https://www.lesswrong.com/posts/W8YscokXMiDnLKJ96/bayesian-i...
- YeGoblynQueenne 3y ago
  >> To clarify, I was informally referring to the standard correspondence with propositional logic [1], but generalizations of this to include quantifiers also seem possible [2]. The order of logic doesn't matter. That "standard" mapping of probabilities to logic doesn't work with classical logical semantics and with the common assumption that all probabilities sum to 1. I pointed out why in my previous comment: mapping true to 1 and false to 0 makes for too many 1's to fit in the open interval (0,1). The wikipedia article (which lacks relevant references) and the LessWrong post do not stop to address this foundational inconsistency and instead plow on with building on top of it, but they're only building a house of cards: one cannot draw safe inferences from inconsistent assumptions. As I say, I get the impression that many such attempts to marry logic and probabilities come from people who understand probabilities, but not logic. They might be considered "standard" in communities that simply wish to do away with logic and only have to think about probabilities, but they are a cop-out supported by sloppy thinking. On the other hand, there are a number of frameworks that avoid the cardinal sin of mapping true and false to 1 and 0, which they do by eschewing classical semantcis, for example Łukasiewicz logic: https://en.wikipedia.org/wiki/%C5%81ukasiewicz_logic https://en.wikipedia.org/wiki/%C5%81ukasiewicz_logic These are the obvious choice of logical semantics if one wishes to combine probability with logic, but somehow I rarely see them used. Possibly because they are not as well-known as propositional logic. In general, I propose to anyone who is interested in the subject to get at least a functional understanding of classical logic, if not as good an understanding as they have of probabilities and statistics. >> I'm curious specifically what doesn't fit. I understand there's tension between logic and probability, and learning and human-like reasoning are obviously still burgeoning fields, but the rest seem fairly well explored with plenty of correspondences. What I mean by "the pieces of the puzzle don't fit" is that the various abstractions we have to describe computation don't work when we try to apply them to human thinking. All the formal systems we have are missing something. Trying to work outside formal systems of course leads away from the realm of science and into the world of wild speculation and wishful thinking. >> Which is kinda what you'd expect from evolution by natural selection. I don't follow. Evolution by natural selection should be expected to make us lose all hope of building consistent theories?
YeGoblynQueenne 3y ago
>> Solomonoff Induction has shown that Bayesian inference will reproduce any underlying computable function given a suitable prior. Btw, when we talk of "functions" we mean a mapping between elements of two or more sets. A "system" is not a function. I don't know a formal definition of a system, but if you're talking about whatever the human brain is doing that's likely to be more easily modelled as a Turing machine. And a Turing machine is not, in itself, a function. It is a computational device that can calculate the result of some set of calculable functions. You can model a Turing machine as a function, e.g. by mapping some set of programs to the set of inputs and outputs of those programs, or something scary like that, but once again this mapping will tell you nothing about the internal operation of any specific Turing machine. If, by any chance, you think of the brain as a universal Turing machine (UTM), then I'd really like to see the function you come up with to represent its behaviour; and let's see how much this function tells you about the internal structure of the brain-UTM.
- YeGoblynQueenne 3y ago
  Further to all this I must assure you that all of the above is not some attempt at sophistry or trying to score internet points: what we discuss here goes right at the heart of my doctoral research. You can get some background on the problems I've dealt with if you search online for Inductive Logic Programming and Statistical Relational Learning. People in my research community have grappled with these questions for a long time and tried to put together the pieces of the puzzle: relations, functions, algorithms, programs, logic, probabilities, statistics, learning, reasoning... the human mind. They have consistently failed because the pieces of the puzzle don't fit. I think in modern machine learning people have simply given up on trying to make any coherent image out of the puzzle pieces and just bashed them until they yielded and are now wedged together in incongruous patterns that sort of show some kind of image, there. But the incongruities are still felt, like nails on a blackboard.

naasking 3y ago

I'm going to consolidate replies to all three of your comments here and only going to address specific salient points that I think are the crux of disagreements: Re: Bayesian inference and logic To clarify, I was informally referring to the standard correspondence with propositional logic [1], but generalizations of this to include quantifiers also seem possible [2]. That said, I agree that my simple sketch is not a good justification for Solomonoff Induction, so a poor attempt at a summary on my part. Re: A "system" is not a function. I don't know a formal definition of a system, but if you're talking about whatever the human brain is doing that's likely to be more easily modelled as a Turing machine. If you agree that a system can be modelled as a function, then there's a formal equivalence. That's what "is" means to me. Certainly you can insist on a definition for "is" that's more like a small-step operational semantics to more faithfully model internals, but for the purposes of this discussion, I don't think the internal details matter if the observable inputs and outputs are the same. If a human brain "is" equivalent to some Turing machine, then I can also say that it "is" equivalent to some term in the lambda calculus. Re: If, by any chance, you think of the brain as a universal Turing machine (UTM) The brain can be described by a finite state automaton due to the Bekenstein Bound. So it's not a universal Turing machine, strictly speaking. Certainly it would be interesting to know how the internals of the brain work in minute detail, but that's ambitious. Re: They have consistently failed because the pieces of the puzzle don't fit. I'm curious specifically what doesn't fit. I understand there's tension between logic and probability, and learning and human-like reasoning are obviously still burgeoning fields, but the rest seem fairly well explored with plenty of correspondences. Re: I think in modern machine learning people have simply given up on trying to make any coherent image out of the puzzle pieces and just bashed them until they yielded and are now wedged together in incongruous patterns that sort of show some kind of image, there. Which is kinda what you'd expect from evolution by natural selection. [1] https://en.wikipedia.org/wiki/Bayes%27_theorem#Correspondence_to_other_mathematical_frameworks https://en.wikipedia.org/wiki/Bayes%27_theorem#Correspondenc... [2] https://www.lesswrong.com/posts/W8YscokXMiDnLKJ96/bayesian-inference-on-1st-order-logic https://www.lesswrong.com/posts/W8YscokXMiDnLKJ96/bayesian-i...

YeGoblynQueenne 3y ago
>> To clarify, I was informally referring to the standard correspondence with propositional logic [1], but generalizations of this to include quantifiers also seem possible [2]. The order of logic doesn't matter. That "standard" mapping of probabilities to logic doesn't work with classical logical semantics and with the common assumption that all probabilities sum to 1. I pointed out why in my previous comment: mapping true to 1 and false to 0 makes for too many 1's to fit in the open interval (0,1). The wikipedia article (which lacks relevant references) and the LessWrong post do not stop to address this foundational inconsistency and instead plow on with building on top of it, but they're only building a house of cards: one cannot draw safe inferences from inconsistent assumptions. As I say, I get the impression that many such attempts to marry logic and probabilities come from people who understand probabilities, but not logic. They might be considered "standard" in communities that simply wish to do away with logic and only have to think about probabilities, but they are a cop-out supported by sloppy thinking. On the other hand, there are a number of frameworks that avoid the cardinal sin of mapping true and false to 1 and 0, which they do by eschewing classical semantcis, for example Łukasiewicz logic: https://en.wikipedia.org/wiki/%C5%81ukasiewicz_logic https://en.wikipedia.org/wiki/%C5%81ukasiewicz_logic These are the obvious choice of logical semantics if one wishes to combine probability with logic, but somehow I rarely see them used. Possibly because they are not as well-known as propositional logic. In general, I propose to anyone who is interested in the subject to get at least a functional understanding of classical logic, if not as good an understanding as they have of probabilities and statistics. >> I'm curious specifically what doesn't fit. I understand there's tension between logic and probability, and learning and human-like reasoning are obviously still burgeoning fields, but the rest seem fairly well explored with plenty of correspondences. What I mean by "the pieces of the puzzle don't fit" is that the various abstractions we have to describe computation don't work when we try to apply them to human thinking. All the formal systems we have are missing something. Trying to work outside formal systems of course leads away from the realm of science and into the world of wild speculation and wishful thinking. >> Which is kinda what you'd expect from evolution by natural selection. I don't follow. Evolution by natural selection should be expected to make us lose all hope of building consistent theories?

YeGoblynQueenne 3y ago

>> Solomonoff Induction has shown that Bayesian inference will reproduce any underlying computable function given a suitable prior. Btw, when we talk of "functions" we mean a mapping between elements of two or more sets. A "system" is not a function. I don't know a formal definition of a system, but if you're talking about whatever the human brain is doing that's likely to be more easily modelled as a Turing machine. And a Turing machine is not, in itself, a function. It is a computational device that can calculate the result of some set of calculable functions. You can model a Turing machine as a function, e.g. by mapping some set of programs to the set of inputs and outputs of those programs, or something scary like that, but once again this mapping will tell you nothing about the internal operation of any specific Turing machine. If, by any chance, you think of the brain as a universal Turing machine (UTM), then I'd really like to see the function you come up with to represent its behaviour; and let's see how much this function tells you about the internal structure of the brain-UTM.

YeGoblynQueenne 3y ago
Further to all this I must assure you that all of the above is not some attempt at sophistry or trying to score internet points: what we discuss here goes right at the heart of my doctoral research. You can get some background on the problems I've dealt with if you search online for Inductive Logic Programming and Statistical Relational Learning. People in my research community have grappled with these questions for a long time and tried to put together the pieces of the puzzle: relations, functions, algorithms, programs, logic, probabilities, statistics, learning, reasoning... the human mind. They have consistently failed because the pieces of the puzzle don't fit. I think in modern machine learning people have simply given up on trying to make any coherent image out of the puzzle pieces and just bashed them until they yielded and are now wedged together in incongruous patterns that sort of show some kind of image, there. But the incongruities are still felt, like nails on a blackboard.