3 ms·

>"I wonder what attracts internet people to Yudkowsky's brand of crackpottery, and specifically, what about Bayes elicits their quasi-religious awe." I think i

>"I wonder what attracts internet people to Yudkowsky's brand of crackpottery, and specifically, what about Bayes elicits their quasi-religious awe." I think it is because a lot of people think the human brain operates on Bayesian logic. Thus maybe the 'quasi-religious', because it is seen as a clue to how we are conscious. The human 'spark' is really math, and it is Bayes. So when talking about human consciousness, it can start to sound maybe a little 'religious' in terminology. I'm reading Anil Seth at moment, and it can be a compelling argument.

trashtester 3y ago
I think there may be reasons to think that human minds operate by something resembling Bayesian logic. However, it appears that we do tend to round some priors to either 1 or 0. And it's specifically when we do that that we start to behave religously. The next time somebody claims that the human brain operates according to bayesian logic, simply ask them what prior probability they (currently, until the next update) would assign to that claim.
- notaurus 3y ago
  > The next time somebody claims that the human brain operates according to bayesian logic, simply ask them what prior probability they (currently, until the next update) would assign to that claim. A = a person is able to (correctly or not) quantify their priors B = brain operates according to Bayesian logic P( A ) ≈ P( A | B )
  - trashtester 3y ago
    So you're saying that P(A) is almost independent of B, as in P( A ) ≈ P( A | B ) ≈ P( A | ¬B ) ? I suppose my argument is that if someone is using priors that are either a hard 1 or a hard 0, they've removed themselves from the ability to use Bayesian logic at all in any situation where data points in the opposite direction of their priors. While you can still call it "Bayesian" if you insert a prior of 0, I think such an argument is a direct contradiction of the purpose of using Bayesian logic. In other words, I would argue that P( ¬A | B ) ≈ 0. Refusual to admit a prior greater than a hard 0 is not compatible with Bayesian logic. Prior probabilities of exactly 0 should be seens as outside of the valid domain within Bayesian logic under most circustances.
    - FrustratedMonky 3y ago
      Maybe we are talking at different levels. The people theorizing that the brain is Bayesian are not saying that humans do it consciously. Like they can examine priors and making decisions. It is just that the neurons in the brain update in a way that can be modeled roughly as Bayesian. It happens without us 'deciding to do it', it is just how the brain updates to process the environment. It is happening continually, as we take in senses, and update our internal model.
      - trashtester 3y ago
        Sure. I understand that part. 99% of humans wouldn't even be able to do it consciously if they tried, so this has to operate on the subconscious level, to the extent that it's an accurate model. But if you start out with randomized priors, you will never reach 1 or 0 regardless of how much data you expose a bayesian system to. Humans, on the other hand, tend to fall into treating a probability as either 0 or 1 quite rapidly. For this step in particular, there seems to be something like rounding or L1 regularization going on. Then, once they're stuck in 0 or 1 priors, people often revert to using evaluation similar to Bayes' Theorem again, but in that case the priors can no longer be updated (except through something like a psychological shock, hallucinogen etc). But, as stated above, you cannot really reach such priors using Bayes' Theorem alone (if we assume the priors are not provided by genes or something that happens before the learning).
        FrustratedMonky 3y ago
        Along these lines, what is happening when someone walks into a room and sees a snake, but then does a double take and sees that it is a rope. It seems humans do have miss-identifying visions. Where they categorize something fast but incorrectly, and then might need 'help/time/shock', to kick start re-categorizing. To re-see it again. Or Like when seeing something for the very first time and they are 'befuddled', can't grasp it. Maybe like Bayes is having to iterate on it much longer. I'm curious since I'm not all that familiar with Bayes. Is this what you are talking about with 1 or 0. People do make very rapid judgments, then settle on an a 'view'. Then it can take something to make them re-adjust.
PaulHoule 3y ago
I've looked at "what went wrong" with the symbolic A.I. of the 1980s in the (probably vain) hope that it can be reconstituted, which is why I put so much energy into the semantic web. One problem is that practical reasoning is fundamentally probabilistic and nobody ever found a way to graft this onto logic programming or production rules that was completely satisfactory. For instance, a medical diagnosis system is trying to estimate the probability distribution of a hidden variable (the diagnosis) which is contingent on other hidden variables which are influenced by the signs and symptoms that you have observed and will be influenced by additional observations that you attempt to make, test orders, etc. Bayesian networks, despite having some problems, are the most realistic approach to that problem and they really do work if the causal structure is fixed and you know what it is. Thus I got interested in Bayesian statistics not because they are a model for human or animal thinking but because they are a model for things that can do some of the tasks humans do (particularly those that "expert systems" can do and in turning natural language into facts) You could think of a task like "is this a picture of a dog?" as being a problem of sampling a probability distribution over a 3,000,000 dimensional space if you're doing it for a 1,000x1,000 pixel image. Done that naive way the problem looks completely impossible but the triumph of machine learning, particularly deep learning, is it can make a guess at that probability distribution given a fraction of the data it would take to sample it directly.

trashtester 3y ago

I think there may be reasons to think that human minds operate by something resembling Bayesian logic. However, it appears that we do tend to round some priors to either 1 or 0. And it's specifically when we do that that we start to behave religously. The next time somebody claims that the human brain operates according to bayesian logic, simply ask them what prior probability they (currently, until the next update) would assign to that claim.

notaurus 3y ago
> The next time somebody claims that the human brain operates according to bayesian logic, simply ask them what prior probability they (currently, until the next update) would assign to that claim. A = a person is able to (correctly or not) quantify their priors B = brain operates according to Bayesian logic P( A ) ≈ P( A | B )
- trashtester 3y ago
  So you're saying that P(A) is almost independent of B, as in P( A ) ≈ P( A | B ) ≈ P( A | ¬B ) ? I suppose my argument is that if someone is using priors that are either a hard 1 or a hard 0, they've removed themselves from the ability to use Bayesian logic at all in any situation where data points in the opposite direction of their priors. While you can still call it "Bayesian" if you insert a prior of 0, I think such an argument is a direct contradiction of the purpose of using Bayesian logic. In other words, I would argue that P( ¬A | B ) ≈ 0. Refusual to admit a prior greater than a hard 0 is not compatible with Bayesian logic. Prior probabilities of exactly 0 should be seens as outside of the valid domain within Bayesian logic under most circustances.
  - FrustratedMonky 3y ago
    Maybe we are talking at different levels. The people theorizing that the brain is Bayesian are not saying that humans do it consciously. Like they can examine priors and making decisions. It is just that the neurons in the brain update in a way that can be modeled roughly as Bayesian. It happens without us 'deciding to do it', it is just how the brain updates to process the environment. It is happening continually, as we take in senses, and update our internal model.
    - trashtester 3y ago
      Sure. I understand that part. 99% of humans wouldn't even be able to do it consciously if they tried, so this has to operate on the subconscious level, to the extent that it's an accurate model. But if you start out with randomized priors, you will never reach 1 or 0 regardless of how much data you expose a bayesian system to. Humans, on the other hand, tend to fall into treating a probability as either 0 or 1 quite rapidly. For this step in particular, there seems to be something like rounding or L1 regularization going on. Then, once they're stuck in 0 or 1 priors, people often revert to using evaluation similar to Bayes' Theorem again, but in that case the priors can no longer be updated (except through something like a psychological shock, hallucinogen etc). But, as stated above, you cannot really reach such priors using Bayes' Theorem alone (if we assume the priors are not provided by genes or something that happens before the learning).
      - FrustratedMonky 3y ago
        Along these lines, what is happening when someone walks into a room and sees a snake, but then does a double take and sees that it is a rope. It seems humans do have miss-identifying visions. Where they categorize something fast but incorrectly, and then might need 'help/time/shock', to kick start re-categorizing. To re-see it again. Or Like when seeing something for the very first time and they are 'befuddled', can't grasp it. Maybe like Bayes is having to iterate on it much longer. I'm curious since I'm not all that familiar with Bayes. Is this what you are talking about with 1 or 0. People do make very rapid judgments, then settle on an a 'view'. Then it can take something to make them re-adjust.

PaulHoule 3y ago

I've looked at "what went wrong" with the symbolic A.I. of the 1980s in the (probably vain) hope that it can be reconstituted, which is why I put so much energy into the semantic web. One problem is that practical reasoning is fundamentally probabilistic and nobody ever found a way to graft this onto logic programming or production rules that was completely satisfactory. For instance, a medical diagnosis system is trying to estimate the probability distribution of a hidden variable (the diagnosis) which is contingent on other hidden variables which are influenced by the signs and symptoms that you have observed and will be influenced by additional observations that you attempt to make, test orders, etc. Bayesian networks, despite having some problems, are the most realistic approach to that problem and they really do work if the causal structure is fixed and you know what it is. Thus I got interested in Bayesian statistics not because they are a model for human or animal thinking but because they are a model for things that can do some of the tasks humans do (particularly those that "expert systems" can do and in turning natural language into facts) You could think of a task like "is this a picture of a dog?" as being a problem of sampling a probability distribution over a 3,000,000 dimensional space if you're doing it for a 1,000x1,000 pixel image. Done that naive way the problem looks completely impossible but the triumph of machine learning, particularly deep learning, is it can make a guess at that probability distribution given a fraction of the data it would take to sample it directly.