5 ms·

>untrue magical powers to Bayes' theorem, probability theory, computers Untrue magical powers? Just because people appreciate those things it doesn't mean they

by Tenoke 4y ago

>untrue magical powers to Bayes' theorem, probability theory, computers Untrue magical powers? Just because people appreciate those things it doesn't mean they ascribe magical powers to them. That's like saying that HN ascribes magical powers to startups or chip architectures.

astrange 4y ago
Well they're rationalists, so they ascribe it magical rationality powers. They invoke Bayes' theorem for its property of being optimal, but it's optimal (and trivial) when you know the priors. The hard part is knowing those, and they don't, so they just use it in a metaphorical/mindset way that doesn't actually mean anything, pretend they don't have actual beliefs so they don't need to take responsibility for them, and say "prior" a lot. (https://metarationality.com/bayesianism-updating https://metarationality.com/bayesianism-updating) In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures, which is why they're overly concerned with imaginary future AIs torturing them, and it ignores "unknown unknowns" and can only handle enumerated futures you already thought of, which is why they have that odd tendency to start polycules because it seems logical and then their girlfriends unexpectedly leave them. (That's also how you know it's a Berkeley new religion - weird sex practices.)
- Tenoke 4y ago
  You are the only person ascribing it magical powers. The majority definitely do not and are well aware of the limitations, the problem with priors is talked about all the time in the community in much more detail. >In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures Another thing frequently discussed in those circles. >(That's also how you know it's a Berkeley new religion - weird sex practices.) Some people being sexually adventurous in California, the horror.
  - astrange 4y ago
    > Another thing frequently discussed in those circles. That’s the problem, they shouldn’t be discussing it ;) > Some people being sexually adventurous in California, the horror. I could’ve said “joining cults like Leverage” instead of polycules but that would be mean. However like I did say, the problem with being a young man who gets into an open relationship because you rationally determined it was optimal is, you’re wrong and your girlfriend is going to leave you. If you’re going to have weird sex, at least have a more fun reason for it. (There’s a tweet I want to link here about a rationalist nude sex party where they all sat around discussing immigration politics instead of having said adventures, but unfortunately I can’t remember how to spell Aeolla to find it.)
    - PebblesRox 4y ago
      https://mobile.twitter.com/Aella_Girl/status/1375892593377112065 https://mobile.twitter.com/Aella_Girl/status/137589259337711...
- FeepingCreature 4y ago
  "you know the priors" - this is just fundamentally not how probabilistic reasoning works. You have some priors, which are of unclear accuracy, and then you iteratively update your beliefs on the basis of evidence and (probably) get, if not the right idea, then at least one that is less wrong. Hence the title! If you had the right priors to start, you wouldn't need probability theory because you'd already know everything. > it can't reject infinitesimally likely but infinitely bad futures Sure it can, because that's value theory, not probability theory.
  - astrange 4y ago
    You don’t have priors or evidence - those are logical concepts that don’t exist in our fuzzy not-so-logical real world. You have some starting beliefs called priors, some new beliefs called evidence, and metaphorically applying one to the other causes you to be overly proud of the result because you feel like a rational process did it instead of your brain. As the page I linked says, the real lesson is “don’t be too sure of stuff” but that doesn’t require pretending you’re doing math or joining any religions.
    - FeepingCreature 4y ago
      You seem to have some confused beliefs about what LessWrong people actually think. Theories like bayesian neurons aside, nobody thinks or advocates that people actually do bayesian updates as part of every thought. The point is that inasmuch as human thinking diverges from the formal math of probability, it will be less accurate - that there is such a thing as an idealized thought process, that thinking and learning has laws that can be understood and applied. That human minds aren't coincidental; that there are reasons why they work the way they do.
      - jerf 4y ago
        I've clocked some time online with the "rationalists". I have learned some interesting things. But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!" I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion. Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)

by Tenoke 4y ago

astrange 4y ago

Well they're rationalists, so they ascribe it magical rationality powers. They invoke Bayes' theorem for its property of being optimal, but it's optimal (and trivial) when you know the priors. The hard part is knowing those, and they don't, so they just use it in a metaphorical/mindset way that doesn't actually mean anything, pretend they don't have actual beliefs so they don't need to take responsibility for them, and say "prior" a lot. (https://metarationality.com/bayesianism-updating https://metarationality.com/bayesianism-updating) In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures, which is why they're overly concerned with imaginary future AIs torturing them, and it ignores "unknown unknowns" and can only handle enumerated futures you already thought of, which is why they have that odd tendency to start polycules because it seems logical and then their girlfriends unexpectedly leave them. (That's also how you know it's a Berkeley new religion - weird sex practices.)

Tenoke 4y ago
You are the only person ascribing it magical powers. The majority definitely do not and are well aware of the limitations, the problem with priors is talked about all the time in the community in much more detail. >In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures Another thing frequently discussed in those circles. >(That's also how you know it's a Berkeley new religion - weird sex practices.) Some people being sexually adventurous in California, the horror.
- astrange 4y ago
  > Another thing frequently discussed in those circles. That’s the problem, they shouldn’t be discussing it ;) > Some people being sexually adventurous in California, the horror. I could’ve said “joining cults like Leverage” instead of polycules but that would be mean. However like I did say, the problem with being a young man who gets into an open relationship because you rationally determined it was optimal is, you’re wrong and your girlfriend is going to leave you. If you’re going to have weird sex, at least have a more fun reason for it. (There’s a tweet I want to link here about a rationalist nude sex party where they all sat around discussing immigration politics instead of having said adventures, but unfortunately I can’t remember how to spell Aeolla to find it.)
  - PebblesRox 4y ago
    https://mobile.twitter.com/Aella_Girl/status/1375892593377112065 https://mobile.twitter.com/Aella_Girl/status/137589259337711...
FeepingCreature 4y ago
"you know the priors" - this is just fundamentally not how probabilistic reasoning works. You have some priors, which are of unclear accuracy, and then you iteratively update your beliefs on the basis of evidence and (probably) get, if not the right idea, then at least one that is less wrong. Hence the title! If you had the right priors to start, you wouldn't need probability theory because you'd already know everything. > it can't reject infinitesimally likely but infinitely bad futures Sure it can, because that's value theory, not probability theory.
- astrange 4y ago
  You don’t have priors or evidence - those are logical concepts that don’t exist in our fuzzy not-so-logical real world. You have some starting beliefs called priors, some new beliefs called evidence, and metaphorically applying one to the other causes you to be overly proud of the result because you feel like a rational process did it instead of your brain. As the page I linked says, the real lesson is “don’t be too sure of stuff” but that doesn’t require pretending you’re doing math or joining any religions.
  - FeepingCreature 4y ago
    You seem to have some confused beliefs about what LessWrong people actually think. Theories like bayesian neurons aside, nobody thinks or advocates that people actually do bayesian updates as part of every thought. The point is that inasmuch as human thinking diverges from the formal math of probability, it will be less accurate - that there is such a thing as an idealized thought process, that thinking and learning has laws that can be understood and applied. That human minds aren't coincidental; that there are reasons why they work the way they do.
    - jerf 4y ago
      I've clocked some time online with the "rationalists". I have learned some interesting things. But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!" I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion. Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)