3 ms·

You seem to have some confused beliefs about what LessWrong people actually think. Theories like bayesian neurons aside, nobody thinks or advocates that people

You seem to have some confused beliefs about what LessWrong people actually think. Theories like bayesian neurons aside, nobody thinks or advocates that people actually do bayesian updates as part of every thought. The point is that inasmuch as human thinking diverges from the formal math of probability, it will be less accurate - that there is such a thing as an idealized thought process, that thinking and learning has laws that can be understood and applied. That human minds aren't coincidental; that there are reasons why they work the way they do.

jerf 4y ago
I've clocked some time online with the "rationalists". I have learned some interesting things. But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!" I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion. Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)
- FeepingCreature 4y ago
  I agree, I just think that the way in which LessWrong goes wrong is largely an entirely different way in which the parent depicts us as going wrong. :P > improving rationality will lead to a lot of massively better life outcomes IMO, to defend this, I'd argue for most people there's maybe oom of ten decisions over a lifetime that in hindsight could turn out to be really important, and if you can keep a calm head and think about it in those moments, you can potentially get a lot of benefits. Now usually there'll be lots of advice for those, and the "rational" (in the advertised, not actual, sense) answer is not always the best one, but even just stepping back, thinking about it, trying to find out how to gather evidence about it, can already possibly help out a ton. Most people get by fine doing well enough. And it doesn't take rationality to tell you not to invest your savings in scams, so the upside is limited. But for those few times where it really matters, there are some genuinely useful tools in there, IMO. (Though personally, I'm in it for the philosophy.)
astrange 4y ago
I agree with your description of (your?) beliefs, so if I implied anything else then I expressed myself poorly. Unfortunately there isn't such a thing as an ideal thought process - that is, if you want to make a correct prediction about the world, there is no demonstratably correct method of doing that. This sort of thing has been tried before (logical positivism) and disproven (Wittgenstein and Gödel's theorem). It's notable that AI research is mostly getting places by just doing gigantic matrix multiplications instead of this. It can be a useful exercise but it's not a good life philosophy - it's a bit robotic. And it gets cult-like if you add things on like "…and that's why you should live in our group home" or "…and that's why you should get rich and donate it all to evil AGI research". This isn't very important to me though, it's just something I've noticed. It seem to come up a lot more lately because people reply to every ML story with "oh no the AIs are going to take our jobs" stuff, but still the largest impact it's had on the real world is Roko causing Elon Musk to date Grimes. I think I meet more people who've developed life philosophies in reaction to knowing about LW ("post rats" or "meta rats") - mostly they explicitly decided to join religions instead of accidentally doing it - and for some reason all of them became Buddhists. That one surprises me.
- FeepingCreature 4y ago
  > there is no demonstratably correct method of doing that AIXI has some correctness proofs. In an admittedly esoteric sense, it does make "optimal use of information". I do think those parts philosophically hold up. For an interesting view on Gödel, see Garrabrant's logical induction paper; you can bypass some aspects of that by working probabilistically. > "…and that's why you should live in our group home" People try things because they think they work, and that's why LW is a cult. > "…and that's why you should get rich and donate it all to evil AGI research". People recommend things because they think they're correct, and that's why LW is a cult. Also, anti-evil AGI-research. That's kind of an important aspect. I don't know what's going on with the postrats either.

jerf 4y ago

I've clocked some time online with the "rationalists". I have learned some interesting things. But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!" I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion. Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)

FeepingCreature 4y ago
I agree, I just think that the way in which LessWrong goes wrong is largely an entirely different way in which the parent depicts us as going wrong. :P > improving rationality will lead to a lot of massively better life outcomes IMO, to defend this, I'd argue for most people there's maybe oom of ten decisions over a lifetime that in hindsight could turn out to be really important, and if you can keep a calm head and think about it in those moments, you can potentially get a lot of benefits. Now usually there'll be lots of advice for those, and the "rational" (in the advertised, not actual, sense) answer is not always the best one, but even just stepping back, thinking about it, trying to find out how to gather evidence about it, can already possibly help out a ton. Most people get by fine doing well enough. And it doesn't take rationality to tell you not to invest your savings in scams, so the upside is limited. But for those few times where it really matters, there are some genuinely useful tools in there, IMO. (Though personally, I'm in it for the philosophy.)

astrange 4y ago

I agree with your description of (your?) beliefs, so if I implied anything else then I expressed myself poorly. Unfortunately there isn't such a thing as an ideal thought process - that is, if you want to make a correct prediction about the world, there is no demonstratably correct method of doing that. This sort of thing has been tried before (logical positivism) and disproven (Wittgenstein and Gödel's theorem). It's notable that AI research is mostly getting places by just doing gigantic matrix multiplications instead of this. It can be a useful exercise but it's not a good life philosophy - it's a bit robotic. And it gets cult-like if you add things on like "…and that's why you should live in our group home" or "…and that's why you should get rich and donate it all to evil AGI research". This isn't very important to me though, it's just something I've noticed. It seem to come up a lot more lately because people reply to every ML story with "oh no the AIs are going to take our jobs" stuff, but still the largest impact it's had on the real world is Roko causing Elon Musk to date Grimes. I think I meet more people who've developed life philosophies in reaction to knowing about LW ("post rats" or "meta rats") - mostly they explicitly decided to join religions instead of accidentally doing it - and for some reason all of them became Buddhists. That one surprises me.

FeepingCreature 4y ago
> there is no demonstratably correct method of doing that AIXI has some correctness proofs. In an admittedly esoteric sense, it does make "optimal use of information". I do think those parts philosophically hold up. For an interesting view on Gödel, see Garrabrant's logical induction paper; you can bypass some aspects of that by working probabilistically. > "…and that's why you should live in our group home" People try things because they think they work, and that's why LW is a cult. > "…and that's why you should get rich and donate it all to evil AGI research". People recommend things because they think they're correct, and that's why LW is a cult. Also, anti-evil AGI-research. That's kind of an important aspect. I don't know what's going on with the postrats either.