5 ms·

> who try really hard to reason properly Who are really big into talking about reasoning with formal tools which few of them make any effort to understand or u

by nullc 2y ago

> who try really hard to reason properly Who are really big into talking about reasoning with formal tools which few of them make any effort to understand or use. They might as well "try really hard to reason properly" with crystals and burning sage. I think actually I'd trust the results of a crystal-reasoner more... at least they're more likely to have a bit of self-doubt that the crystals are all nonsense. Go ask one of them -- one without immediate access to an LLM-- what the conjugiate prior is for observations of a binomial trial (or the same question in lay language). You'll get blank stares, yet this should be a bread and butter question for exactly the kind of reasoning they (claim to) advance. There is a level of mathematical reasoning that basically anyone can learn, such as multiplying payoff matrices with probabilities and getting expected values. But going beyond that (e.g. adding uncertainties and correlations, or reasoning backwards from observations to model parameters) being extremely mathematically complex very fast, and rapidly unreasonable to apply to most real situations. Even if you're comfortable with the required graduate level mathematics it just takes too much time. Maybe someone could make a handbook of shortcuts to do the math for many common cases, it's the sort of thing you might expect a "rationalist group" to do, but instead they spend their time making excuses for fringe edgelord politics disguised with a bunch of inapplicable formal language. Gotta eugenics the plebs for their own good and if you don't agree it must be because you reject MATH. They use bayesian this and that as magic words and to put on a show of intellectual superiority. But for the most part they can't and don't use these techniques, and for perfectly good reasons-- too hard, pointless without better data, etc. But they endlessly talk about them. The whole community is a mix of larpers and people who feel inferior because they don't know the other people are larping. In their original form, largely harmless larpers... other than the rejection of "cognitive biases" resulting in occasionally promoting gonzo policies and grifting/embezzling nonprofit monies (E.g. SIAI). But as of recent years they've metastasized into an anti-AI apocalypse cult. Their founder has been out in the media advocating the use of nuclear weapons against civilian populations to suppress AI development, so we should just be entirely unsurprised that there is a radical offshoot group out there murdering people.

afiori 2y ago
My impression is quite different, not in the sense that your comment is incorrect but that the > > Who try really hard to reason properly is not in my opinion the distinctive element of the group. I think that the parent would have been more correct had he written > who try really hard to apply reasoning to areas people often don't reason about. As this implies an entirely different class of actions.
- nullc 2y ago
  Hm. So I think if you define reasoning broadly -- as in collecting relevant data, enumerating and raking priorities, thinking through expected consequences... people do engage in this quite often in all areas of life, far more often than lesswrongers would credit. If you define it more narrowly, as in using particular techniques that are popular to talk about in that community like bayes theorem, updating priors and so on-- then indeed it's not common but it's also not practiced by practically any of the community members. I would say that there is a mix of both definitions in use in those communities, but for the former people already do it often and for the latter not even the lesswrongers do (with a few exceptions).
empath75 2y ago
The fundamental problem with rationalism and just using pure reason and logic in general is that logic has no (necessary) connection to reality. It's a game played with symbols, and how you choose to map concepts and things in reality to those symbols determines your conclusion. There's a lot of room to setup your predicates in a way that justifies any conclusion you care to reach, no matter how monstrous, and there are lots of ways to turn remote probabilities into near-certainties. Frequently where they go wrong is injecting infinities or near-infinities into their calculations and once you put the lives of billions of people up against one person almost anything is justified.

by nullc 2y ago

afiori 2y ago

My impression is quite different, not in the sense that your comment is incorrect but that the > > Who try really hard to reason properly is not in my opinion the distinctive element of the group. I think that the parent would have been more correct had he written > who try really hard to apply reasoning to areas people often don't reason about. As this implies an entirely different class of actions.

nullc 2y ago
Hm. So I think if you define reasoning broadly -- as in collecting relevant data, enumerating and raking priorities, thinking through expected consequences... people do engage in this quite often in all areas of life, far more often than lesswrongers would credit. If you define it more narrowly, as in using particular techniques that are popular to talk about in that community like bayes theorem, updating priors and so on-- then indeed it's not common but it's also not practiced by practically any of the community members. I would say that there is a mix of both definitions in use in those communities, but for the former people already do it often and for the latter not even the lesswrongers do (with a few exceptions).

empath75 2y ago

The fundamental problem with rationalism and just using pure reason and logic in general is that logic has no (necessary) connection to reality. It's a game played with symbols, and how you choose to map concepts and things in reality to those symbols determines your conclusion. There's a lot of room to setup your predicates in a way that justifies any conclusion you care to reach, no matter how monstrous, and there are lots of ways to turn remote probabilities into near-certainties. Frequently where they go wrong is injecting infinities or near-infinities into their calculations and once you put the lives of billions of people up against one person almost anything is justified.