18 ms·

They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclus

They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value. There is a term for this. "Getting stuck up your own butt." It wouldn't be so bad except that said people often take on an air of absolute superiority because they used "only logic" and in their head they can not be wrong. Many people end up thinking like this as teenagers or 20 somethings, but most will have someone in their life who smacks them over the head and tells them to stop being so foolish, but if you have enough money and the Internet you can insulate yourself from that kind of oversight.

OtherShrezzing 1y ago
They _are_ the effective altruism crowd.
troyastorino 1y ago
The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current people, and that the number of future people vastly outweighs everyone alive today (or who has ever lived), then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.
- jdmichal 1y ago
  I'm not familiar with any of these communities. Is there also a general bias towards one side between "the most important thing gets the *most* resources" and "the most important thing gets *all* the resources"? Or, in other words, the most important thing is the only important thing? IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)
  - nearbuy 1y ago
    It's not the case. Effective altruists give to dozens of different causes, such as malaria prevention, environmentalism, animal welfare, and (perhaps most controversially) extinction risk. It can't tell you which root values to care about. It just asks you to consider whether the charity is impactful. Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values. It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
  - JoshuaDavid 1y ago
    The general idea is that on the margin (in the economics sense), more resources should go to the most effective+neglected thing, and.the amount of resources I control is approximately zero in a global sense, so I personally should direct all of my personal giving to the highest impact thing.
    - skinnymuch 1y ago
      And in their logic the highest impact is to donate money, take high paying jobs regardless of morality, and not focusing on any structural or root issues.
      - 1y ago
TimTheTinker 1y ago
> their initial conditions were highly artificial There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation).
- nyeah 1y ago
  Yes, there's a name for it. They're dumbasses. “[...] Clevinger was one of those people with lots of intelligence and no brains, and everyone knew it except those who soon found it out. In short, he was a dope." - Joseph Heller, Catch-22 https://www.goodreads.com/quotes/7522733-in-short-clevinger-was-one-of-those-people-with-lots https://www.goodreads.com/quotes/7522733-in-short-clevinger-...
  - TeMPOraL 1y ago
    You mean people projecting this problem onto them with zero evidence beyond their own preconceptions and an occasional stereotype they read online? HN should be better than this.
    - nyeah 1y ago
      Let's read the thread: "There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation)." Can people suffer from that impairment? Is that possible? If not, please explain how wrong assumptions can be eliminated without actively looking for them. If the impairment is real, what would you call its victims? Pick your own terminology.
UncleOxidant 1y ago
> They remind me of the "Effective Altruism" crowd Isn't there a lot of overlap between the two groups? I recently read a great book that examines these various groups and their commonality: More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity by Adam Becker. Highly recommended.
HPsquared 1y ago
People who confuse the map for the territory.
noname120 1y ago
> They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value. Do you have examples of that? I have a different perception, most of the EAs I've met are very grounded and sharp. For example the most recent issue of their newsletter: https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f799cca137ef74763&id=49c2ab45b2 https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f79... I'm not sure where there are any “hypothetical logical thought exercises” that “end up coming to insane conclusions” in there. For the first part where you say “not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value” this is quite the opposite of my experience with them. They are very receptive to criticism and reconsider their point of view in reaction to that. They are generally well-aware of the limits of data-driven initiatives and the dangers of indulging into purely abstract thinking that can lead to conclusions that indeed don't make sense.
- jandrese 1y ago
  One example is Newcomb's problem. It presupposes a ridiculous scenario where a godlike being acts irrationally and then people try to base their life around "winning" the game that will never ever happen to them.
- jhbadger 1y ago
  As Adam Becker shows in his book, EAs started out being reasonable "give to charity as much as you can, and research which charities do the most good" but have gotten into absurdities like "it is more important to fund rockets than help starving people or prevent malaria because maybe an asteroid will hit the Earth, killing everyone, starving or not".
  - LargeWu 1y ago
    It's also not a very big leap from "My purpose is to do whatever is the greatest good" to "It doesn't matter if I hurt people as long as the overall net result is good (by some arbitrary standard)"
cassepipe 1y ago
I read the whole tree of responses under this comment and I could only convince myself that when people have no arguments they try to make you look bad. Most of criticisms are just "But they think they are better than us !" and the rest is "But sometimes they are wrong !" I don't know about the community and couldn't care less but their writings have brought me some almost life saving fresh air in how to think about the world. It is very sad to me to read so many falsely elaborate responses from supposedly intelligent people having their ego hurt but in the end it reminds me why I like rationalists and I don't like most people.
- ajkjk 1y ago
  Here's a theory of what's happening, both with you here in this comment section and with the rationalists in general. Humans are generally better at perceiving threats than they are at putting those threats into words. When something seems "dangerous" abstractly, they will come up with words for why---but those words don't necessarily reflect the actual threat, because the threat might be hard to describe. Nevertheless the valence of their response reflects their actual emotion on the subject. In this case: the rationalist philosophy basically creeps people out. There is something "insidious" about it. And this is not a delusion on the part of the people judging them: it really does threaten them, and likely for good reason. The explanation is something like "we extrapolate from the way that rationalists think and realize that their philosophy leads to dangerous conclusions." Some of these conclusions have already been made by the rationalists---like valuing people far away abstractly over people next door, by trying to quantify suffering and altruism like a math problem (or to place moral weight on animals over humans, or people in the future over people today). Other conclusions are just implied, waiting to be made later. But the human mind detects them anyway as implications of the way of thinking, and reacts accordingly: thinking like this is dangerous and should be argued against. This extrapolation is hard to put into words, so everyone who tries to express their discomfort misses the target somewhat, and then, if you are the sort of person who only takes things literally, it sounds like they are all just attacking someone out of judgment or bitterness or something instead of for real reasons. But I can't emphasize this enough: their emotions are real, they're just failing to put them into words effectively. It's a skill issue. You will understand what's happening better if you understand that this is what's going on and then try to take their emotions seriously even if they are not communicating them very well. So that's what's going on here. But I think I can also do a decent job of describing the actual problem that people have with the rationalist mindset. It's something like this: Humans have an innate moral intuition that "personal" morality, the kind that takes care of themselves and their family and friends and community, is supposed to be sacrosanct: people are supposed to both practice it and protect the necessity of practicing it. We simply can't trust the world to be a safe place if people don't think of looking out for the people around them as a fundamental moral duty. And once those people are safe, protecting more people, such as a tribe or a nation or all of humanity or all of the planet, becomes permissible. Sometimes people don't or can't practice this protection for various reasons, and that's morally fine, because it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it as a better way to live: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors; or, it's better to protect animals than people, because there are more of them". It's fine to work on important far-away problems once local problems are solved, if that's what you want. But it can't take priority, regardless of how the math works out. To work on global numbers-game problems instead of local problems, and to justify that with arguments, and to try to convince other people to also do that---that's dangerous as hell. It proves too much: it argues that humans at large ought to dismantle their personal moralities in favor of processing the world like a paperclip-maximizing robot. And that is exactly as dangerous as a paperclip-maximizing robot is. Just at a slower timescale. (No surprise that this movement is popular among social outcasts, for whom local morality is going to feel less important, and (I suspect) autistic people, who probably experience less direct moral empathy for the people around them, as well as to the economically-insulated well-to-do tech-nerd types who are less likely to be directly exposed to suffering in their immediate communities.) Ironically paperclip-maximizing-robots are exactly the thing that the rationalists are so worried about. They are a group of people who missed, and then disavowed, and now advocate disavowing, this "personal" morality, and unsurprisingly they view the world in a lens that doesn't include it, which means mostly being worried about problems of the same sort. But it provokes a strong negative reaction from everyone who thinks about the world in terms of that personal duty to safety, because that is the foundation of all morality, and is utterly essential to preserve, because it makes sure that whatever else you are doing doesn't go awry. (edit: let me add that your aversion to the criticisms of rationalists is not unreasonable either. Given that you're parsing the criticisms as unreasonable, which they likely are (because of the skill issue), what you're seeing is a movement with value that seems to be being unfairly attacked. And you're right, the value is actually there! But the ultimate goal here is a synthesis: to get the value of the rationalist movement but to synthesize it with the recognition of the red flags that it sets off. Ignoring either side, the value or the critique, is ultimately counterproductive: the right goal is to synthesize both into a productive middle ground. (This is the arc of philosophy; it's what philosophy is. Not re-reading Plato.) The rationalists are probably morally correct in being motivated to highly-scaling actions e.g. the purview of "Effective Altruism". They are getting attacked for what they're discarding to do that, not for caring about it in the first place.)
not_your_mentat 1y ago
The notion that our moral obligation somehow demands we reduce the suffering of wild animals in an ecosystem, living their lives as they have done since predation evolved and as they will do long after humans have ceased to be, is such a wild misunderstanding of who we are and what we are and what the universe is. I love my Bay Area friends. To quote the great Gwen Stefani, “This sh!t is bananas.”
trod1234 1y ago
Except with Effective Altruism (EA), its not pure logic. Logic requires properties of metaphysical objectivity. If you use the true meaning of words it would be called irrationality, delusion, sophism, or fallacy when such things are claimed true when in fact they are false.
deleted 1y ago
[deleted]
jhbadger 1y ago
There is a great recent book by Adam Becker "More Everything Forever" that deals with the overlapping circles of "effective altruists", "rationalists", and "accelerationists". He's not very sympathetic to the movements as he sees them as mostly rationalizing what their adherents wanted to do anyway -- funding things like rockets and AI over feeding the needy because they see the former as helping more people in the future than dealing with real problems today.
yawnxyz 1y ago
Having just moved to the Bay Area, I've met a few AI "safety researchers" who seems to come from this EA / Rationalist camp, and they all behave more like preachers than thinkers / academics / researchers. I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...
- comp_throw7 1y ago
  > I don't think any "Rationalists" I ever met would actually consider concepts like scientific method... In that case I don't think you've met any of the people under discussion.
  - yawnxyz 1y ago
    maybe, but I Stanford scientists are a different breed of rationalists, and not Rationalists
noosphr 1y ago
Incidentally a good book on logic is the best antidote to that type of thinking. Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly. I also think the ambiguity of meaning in natural language is why statistical llms are so popular with this crowd. You don't need to think about meaning and parsing. Whatever the llm assumes is the meaning is whatever the meaning is.
- Viliam1234 1y ago
  > Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly. Ironically, that reminds me of "37 Ways That Words Can Be Wrong", written by Eliezer Yudkowsky in 2008...
protocolture 1y ago
>They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic I have read Effective Altruists like that. But I also remember seeing a lot of money donated to a bunch of really decent sounding causes because someone spent 5 minutes asking themselves what they wanted their donation to maximise, decided on "Lives saved" and figured out who is doing the best at that.
sanderjd 1y ago
lol, I love that you identified the exact correct academic term for this phenomenon.
foobar_______ 1y ago
I really appreciate this comment. Now in my 30s, I was not even aware of EA or Rationalist groups in my 20s but your description of a person high on the smell of their own farts and putting logic as king was me. The world is so chaotic that it felt good and right that I could lean on something "true" like logic. It took me to the "correct" answers according to logic but very misguided answers as I see them now. Humans are driven by emotions and the world is probabilistic so relying so heavily on logic just seems so misguided now that I'm older.
- Viliam1234 1y ago
  By the way, "the world is probabilistic" is one of the key lessons of the rationality community. (Infinite certainty would require infinite amount of evidence, according to the Bayes' Theorem.) When you find someone on internet trying to assign probabilities to their beliefs, chances are you have met a rationalist.
vishnugupta 1y ago
I don’t have anything to add to the discussion except that EA was the very first thing that popped into my mind when I read Rationalists.
mlinhares 1y ago
I'd recommend listening to the behind the bastards episode on some of these folks, it is wild https://podcasts.apple.com/us/podcast/part-one-the-zizians-how-harry-potter-fanfic-inspired/id1373812661?i=1000698710498 https://podcasts.apple.com/us/podcast/part-one-the-zizians-h...
j_timberlake 1y ago
This is a perfect example of a Hacker News hot-take, you complain about their attitude while making zero reference to their actual accomplishments or failures. No one will downvote you for posting something factually incorrect, because you didn't post any facts at all. For anyone else reading, a good example of what EAs do can be seen with the GiveWell charity: https://www.givewell.org/top-charities-fund https://www.givewell.org/top-charities-fund Lots of anti-malaria and vitamin stuff (as a cheap way to save lots of lives). There are also tons of EA animal charities too, such as Humane League: https://thehumaneleague.org/our-impact https://thehumaneleague.org/our-impact
- habinero 1y ago
  A weird cult doing helpful community service doesn't cancel out the harm of them being a cult. If they want to donate to charity, they can just donate. You don't gotta make a religion out of it.
  - j_timberlake 1y ago
    Just like the other guy, you skip the part where you actually name a harm done by them, because then people could easily shoot down your comment. Or name how EAs are a religion, I haven't even seen that troll before.
maybelsyrup 1y ago
> They remind me of the "Effective Altruism" crowd Honestly thought they were the same people
- Viliam1234 1y ago
  Not all rationalists are EAs, and not all EAs are rationalists, but there are many people who are both. The very idea of using evidence and numbers to choose the optimal charity has a rationalist vibe. But of course, one can donate to an effective charity without being interested in LessWrong-style rationality. https://www.lesswrong.com/ https://www.lesswrong.com/ and https://forum.effectivealtruism.org/ https://forum.effectivealtruism.org/ run on the same software, and some people have an account at both of them, but they are generally separate web communities.
fanwood 1y ago
Absolutely agree. The whole movement is unbearably naive

OtherShrezzing 1y ago

They _are_ the effective altruism crowd.

troyastorino 1y ago

The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current people, and that the number of future people vastly outweighs everyone alive today (or who has ever lived), then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.

jdmichal 1y ago
I'm not familiar with any of these communities. Is there also a general bias towards one side between "the most important thing gets the *most* resources" and "the most important thing gets *all* the resources"? Or, in other words, the most important thing is the only important thing? IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)
- nearbuy 1y ago
  It's not the case. Effective altruists give to dozens of different causes, such as malaria prevention, environmentalism, animal welfare, and (perhaps most controversially) extinction risk. It can't tell you which root values to care about. It just asks you to consider whether the charity is impactful. Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values. It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
- JoshuaDavid 1y ago
  The general idea is that on the margin (in the economics sense), more resources should go to the most effective+neglected thing, and.the amount of resources I control is approximately zero in a global sense, so I personally should direct all of my personal giving to the highest impact thing.
  - skinnymuch 1y ago
    And in their logic the highest impact is to donate money, take high paying jobs regardless of morality, and not focusing on any structural or root issues.
    - 1y ago

TimTheTinker 1y ago

> their initial conditions were highly artificial There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation).

nyeah 1y ago
Yes, there's a name for it. They're dumbasses. “[...] Clevinger was one of those people with lots of intelligence and no brains, and everyone knew it except those who soon found it out. In short, he was a dope." - Joseph Heller, Catch-22 https://www.goodreads.com/quotes/7522733-in-short-clevinger-was-one-of-those-people-with-lots https://www.goodreads.com/quotes/7522733-in-short-clevinger-...
- TeMPOraL 1y ago
  You mean people projecting this problem onto them with zero evidence beyond their own preconceptions and an occasional stereotype they read online? HN should be better than this.
  - nyeah 1y ago
    Let's read the thread: "There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they are found to be inconsistent with sound observation and interpretation)." Can people suffer from that impairment? Is that possible? If not, please explain how wrong assumptions can be eliminated without actively looking for them. If the impairment is real, what would you call its victims? Pick your own terminology.

UncleOxidant 1y ago

> They remind me of the "Effective Altruism" crowd Isn't there a lot of overlap between the two groups? I recently read a great book that examines these various groups and their commonality: More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity by Adam Becker. Highly recommended.

HPsquared 1y ago

People who confuse the map for the territory.

noname120 1y ago

> They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value. Do you have examples of that? I have a different perception, most of the EAs I've met are very grounded and sharp. For example the most recent issue of their newsletter: https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f799cca137ef74763&id=49c2ab45b2 https://us8.campaign-archive.com/?e=7023019c13&u=52b028e7f79... I'm not sure where there are any “hypothetical logical thought exercises” that “end up coming to insane conclusions” in there. For the first part where you say “not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value” this is quite the opposite of my experience with them. They are very receptive to criticism and reconsider their point of view in reaction to that. They are generally well-aware of the limits of data-driven initiatives and the dangers of indulging into purely abstract thinking that can lead to conclusions that indeed don't make sense.

jandrese 1y ago
One example is Newcomb's problem. It presupposes a ridiculous scenario where a godlike being acts irrationally and then people try to base their life around "winning" the game that will never ever happen to them.
jhbadger 1y ago
As Adam Becker shows in his book, EAs started out being reasonable "give to charity as much as you can, and research which charities do the most good" but have gotten into absurdities like "it is more important to fund rockets than help starving people or prevent malaria because maybe an asteroid will hit the Earth, killing everyone, starving or not".
- LargeWu 1y ago
  It's also not a very big leap from "My purpose is to do whatever is the greatest good" to "It doesn't matter if I hurt people as long as the overall net result is good (by some arbitrary standard)"

cassepipe 1y ago

I read the whole tree of responses under this comment and I could only convince myself that when people have no arguments they try to make you look bad. Most of criticisms are just "But they think they are better than us !" and the rest is "But sometimes they are wrong !" I don't know about the community and couldn't care less but their writings have brought me some almost life saving fresh air in how to think about the world. It is very sad to me to read so many falsely elaborate responses from supposedly intelligent people having their ego hurt but in the end it reminds me why I like rationalists and I don't like most people.

ajkjk 1y ago
Here's a theory of what's happening, both with you here in this comment section and with the rationalists in general. Humans are generally better at perceiving threats than they are at putting those threats into words. When something seems "dangerous" abstractly, they will come up with words for why---but those words don't necessarily reflect the actual threat, because the threat might be hard to describe. Nevertheless the valence of their response reflects their actual emotion on the subject. In this case: the rationalist philosophy basically creeps people out. There is something "insidious" about it. And this is not a delusion on the part of the people judging them: it really does threaten them, and likely for good reason. The explanation is something like "we extrapolate from the way that rationalists think and realize that their philosophy leads to dangerous conclusions." Some of these conclusions have already been made by the rationalists---like valuing people far away abstractly over people next door, by trying to quantify suffering and altruism like a math problem (or to place moral weight on animals over humans, or people in the future over people today). Other conclusions are just implied, waiting to be made later. But the human mind detects them anyway as implications of the way of thinking, and reacts accordingly: thinking like this is dangerous and should be argued against. This extrapolation is hard to put into words, so everyone who tries to express their discomfort misses the target somewhat, and then, if you are the sort of person who only takes things literally, it sounds like they are all just attacking someone out of judgment or bitterness or something instead of for real reasons. But I can't emphasize this enough: their emotions are real, they're just failing to put them into words effectively. It's a skill issue. You will understand what's happening better if you understand that this is what's going on and then try to take their emotions seriously even if they are not communicating them very well. So that's what's going on here. But I think I can also do a decent job of describing the actual problem that people have with the rationalist mindset. It's something like this: Humans have an innate moral intuition that "personal" morality, the kind that takes care of themselves and their family and friends and community, is supposed to be sacrosanct: people are supposed to both practice it and protect the necessity of practicing it. We simply can't trust the world to be a safe place if people don't think of looking out for the people around them as a fundamental moral duty. And once those people are safe, protecting more people, such as a tribe or a nation or all of humanity or all of the planet, becomes permissible. Sometimes people don't or can't practice this protection for various reasons, and that's morally fine, because it's a local problem that can be solved locally. But it's very insidious to turn around and justify not practicing it as a better way to live: "actually it's better not to behave morally; it's better to allocate resources to people far away; it's better to dedicate ourselves to fighting nebulous threats like AI safety or other X-risks instead of our neighbors; or, it's better to protect animals than people, because there are more of them". It's fine to work on important far-away problems once local problems are solved, if that's what you want. But it can't take priority, regardless of how the math works out. To work on global numbers-game problems instead of local problems, and to justify that with arguments, and to try to convince other people to also do that---that's dangerous as hell. It proves too much: it argues that humans at large ought to dismantle their personal moralities in favor of processing the world like a paperclip-maximizing robot. And that is exactly as dangerous as a paperclip-maximizing robot is. Just at a slower timescale. (No surprise that this movement is popular among social outcasts, for whom local morality is going to feel less important, and (I suspect) autistic people, who probably experience less direct moral empathy for the people around them, as well as to the economically-insulated well-to-do tech-nerd types who are less likely to be directly exposed to suffering in their immediate communities.) Ironically paperclip-maximizing-robots are exactly the thing that the rationalists are so worried about. They are a group of people who missed, and then disavowed, and now advocate disavowing, this "personal" morality, and unsurprisingly they view the world in a lens that doesn't include it, which means mostly being worried about problems of the same sort. But it provokes a strong negative reaction from everyone who thinks about the world in terms of that personal duty to safety, because that is the foundation of all morality, and is utterly essential to preserve, because it makes sure that whatever else you are doing doesn't go awry. (edit: let me add that your aversion to the criticisms of rationalists is not unreasonable either. Given that you're parsing the criticisms as unreasonable, which they likely are (because of the skill issue), what you're seeing is a movement with value that seems to be being unfairly attacked. And you're right, the value is actually there! But the ultimate goal here is a synthesis: to get the value of the rationalist movement but to synthesize it with the recognition of the red flags that it sets off. Ignoring either side, the value or the critique, is ultimately counterproductive: the right goal is to synthesize both into a productive middle ground. (This is the arc of philosophy; it's what philosophy is. Not re-reading Plato.) The rationalists are probably morally correct in being motivated to highly-scaling actions e.g. the purview of "Effective Altruism". They are getting attacked for what they're discarding to do that, not for caring about it in the first place.)

not_your_mentat 1y ago

The notion that our moral obligation somehow demands we reduce the suffering of wild animals in an ecosystem, living their lives as they have done since predation evolved and as they will do long after humans have ceased to be, is such a wild misunderstanding of who we are and what we are and what the universe is. I love my Bay Area friends. To quote the great Gwen Stefani, “This sh!t is bananas.”

trod1234 1y ago

Except with Effective Altruism (EA), its not pure logic. Logic requires properties of metaphysical objectivity. If you use the true meaning of words it would be called irrationality, delusion, sophism, or fallacy when such things are claimed true when in fact they are false.

deleted 1y ago

[deleted]

jhbadger 1y ago

There is a great recent book by Adam Becker "More Everything Forever" that deals with the overlapping circles of "effective altruists", "rationalists", and "accelerationists". He's not very sympathetic to the movements as he sees them as mostly rationalizing what their adherents wanted to do anyway -- funding things like rockets and AI over feeding the needy because they see the former as helping more people in the future than dealing with real problems today.

yawnxyz 1y ago

Having just moved to the Bay Area, I've met a few AI "safety researchers" who seems to come from this EA / Rationalist camp, and they all behave more like preachers than thinkers / academics / researchers. I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...

comp_throw7 1y ago
> I don't think any "Rationalists" I ever met would actually consider concepts like scientific method... In that case I don't think you've met any of the people under discussion.
- yawnxyz 1y ago
  maybe, but I Stanford scientists are a different breed of rationalists, and not Rationalists

noosphr 1y ago

Incidentally a good book on logic is the best antidote to that type of thinking. Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly. I also think the ambiguity of meaning in natural language is why statistical llms are so popular with this crowd. You don't need to think about meaning and parsing. Whatever the llm assumes is the meaning is whatever the meaning is.

Viliam1234 1y ago
> Once you learn the difference between a valid and a sound argument and then realize just how ambiguous every English sentence is the idea that just because you have a logical argument you have something useful in everyday life becomes laughable rather quickly. Ironically, that reminds me of "37 Ways That Words Can Be Wrong", written by Eliezer Yudkowsky in 2008...

protocolture 1y ago

>They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic I have read Effective Altruists like that. But I also remember seeing a lot of money donated to a bunch of really decent sounding causes because someone spent 5 minutes asking themselves what they wanted their donation to maximise, decided on "Lives saved" and figured out who is doing the best at that.

sanderjd 1y ago

lol, I love that you identified the exact correct academic term for this phenomenon.

foobar_______ 1y ago

I really appreciate this comment. Now in my 30s, I was not even aware of EA or Rationalist groups in my 20s but your description of a person high on the smell of their own farts and putting logic as king was me. The world is so chaotic that it felt good and right that I could lean on something "true" like logic. It took me to the "correct" answers according to logic but very misguided answers as I see them now. Humans are driven by emotions and the world is probabilistic so relying so heavily on logic just seems so misguided now that I'm older.

Viliam1234 1y ago
By the way, "the world is probabilistic" is one of the key lessons of the rationality community. (Infinite certainty would require infinite amount of evidence, according to the Bayes' Theorem.) When you find someone on internet trying to assign probabilities to their beliefs, chances are you have met a rationalist.

vishnugupta 1y ago

I don’t have anything to add to the discussion except that EA was the very first thing that popped into my mind when I read Rationalists.

mlinhares 1y ago

I'd recommend listening to the behind the bastards episode on some of these folks, it is wild https://podcasts.apple.com/us/podcast/part-one-the-zizians-how-harry-potter-fanfic-inspired/id1373812661?i=1000698710498 https://podcasts.apple.com/us/podcast/part-one-the-zizians-h...

j_timberlake 1y ago

This is a perfect example of a Hacker News hot-take, you complain about their attitude while making zero reference to their actual accomplishments or failures. No one will downvote you for posting something factually incorrect, because you didn't post any facts at all. For anyone else reading, a good example of what EAs do can be seen with the GiveWell charity: https://www.givewell.org/top-charities-fund https://www.givewell.org/top-charities-fund Lots of anti-malaria and vitamin stuff (as a cheap way to save lots of lives). There are also tons of EA animal charities too, such as Humane League: https://thehumaneleague.org/our-impact https://thehumaneleague.org/our-impact

habinero 1y ago
A weird cult doing helpful community service doesn't cancel out the harm of them being a cult. If they want to donate to charity, they can just donate. You don't gotta make a religion out of it.
- j_timberlake 1y ago
  Just like the other guy, you skip the part where you actually name a harm done by them, because then people could easily shoot down your comment. Or name how EAs are a religion, I haven't even seen that troll before.

maybelsyrup 1y ago

> They remind me of the "Effective Altruism" crowd Honestly thought they were the same people

Viliam1234 1y ago
Not all rationalists are EAs, and not all EAs are rationalists, but there are many people who are both. The very idea of using evidence and numbers to choose the optimal charity has a rationalist vibe. But of course, one can donate to an effective charity without being interested in LessWrong-style rationality. https://www.lesswrong.com/ https://www.lesswrong.com/ and https://forum.effectivealtruism.org/ https://forum.effectivealtruism.org/ run on the same software, and some people have an account at both of them, but they are generally separate web communities.

fanwood 1y ago

Absolutely agree. The whole movement is unbearably naive