18 ms·

The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained

The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current people, and that the number of future people vastly outweighs everyone alive today (or who has ever lived), then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.

jdmichal 1y ago
I'm not familiar with any of these communities. Is there also a general bias towards one side between "the most important thing gets the *most* resources" and "the most important thing gets *all* the resources"? Or, in other words, the most important thing is the only important thing? IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)
- nearbuy 1y ago
  It's not the case. Effective altruists give to dozens of different causes, such as malaria prevention, environmentalism, animal welfare, and (perhaps most controversially) extinction risk. It can't tell you which root values to care about. It just asks you to consider whether the charity is impactful. Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values. It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
- JoshuaDavid 1y ago
  The general idea is that on the margin (in the economics sense), more resources should go to the most effective+neglected thing, and.the amount of resources I control is approximately zero in a global sense, so I personally should direct all of my personal giving to the highest impact thing.
  - skinnymuch 1y ago
    And in their logic the highest impact is to donate money, take high paying jobs regardless of morality, and not focusing on any structural or root issues.
    - JoshuaDavid 1y ago
      Yeah, the logic is basically "sure there are lots of structural or root issues, but I'm not confident I can make a substantial positive impact on those with the resources I have whereas I am confident that spending money to prevent people (mostly kids who would otherwise have survived to adulthood) from dying of malaria is a substantial positive impact at ~$5000 / life saved". I find that argument compelling, though I know many don't. Those many are free to focus on structural or root issues, or to try to make the case that addressing those issues is not just good, but better than reducing the impact of malaria.
dasil003 1y ago
Intelligence and rational thought is useful, but like any strategy it has its tradeoffs and limitations. No amount of intelligence can overcome the chaos of long time horizons, especially when we're talking about human civilization. IMHO it's reasonable to pick a long-term problem/risk and focus on solving it. But it's pure hubris to think rationality will give you anything approaching high confidence of what the biggest problems and risks actually are on a 20-50 year time horizon, let alone 200-500 years or longer. The whole reason we even have time to think this way is because we are at the peak of an industrial civilization that has created a level of abundance that allows a lot of people a lot of time to think. But the whole situation that we live in is not stable at all, "progress" could continue, or we could hit a peak and regress. As much as we can see a lot of long-term trajectories (eg. peak oil, global warming), we really have no idea what will be the triggers and inflection points that change the social fabric in ways that are unforeseeable and quickly invalidate whatever prior assumptions all that deep thinking was resting upon. I mean 50 years ago we thought overpopulation was the biggest risk, and that thinking has completely flipped even without a major trajectory change for industrial civilization in that time.
- lisper 1y ago
  I think one can levy a much more specific critique of rationalism: rationalism is in some sense self-defeating. If you are rational you will necessarily conclude that the fundamental dynamic that drives the (interesting parts of) the universe is Darwinian evolution, which is not rational. It blindly selects for reproductive fitness at the expense of all else. If you are a gene, you can probably produce more offspring in an already-industrialized environment by making brains that lean more towards misogyny and sexual promiscuity than gender equality and intellectual achievement. The real conflict here is between Darwinism and enlightenment ideals. But I have yet to see any self-styled Rationalists take this seriously.
  - mettamage 1y ago
    I always liken this to that we’re all asteroids floating in space. There’s no free will and everything is determined. We just see the whole thing unfold from one conscious perspective. Emotionally I don’t subscribe to this view. Rationally I do. My critique for rational people is that they don’t seem to fully take experience into account. It’s assumptions + rationality + experience/data + whatever strong inclinations one has that seems to be the full picture for me.
AnthonyMouse 1y ago
> Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are. Suppose you clock "malicious AI" as a huge risk and then hamper AI, but it turns out the bigger risk is not doing space exploration, which AI would have accelerated, because something catastrophic yet already-inevitable is going to happen to the Earth in a few hundred years and if we're not sustainably multi-planetary by then it's all over. The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.
- TeMPOraL 1y ago
  > Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are. Rationalist community understands that very well. They even know how to put bounds on the unknowns and their own lack of information. > The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one. Right. Good thing they'd agree with you 100% on this.
  - AnthonyMouse 1y ago
    The nature of unknowns is that you don't know them. What's the probability of AI singularity? It has never happened before so you have no priors and any number you assign will be pure speculation.
    - TeMPOraL 1y ago
      Same is true about anything you're trying to forecast, by definition of it being in the future. And yet people have figured out how to make predictions more narrow than shrugging.
      - freejazz 1y ago
im3w1l 1y ago
I think most everyone can agree with this: Being 100% rigorous and rational, reasoning from first principles and completely discarding received wisdom is a great trait in a philosopher but a terrible trait in a policymaker. Because for the former, exploring ideas for the benefit of future generations is more important than whether they ultimately reach the right conclusion or not.
- Johanx64 1y ago
  > Being 100% rigorous and rational, reasoning from first principles It really annoys me when people say that those religious cultists do that. They derive their bullshit from faulty, poorly thought out premises. If you fuck up the very firsts calculations of the algorithm, it doesn't matter how rigorous all the subsequent steps are. There results are going to be all wrong.
- wagwang 1y ago
  The big problem is that human values are similar to neural network weights that can't be clearly defined into true/false axioms like "human life is inherently valuable" and "murder is wrong". The easiest axiom to break is something like "every human life is equally valuable", or even "every human life is born equal".
deleted 1y ago
[deleted]
nradov 1y ago
In what sense are people in those communities "quite smart"? Stupid is as stupid does. There are plenty of people who get good grades and score highly on standardized tests, but are in fact nothing but pontificating blowhards and useless wankers.
- astrange 1y ago
  They're members of a religion which says that if you do math in your head the right way you'll be correct about everything, and so they think they're correct about everything. They also secondarily believe everyone has an IQ which is their DBZ power level; they believe anything they see that has math in it, and IQ is math, so they believe anything they see about IQ. So if you avoid trying to find out your own IQ you can just believe it's really high and then you're good. Unfortunately this lead them to the conclusion that computers have more IQ than them and so would automatically win any intellectual DBZ laser beam fight against them / enslave them / take over the world.
  - jeffhwang 1y ago
    If only I could +1 this more than once! I have learned valuable things occasionally from people in the rationalist community but this overall lack of humility —and strangely blinkered view of humanities and important topics like say history of science relevant to “STEM”—ultimately turned me off to the movement as a whole. And I love science and math! It just shouldn’t belong to people with this (imo) childish model of people, IQ, etc.
  - imtringued 1y ago
    According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. Meanwhile building a single giant super AI is technologically feasible, so they weigh the intelligence of a single person vs all AIs operating as a collective hivemind.
    - Viliam1234 1y ago
      > According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. An actual argument would be that intelligence doesn't work like that. Two people with IQ 100 cooperating together does not produce an IQ 200 solution. There is the "wisdom of crowds". If a random member of a group is more than 50% likely to be correct, the average of the group is more likely to be correct than its members individually. But that has a few assumptions, for example that each member tries to figure out things independently (as opposed to everyone waiting for the highest-status member to express their opinion, and then agreeing with it -- in that case the entire group is only as smart as the highest-status member). But you cannot leverage this to simply invite 1000 random people in your group and ask them to invent a Theory of Everything; because the assumption that each member is more than 50% likely to be correct does not apply in this case. So that is one of the limits of people working together. (And this already conveniently ignores many other problems found in real life, such as conflict of interests, etc.)
bena 1y ago
Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with. I get the feeling these people often want to seem smarter than they are, regardless of how smart they are. And they want to get money to ostensibly "consider these issues", but really they want money for nothing. If they wanted to do right by the future masses, they should be looking to the things that are affecting us right now. But they treat those issues as if they'll work out in the wash.
- aspenmayer 1y ago
  > Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with. The current sums invested and donated in altruist causes are rounding errors themselves compared to GDPs of countries, so the revealed preferences of those investing and donating to altruist causes is to care about the future and the present also. Are you saying that they should give a greater preference to help those who already exist rather than those who may exist in the future? I see a lot of Peter Singer’s ideas in modern “effective” altruism, but I get the sense from your comment that you don’t think that they have good reasons for doing what they do, or that their reason leads them to support well-meaning but ineffective solutions. I am trying to understand your position without misrepresenting your point or goals. Are you naysaying or do you have an alternative? https://en.wikipedia.org/wiki/Peter_Singer https://en.wikipedia.org/wiki/Peter_Singer
  - bena 1y ago
    I think it's essentially a grift. An excuse to do nothing while looking like you care and reaping rewards. If they wanted to help, they should be focused on the now. Global poverty, climate change, despotic world leaders. They should be aligning themselves against such things. But instead what we see is essentially not that. Effective altruism is a lot like the Democratic People's Republic of Korea, a bit of a misnomer.
dmurray 1y ago
The other weird direction it leads is space travel. If you assume we eventually figure out long distance space travel and humanity spreads across the galaxy, there could in the future be quadrillions of people, growing at some kind of exponential rate. So accelerating the space race by even an hour is equivalent to bringing billions of new souls into existence.
- munksbeer 1y ago
  I don't see how bringing new souls (whatever those are) into existence should naturally qualify as a good thing? Perhaps you're arguing as an illustration of the way this group of people think, in which case I understand your point.
  - dmurray 1y ago
    You can make arguments for it starting with "killing billions of people would definitely be bad" or "some fraction of those people will likely share my genes, which I have a biological drive to pass on". It encodes a slight bias towards human existence being a positive thing for us humans, but I don't think it's the shakiest part of that reasoning.
- adastra22 1y ago
  Nitpick: it bottoms out at quadratic growth in the limit, not exponential.
Johanx64 1y ago
>People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense. They don't even do this. If you're reasoning in purely logical and deductive way - it's blatantly obvious that living beings experience way more pain and suffering, than pleasure and joy. If you do the math, humanity getting wiped out in effect is the best thing that could happen. Which is why accelerationism ignoring all the AGI risks is correct strategy presuming the AGI will either wipe us out (good outcome) or provide technologies that improve the human condition and reduce suffering (good outcome). Logical and deductive reasoning based on completely baseless and obviously incorrect premises is flat out idiotic. You can't deprive non-existent people out of anything. And if you do, I hope you're ready for purely logical, deductive follow up - every droplet of sperm is sacred and should be used to impregnate.
georgeecollins 1y ago
For anyone who takes Effective Altruism or Rationalism seriously I strongly recommend reading "You Are Not a Gadget" It was written more than ten years ago and was so prescient about the issues of social media and also contained one of the most devastating problems with EA, the idea of a circle of empathy. You don't have to agree with any of this. I am not defending every idea the author has. But I recommend that book.
- the-mitr 1y ago
  Thanks for the reco.
ineptech 1y ago
What's the seductive-but-wrong part of EA? As far as I can tell, the vast majority of the opposition to it boils down to "Maybe you shouldn't donate money to pet shelters when people are dying of preventable diseases" vs "But donating money to pet shelters feels better!"
- Filligree 1y ago
  Yes, but they’re smug about it, which means they’re wrong. Of course it sounds ridiculous when you spell it out this way.
  - skinnymuch 1y ago
    Or something like focusing on high paying job and donating money does absolutely nothing to solve any root or structural problems which defeats the supposed purpose of caring about future people or being effective altruist in the first place Of course the way your comment is written makes criticism sound silly.
- DowsingSpoon 1y ago
  It might be the parts that lead a person to commit large scale fraud with the idea that the good they can do with the stolen money outweighs all the negatives. Or, at least, that’s the popular idea of what happened to Sam Bankman-Fried. I have no idea what was actually going through that man’s mind. In any case, EA smells strongly of “the ends justify the means” which most popular moral philosophies reject with strong arguments. One which resonates with me is that there are no “ends.” The path itself is the goal.
  - com2kid 1y ago
    > the ends justify the means” which most popular moral philosophies reject with strong arguments. This is a false statement. Our entire modern world is built on the basis of the ends justify the means. Every time money is spent on long term infrastructure vs giving poor kids food right now, every time a war is fought, every time a doctor triages injuries at a disaster.
PaulHoule 1y ago
EA always rubbed me the wrong way. (1) The kind of Gatesian solutions they like to fund like mosquito nets are part of the problem, not part of the solution as I see it. If things are going to get better in Africa, it will be because Africans grow their economy and pay taxes and their governments can provide the services that they want. Expecting NGOs to do everything for them is the same kind of neoliberal thinking that has rotted state capacity in the core and set us up for a political crisis. (2) It is one thing to do something wrong, realize it was a mistake, and then make amends. It's another thing to do plan to do something wrong and to try to offset it somehow. Many of the high paying jobs that EA wants young people to enter are "part of the problem" when it comes to declining stage capacity, legitimation crisis, and not dealing with immediate problems -- like the fact that one of these days there's going to be a heat wave that is a mass causality event. Furthermore (3) Time discounting is a central part of economic planning https://en.wikipedia.org/wiki/Social_discount_rate https://en.wikipedia.org/wiki/Social_discount_rate It is controversial as hell, but one of the many things the Soviet Union got wrong before the 1980s was planning with a discount rate of zero, which led to many economically and ecologically harmful projects. If you seriously think it should be zero you should also be considering whether anybody should work in the finance industry at all or if we should have dropped a hydrogen bomb on Exxon's headquarters yesterday. At some point speculations about the future are just speculation. When it comes to the nuclear waste issue, for instance, I don't think we have any idea what state people are going to be in 20,000 years. They might be really pissed that buried spent nuclear fuel some place they can't get at it. Even the plan to burn plutonium completely in fast breeder reactors has an air of unreality about it, even though it happens on a relatively short 1000 year timescale we can't be sure at all that anyone will be around to finish the job. (4) If you are looking for low-probability events to worry about I think you could find a lot of them. If it was really a movement of free thinkers they'd be concerned about 4,000 horsemen of the apocalypse, not the 4 or so that they are allowed to talk about -- but talk about a bunch of people who'll cancel you if you "think different". Somehow climate change and legitimation crisis just get... ignored. (5) Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion, not least "The Singularity" was talked about by Jesuit Priest Teilhard de Chardin long before sci-fi writer Vernor Vinge used it as the hinge of a mystery novel.
semi-extrinsic 1y ago
> then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. This is the logic of someone who has failed to comprehend the core ideas of Calculus 101. You cannot use intuitive reasoning when it comes to infinite sums of numbers with extremely large uncertainties. All that results is making a fool out of yourself.
- j7ake 1y ago
  This summarizes what irks me about the community. They use technical terms (eg expected value, KL divergence) in their verbal reasoning only to sound rational, but don’t ever mean to use those terms technically.
benchly 1y ago
This tracks based on my limited contact with LessWrong during the whole Roko's Basilisk thing. I quickly lost interest in Roko's Basilisk, but that is what brought me in the door and started me looking around the discussions. At first, it was quite seductive. There was a strange fearlessness there, a willingness to say and admit some things about humanity, our limitations and how we tend to think that other great thinkers maybe danced around in the past. After awhile it became clear that while there were a select few individuals who had found some balance between purely rational thinking and how reality actually works, most of the rest had their heads so far up their asses that they'd fart and call it a cool breeze. Reminded me of my brief obsession with Game Theory and realizing that even it's creators knew it's utility was not quite as advertised to the layman (as in it would not really help you predict or plan for anything at all, just model how decisions might be made).
- benchly 1y ago
  Peardon the "it's" autocorrect, I can't seem to edit on mobile (Harmonic)
  - gsf_emergency_2 1y ago
    Just musing out loud. "Utility" is like, for Physics postgrads: "gauge" Physics undergrads: "wavefunction" Grade schoolers: "temperature" These concepts definitely useful for the hw sets, no understanding needed (or expected!)
gammarator 1y ago
It seems self-evident to me that epistemic humility _requires_ temporal discounting. We should not be confident we can predict the future well enough to integrate utility centuries forward.

jdmichal 1y ago

I'm not familiar with any of these communities. Is there also a general bias towards one side between "the most important thing gets the *most* resources" and "the most important thing gets *all* the resources"? Or, in other words, the most important thing is the only important thing? IMO it's fine to pick a favorite and devote extra resources to it. But that turns less fine when one also starts working to deprive everything else of any oxygen because it's not your favorite. (And I'm aware that this criticism applies to lots of communities.)

nearbuy 1y ago
It's not the case. Effective altruists give to dozens of different causes, such as malaria prevention, environmentalism, animal welfare, and (perhaps most controversially) extinction risk. It can't tell you which root values to care about. It just asks you to consider whether the charity is impactful. Even if an individual person chooses to direct all their donations to a single cause, there's no way to get everyone to donate to a single cause (nor is EA attempting to). Money gets spread around because people have different values. It absolutely does take some money away from other causes, but only in the sense that all charities do: if you give a lot to one charity, you may have less money to give to others.
JoshuaDavid 1y ago
The general idea is that on the margin (in the economics sense), more resources should go to the most effective+neglected thing, and.the amount of resources I control is approximately zero in a global sense, so I personally should direct all of my personal giving to the highest impact thing.
- skinnymuch 1y ago
  And in their logic the highest impact is to donate money, take high paying jobs regardless of morality, and not focusing on any structural or root issues.
  - JoshuaDavid 1y ago
    Yeah, the logic is basically "sure there are lots of structural or root issues, but I'm not confident I can make a substantial positive impact on those with the resources I have whereas I am confident that spending money to prevent people (mostly kids who would otherwise have survived to adulthood) from dying of malaria is a substantial positive impact at ~$5000 / life saved". I find that argument compelling, though I know many don't. Those many are free to focus on structural or root issues, or to try to make the case that addressing those issues is not just good, but better than reducing the impact of malaria.

dasil003 1y ago

Intelligence and rational thought is useful, but like any strategy it has its tradeoffs and limitations. No amount of intelligence can overcome the chaos of long time horizons, especially when we're talking about human civilization. IMHO it's reasonable to pick a long-term problem/risk and focus on solving it. But it's pure hubris to think rationality will give you anything approaching high confidence of what the biggest problems and risks actually are on a 20-50 year time horizon, let alone 200-500 years or longer. The whole reason we even have time to think this way is because we are at the peak of an industrial civilization that has created a level of abundance that allows a lot of people a lot of time to think. But the whole situation that we live in is not stable at all, "progress" could continue, or we could hit a peak and regress. As much as we can see a lot of long-term trajectories (eg. peak oil, global warming), we really have no idea what will be the triggers and inflection points that change the social fabric in ways that are unforeseeable and quickly invalidate whatever prior assumptions all that deep thinking was resting upon. I mean 50 years ago we thought overpopulation was the biggest risk, and that thinking has completely flipped even without a major trajectory change for industrial civilization in that time.

lisper 1y ago
I think one can levy a much more specific critique of rationalism: rationalism is in some sense self-defeating. If you are rational you will necessarily conclude that the fundamental dynamic that drives the (interesting parts of) the universe is Darwinian evolution, which is not rational. It blindly selects for reproductive fitness at the expense of all else. If you are a gene, you can probably produce more offspring in an already-industrialized environment by making brains that lean more towards misogyny and sexual promiscuity than gender equality and intellectual achievement. The real conflict here is between Darwinism and enlightenment ideals. But I have yet to see any self-styled Rationalists take this seriously.
- mettamage 1y ago
  I always liken this to that we’re all asteroids floating in space. There’s no free will and everything is determined. We just see the whole thing unfold from one conscious perspective. Emotionally I don’t subscribe to this view. Rationally I do. My critique for rational people is that they don’t seem to fully take experience into account. It’s assumptions + rationality + experience/data + whatever strong inclinations one has that seems to be the full picture for me.

AnthonyMouse 1y ago

> Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are. Suppose you clock "malicious AI" as a huge risk and then hamper AI, but it turns out the bigger risk is not doing space exploration, which AI would have accelerated, because something catastrophic yet already-inevitable is going to happen to the Earth in a few hundred years and if we're not sustainably multi-planetary by then it's all over. The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one.

TeMPOraL 1y ago
> Incidentally, the flaw in this theory is in thinking you understand what all the existential risks are. Rationalist community understands that very well. They even know how to put bounds on the unknowns and their own lack of information. > The thing evolution teaches us is that diversity is a group survival trait. Anybody insisting "nobody anywhere should do X" is more likely to cause an ELE than prevent one. Right. Good thing they'd agree with you 100% on this.
- AnthonyMouse 1y ago
  The nature of unknowns is that you don't know them. What's the probability of AI singularity? It has never happened before so you have no priors and any number you assign will be pure speculation.
  - TeMPOraL 1y ago
    Same is true about anything you're trying to forecast, by definition of it being in the future. And yet people have figured out how to make predictions more narrow than shrugging.
    - freejazz 1y ago

im3w1l 1y ago

I think most everyone can agree with this: Being 100% rigorous and rational, reasoning from first principles and completely discarding received wisdom is a great trait in a philosopher but a terrible trait in a policymaker. Because for the former, exploring ideas for the benefit of future generations is more important than whether they ultimately reach the right conclusion or not.

Johanx64 1y ago
> Being 100% rigorous and rational, reasoning from first principles It really annoys me when people say that those religious cultists do that. They derive their bullshit from faulty, poorly thought out premises. If you fuck up the very firsts calculations of the algorithm, it doesn't matter how rigorous all the subsequent steps are. There results are going to be all wrong.
wagwang 1y ago
The big problem is that human values are similar to neural network weights that can't be clearly defined into true/false axioms like "human life is inherently valuable" and "murder is wrong". The easiest axiom to break is something like "every human life is equally valuable", or even "every human life is born equal".

deleted 1y ago

[deleted]

nradov 1y ago

In what sense are people in those communities "quite smart"? Stupid is as stupid does. There are plenty of people who get good grades and score highly on standardized tests, but are in fact nothing but pontificating blowhards and useless wankers.

astrange 1y ago
They're members of a religion which says that if you do math in your head the right way you'll be correct about everything, and so they think they're correct about everything. They also secondarily believe everyone has an IQ which is their DBZ power level; they believe anything they see that has math in it, and IQ is math, so they believe anything they see about IQ. So if you avoid trying to find out your own IQ you can just believe it's really high and then you're good. Unfortunately this lead them to the conclusion that computers have more IQ than them and so would automatically win any intellectual DBZ laser beam fight against them / enslave them / take over the world.
- jeffhwang 1y ago
  If only I could +1 this more than once! I have learned valuable things occasionally from people in the rationalist community but this overall lack of humility —and strangely blinkered view of humanities and important topics like say history of science relevant to “STEM”—ultimately turned me off to the movement as a whole. And I love science and math! It just shouldn’t belong to people with this (imo) childish model of people, IQ, etc.
- imtringued 1y ago
  According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. Meanwhile building a single giant super AI is technologically feasible, so they weigh the intelligence of a single person vs all AIs operating as a collective hivemind.
  - Viliam1234 1y ago
    > According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. An actual argument would be that intelligence doesn't work like that. Two people with IQ 100 cooperating together does not produce an IQ 200 solution. There is the "wisdom of crowds". If a random member of a group is more than 50% likely to be correct, the average of the group is more likely to be correct than its members individually. But that has a few assumptions, for example that each member tries to figure out things independently (as opposed to everyone waiting for the highest-status member to express their opinion, and then agreeing with it -- in that case the entire group is only as smart as the highest-status member). But you cannot leverage this to simply invite 1000 random people in your group and ask them to invent a Theory of Everything; because the assumption that each member is more than 50% likely to be correct does not apply in this case. So that is one of the limits of people working together. (And this already conveniently ignores many other problems found in real life, such as conflict of interests, etc.)

bena 1y ago

Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with. I get the feeling these people often want to seem smarter than they are, regardless of how smart they are. And they want to get money to ostensibly "consider these issues", but really they want money for nothing. If they wanted to do right by the future masses, they should be looking to the things that are affecting us right now. But they treat those issues as if they'll work out in the wash.

aspenmayer 1y ago
> Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with. The current sums invested and donated in altruist causes are rounding errors themselves compared to GDPs of countries, so the revealed preferences of those investing and donating to altruist causes is to care about the future and the present also. Are you saying that they should give a greater preference to help those who already exist rather than those who may exist in the future? I see a lot of Peter Singer’s ideas in modern “effective” altruism, but I get the sense from your comment that you don’t think that they have good reasons for doing what they do, or that their reason leads them to support well-meaning but ineffective solutions. I am trying to understand your position without misrepresenting your point or goals. Are you naysaying or do you have an alternative? https://en.wikipedia.org/wiki/Peter_Singer https://en.wikipedia.org/wiki/Peter_Singer
- bena 1y ago
  I think it's essentially a grift. An excuse to do nothing while looking like you care and reaping rewards. If they wanted to help, they should be focused on the now. Global poverty, climate change, despotic world leaders. They should be aligning themselves against such things. But instead what we see is essentially not that. Effective altruism is a lot like the Democratic People's Republic of Korea, a bit of a misnomer.

dmurray 1y ago

The other weird direction it leads is space travel. If you assume we eventually figure out long distance space travel and humanity spreads across the galaxy, there could in the future be quadrillions of people, growing at some kind of exponential rate. So accelerating the space race by even an hour is equivalent to bringing billions of new souls into existence.

munksbeer 1y ago
I don't see how bringing new souls (whatever those are) into existence should naturally qualify as a good thing? Perhaps you're arguing as an illustration of the way this group of people think, in which case I understand your point.
- dmurray 1y ago
  You can make arguments for it starting with "killing billions of people would definitely be bad" or "some fraction of those people will likely share my genes, which I have a biological drive to pass on". It encodes a slight bias towards human existence being a positive thing for us humans, but I don't think it's the shakiest part of that reasoning.
adastra22 1y ago
Nitpick: it bottoms out at quadratic growth in the limit, not exponential.

Johanx64 1y ago

>People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense. They don't even do this. If you're reasoning in purely logical and deductive way - it's blatantly obvious that living beings experience way more pain and suffering, than pleasure and joy. If you do the math, humanity getting wiped out in effect is the best thing that could happen. Which is why accelerationism ignoring all the AGI risks is correct strategy presuming the AGI will either wipe us out (good outcome) or provide technologies that improve the human condition and reduce suffering (good outcome). Logical and deductive reasoning based on completely baseless and obviously incorrect premises is flat out idiotic. You can't deprive non-existent people out of anything. And if you do, I hope you're ready for purely logical, deductive follow up - every droplet of sperm is sacred and should be used to impregnate.

georgeecollins 1y ago

For anyone who takes Effective Altruism or Rationalism seriously I strongly recommend reading "You Are Not a Gadget" It was written more than ten years ago and was so prescient about the issues of social media and also contained one of the most devastating problems with EA, the idea of a circle of empathy. You don't have to agree with any of this. I am not defending every idea the author has. But I recommend that book.

the-mitr 1y ago
Thanks for the reco.

ineptech 1y ago

What's the seductive-but-wrong part of EA? As far as I can tell, the vast majority of the opposition to it boils down to "Maybe you shouldn't donate money to pet shelters when people are dying of preventable diseases" vs "But donating money to pet shelters feels better!"

Filligree 1y ago
Yes, but they’re smug about it, which means they’re wrong. Of course it sounds ridiculous when you spell it out this way.
- skinnymuch 1y ago
  Or something like focusing on high paying job and donating money does absolutely nothing to solve any root or structural problems which defeats the supposed purpose of caring about future people or being effective altruist in the first place Of course the way your comment is written makes criticism sound silly.
DowsingSpoon 1y ago
It might be the parts that lead a person to commit large scale fraud with the idea that the good they can do with the stolen money outweighs all the negatives. Or, at least, that’s the popular idea of what happened to Sam Bankman-Fried. I have no idea what was actually going through that man’s mind. In any case, EA smells strongly of “the ends justify the means” which most popular moral philosophies reject with strong arguments. One which resonates with me is that there are no “ends.” The path itself is the goal.
- com2kid 1y ago
  > the ends justify the means” which most popular moral philosophies reject with strong arguments. This is a false statement. Our entire modern world is built on the basis of the ends justify the means. Every time money is spent on long term infrastructure vs giving poor kids food right now, every time a war is fought, every time a doctor triages injuries at a disaster.

PaulHoule 1y ago

EA always rubbed me the wrong way. (1) The kind of Gatesian solutions they like to fund like mosquito nets are part of the problem, not part of the solution as I see it. If things are going to get better in Africa, it will be because Africans grow their economy and pay taxes and their governments can provide the services that they want. Expecting NGOs to do everything for them is the same kind of neoliberal thinking that has rotted state capacity in the core and set us up for a political crisis. (2) It is one thing to do something wrong, realize it was a mistake, and then make amends. It's another thing to do plan to do something wrong and to try to offset it somehow. Many of the high paying jobs that EA wants young people to enter are "part of the problem" when it comes to declining stage capacity, legitimation crisis, and not dealing with immediate problems -- like the fact that one of these days there's going to be a heat wave that is a mass causality event. Furthermore (3) Time discounting is a central part of economic planning https://en.wikipedia.org/wiki/Social_discount_rate https://en.wikipedia.org/wiki/Social_discount_rate It is controversial as hell, but one of the many things the Soviet Union got wrong before the 1980s was planning with a discount rate of zero, which led to many economically and ecologically harmful projects. If you seriously think it should be zero you should also be considering whether anybody should work in the finance industry at all or if we should have dropped a hydrogen bomb on Exxon's headquarters yesterday. At some point speculations about the future are just speculation. When it comes to the nuclear waste issue, for instance, I don't think we have any idea what state people are going to be in 20,000 years. They might be really pissed that buried spent nuclear fuel some place they can't get at it. Even the plan to burn plutonium completely in fast breeder reactors has an air of unreality about it, even though it happens on a relatively short 1000 year timescale we can't be sure at all that anyone will be around to finish the job. (4) If you are looking for low-probability events to worry about I think you could find a lot of them. If it was really a movement of free thinkers they'd be concerned about 4,000 horsemen of the apocalypse, not the 4 or so that they are allowed to talk about -- but talk about a bunch of people who'll cancel you if you "think different". Somehow climate change and legitimation crisis just get... ignored. (5) Although it is run by people who say they are militant atheists, the movement has all the trappings of a religion, not least "The Singularity" was talked about by Jesuit Priest Teilhard de Chardin long before sci-fi writer Vernor Vinge used it as the hinge of a mystery novel.

semi-extrinsic 1y ago

> then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. This is the logic of someone who has failed to comprehend the core ideas of Calculus 101. You cannot use intuitive reasoning when it comes to infinite sums of numbers with extremely large uncertainties. All that results is making a fool out of yourself.

j7ake 1y ago
This summarizes what irks me about the community. They use technical terms (eg expected value, KL divergence) in their verbal reasoning only to sound rational, but don’t ever mean to use those terms technically.

benchly 1y ago

This tracks based on my limited contact with LessWrong during the whole Roko's Basilisk thing. I quickly lost interest in Roko's Basilisk, but that is what brought me in the door and started me looking around the discussions. At first, it was quite seductive. There was a strange fearlessness there, a willingness to say and admit some things about humanity, our limitations and how we tend to think that other great thinkers maybe danced around in the past. After awhile it became clear that while there were a select few individuals who had found some balance between purely rational thinking and how reality actually works, most of the rest had their heads so far up their asses that they'd fart and call it a cool breeze. Reminded me of my brief obsession with Game Theory and realizing that even it's creators knew it's utility was not quite as advertised to the layman (as in it would not really help you predict or plan for anything at all, just model how decisions might be made).

benchly 1y ago
Peardon the "it's" autocorrect, I can't seem to edit on mobile (Harmonic)
- gsf_emergency_2 1y ago
  Just musing out loud. "Utility" is like, for Physics postgrads: "gauge" Physics undergrads: "wavefunction" Grade schoolers: "temperature" These concepts definitely useful for the hw sets, no understanding needed (or expected!)

gammarator 1y ago

It seems self-evident to me that epistemic humility _requires_ temporal discounting. We should not be confident we can predict the future well enough to integrate utility centuries forward.