33 ms·

The Black Hole Objection to Longtermism and Consequentialism

11 points by headalgorithm 3y ago

PaulHoule 3y ago
Goes to show how thinking too hard about trolley problems destroys your capacity for moral reasoning.
- vlovich123 3y ago
  And also destroys your ability to pose realistic hypotheticals. Just like mathematics solved Zeno’s paradox, CS has solved the trolley problem. But it makes you seem smart to continue to pose unrealistic hypotheticals because you can avoid listening to people challenging your premise. Now people will say that the answer to the trolley problem - AI avoids the trolley problem in the first place + you can minimize the total kinetic energy in the system when you simulate forcing into it - is a cheat because humans can’t do it and it doesn’t address the underlying moral question. Who cares? We know that the trolley problem doesn’t actually appear in reality in the clean way it’s presented anyway. So if the philosopher is cheating with an unrealistic scenario that would never happen, it’s fair for the student to find a solution by cheating in response (ie Kobayashi Maru).
  - PaulHoule 3y ago
    There are those cases also where you want to let the trolley go one way then push it back the top of the hill, throw the switch, and let it get the other way.
mehlmao 3y ago
the wise man bowed his head solemnly and spoke: "theres actually zero difference between good & bad things. you imbecile. you fucking moron"
smitty1e 3y ago
> If you flip the switch, you kill yourself and everyone you know. If I'm an atheist and materialist in my views, then I am not grasped by any larger context in which to evaluate this decision. That is: why not a nihilistic flip of the switch?
- mensetmanusman 3y ago
  There is no foundational reason for any decision as a nihilist, so no reason to ask the question.
- DemocracyFTW2 3y ago
  If you're a nihilist it is immaterial to me what else you say. Because you're saying that not only am I irrelevant, everyone and everything that exists and has existed is so, too. Gives me every right in the world to not listen to you.
  - smitty1e 3y ago
    Agreed, but not seeing the difference between nihilism and atheism. Restated: whatever meaning we do find in life seem theistic to me.
kstenerud 3y ago
And this is why it's a fool's errand to try to apply simplistic rules to a complex universe. Your model will never be complete enough to make a 100% accurate decision until the model is the size of the universe. And thus your stubbornness leads to this kind of nonsense.
- rhn_mk1 3y ago
  And yet, people keep trying. Consequentialism, deonthological ethics, ten commandments, civil laws. Your rules will never be complete enough to make 100% accurate decisions. When will we learn to avoid this kind of nonsense?
  - kstenerud 3y ago
    Civil and criminal laws don't try to be universally complete and accurate. Their fallibility and temporal deprecation is understood and accounted for (unlike the ten commandments). Consequentialism leads to splitting hairs as you reach the fuzzy edges (and more often long before that), giving us self-serving philosophies like effective altruism. We live in a messy world. Any rules we live by need to know and accept their limits.
    - rhn_mk1 3y ago
      You strive for simplicity, but you just threw away any attempt at ethics together with it. Ethics, as in, a set of rules to separate the right from the wrong. I'm not aware of any ethical system which does not try to be universal and accurate. And if the world is so complex that you can't describe it in rules, then ethics is impossible. Needless to say, I disagree with that notion. Not on the grounds that it's an absurd conclusion, but because an ethical framework doesn't need to be complex in principle, and because simple rules can be accurate.
      - kstenerud 3y ago
        That is where we disagree. I don't see any morality or ethical framework (existent or as-yet-invented) as capable of capturing reality. They can certainly imitate it to varying degrees (even to the level of usefulness in many cases). But they can never capture reality, and I would consider anyone who rigidly adheres to one as a dangerous fool.
yosito 3y ago
> According to consequentialism, we should act to maximize good consequences. According to longtermism, we should act to benefit the long-term future. If either is correct, then it would be morally good to destroy Earth to seed a new, life-supporting universe. After such an illogical, non sequitur intro, I'm surprised that this is being upvoted.
- DemocracyFTW2 3y ago
  But this is not a bad rendering of what longtermism suggests we should think like. The author only reports it.
joe_the_user 3y ago
A lot of people's reaction to longtermism is hostility and derision and that's often my reaction as well. However, I want to give a coherent explanation for my reaction, since this stuff appears often. Human beings are known for values that are risk-averse and loss-averse relative to the supposed "rational" positions of optimizing an abstract utility function. If someone has, say, a plan to murder X many people to produce Y many future, better people, I have no objection to social sanctions such as prison for them. I'm loss averse, I'm not promiscuously maximizing people-goodness and this is a very natural, human reaction. It's completely reasonable that we will spend far more resources defending the things, the people, the lives and even the pets we have, compared to the resources spent getting hypothetically different versions of these things. Part of it is one never knows if the hypothetical things will be as good and part of it is one never knows if the calculations we make actually will pan out. And the "longtermists" often multiply together numbers together they have no basis for knowing in the first place (people in black wholes, people in galactic civilizations, etc) - they fail to conceive that some things more unknowable than probable or improbable and that in such situation loss aversion is especially reasonable - despite going against their
vlovich123 3y ago
The arguments are extremely flawed on several fronts. 1. We don’t know how to do good risk modeling with huge unknowns. How do you know the universe you create will meet the criteria that would put into the benefit part of the cost/benefit curve? So we don’t know X. We don’t know what the magnitude of the benefit is. Is it really 1e9? And we don’t know the probability of failure. Is it really 1e-6? So sure, for some random set of hypothetical numbers, the cost benefit trade off is there. But we don’t know what the real numbers are and we don’t know what the uncertainty space is. 2. This argument implicitly puts cost/benefit into a single linear variable. But we know that’s not the case. What if the harm is an exponential and the probability of it / magnitude has different scaling factors than your benefit function? 3. If all you need is a planet and you have this tech, what is the likelihood that you can’t use a planet other than Earth or you can’t migrate humans away first? This is like those mathematical logic puzzles that get you to conclude that 1=2 because an interim step was a division by 0.
Semiapies 3y ago
How do things make it to the front page with just eight votes?

11 points by headalgorithm 3y ago

PaulHoule 3y ago

Goes to show how thinking too hard about trolley problems destroys your capacity for moral reasoning.

vlovich123 3y ago
And also destroys your ability to pose realistic hypotheticals. Just like mathematics solved Zeno’s paradox, CS has solved the trolley problem. But it makes you seem smart to continue to pose unrealistic hypotheticals because you can avoid listening to people challenging your premise. Now people will say that the answer to the trolley problem - AI avoids the trolley problem in the first place + you can minimize the total kinetic energy in the system when you simulate forcing into it - is a cheat because humans can’t do it and it doesn’t address the underlying moral question. Who cares? We know that the trolley problem doesn’t actually appear in reality in the clean way it’s presented anyway. So if the philosopher is cheating with an unrealistic scenario that would never happen, it’s fair for the student to find a solution by cheating in response (ie Kobayashi Maru).
- PaulHoule 3y ago
  There are those cases also where you want to let the trolley go one way then push it back the top of the hill, throw the switch, and let it get the other way.

mehlmao 3y ago

the wise man bowed his head solemnly and spoke: "theres actually zero difference between good & bad things. you imbecile. you fucking moron"

smitty1e 3y ago

> If you flip the switch, you kill yourself and everyone you know. If I'm an atheist and materialist in my views, then I am not grasped by any larger context in which to evaluate this decision. That is: why not a nihilistic flip of the switch?

mensetmanusman 3y ago
There is no foundational reason for any decision as a nihilist, so no reason to ask the question.
DemocracyFTW2 3y ago
If you're a nihilist it is immaterial to me what else you say. Because you're saying that not only am I irrelevant, everyone and everything that exists and has existed is so, too. Gives me every right in the world to not listen to you.
- smitty1e 3y ago
  Agreed, but not seeing the difference between nihilism and atheism. Restated: whatever meaning we do find in life seem theistic to me.

kstenerud 3y ago

And this is why it's a fool's errand to try to apply simplistic rules to a complex universe. Your model will never be complete enough to make a 100% accurate decision until the model is the size of the universe. And thus your stubbornness leads to this kind of nonsense.

rhn_mk1 3y ago
And yet, people keep trying. Consequentialism, deonthological ethics, ten commandments, civil laws. Your rules will never be complete enough to make 100% accurate decisions. When will we learn to avoid this kind of nonsense?
- kstenerud 3y ago
  Civil and criminal laws don't try to be universally complete and accurate. Their fallibility and temporal deprecation is understood and accounted for (unlike the ten commandments). Consequentialism leads to splitting hairs as you reach the fuzzy edges (and more often long before that), giving us self-serving philosophies like effective altruism. We live in a messy world. Any rules we live by need to know and accept their limits.
  - rhn_mk1 3y ago
    You strive for simplicity, but you just threw away any attempt at ethics together with it. Ethics, as in, a set of rules to separate the right from the wrong. I'm not aware of any ethical system which does not try to be universal and accurate. And if the world is so complex that you can't describe it in rules, then ethics is impossible. Needless to say, I disagree with that notion. Not on the grounds that it's an absurd conclusion, but because an ethical framework doesn't need to be complex in principle, and because simple rules can be accurate.
    - kstenerud 3y ago
      That is where we disagree. I don't see any morality or ethical framework (existent or as-yet-invented) as capable of capturing reality. They can certainly imitate it to varying degrees (even to the level of usefulness in many cases). But they can never capture reality, and I would consider anyone who rigidly adheres to one as a dangerous fool.

yosito 3y ago

> According to consequentialism, we should act to maximize good consequences. According to longtermism, we should act to benefit the long-term future. If either is correct, then it would be morally good to destroy Earth to seed a new, life-supporting universe. After such an illogical, non sequitur intro, I'm surprised that this is being upvoted.

DemocracyFTW2 3y ago
But this is not a bad rendering of what longtermism suggests we should think like. The author only reports it.

joe_the_user 3y ago

A lot of people's reaction to longtermism is hostility and derision and that's often my reaction as well. However, I want to give a coherent explanation for my reaction, since this stuff appears often. Human beings are known for values that are risk-averse and loss-averse relative to the supposed "rational" positions of optimizing an abstract utility function. If someone has, say, a plan to murder X many people to produce Y many future, better people, I have no objection to social sanctions such as prison for them. I'm loss averse, I'm not promiscuously maximizing people-goodness and this is a very natural, human reaction. It's completely reasonable that we will spend far more resources defending the things, the people, the lives and even the pets we have, compared to the resources spent getting hypothetically different versions of these things. Part of it is one never knows if the hypothetical things will be as good and part of it is one never knows if the calculations we make actually will pan out. And the "longtermists" often multiply together numbers together they have no basis for knowing in the first place (people in black wholes, people in galactic civilizations, etc) - they fail to conceive that some things more unknowable than probable or improbable and that in such situation loss aversion is especially reasonable - despite going against their

vlovich123 3y ago

The arguments are extremely flawed on several fronts. 1. We don’t know how to do good risk modeling with huge unknowns. How do you know the universe you create will meet the criteria that would put into the benefit part of the cost/benefit curve? So we don’t know X. We don’t know what the magnitude of the benefit is. Is it really 1e9? And we don’t know the probability of failure. Is it really 1e-6? So sure, for some random set of hypothetical numbers, the cost benefit trade off is there. But we don’t know what the real numbers are and we don’t know what the uncertainty space is. 2. This argument implicitly puts cost/benefit into a single linear variable. But we know that’s not the case. What if the harm is an exponential and the probability of it / magnitude has different scaling factors than your benefit function? 3. If all you need is a planet and you have this tech, what is the likelihood that you can’t use a planet other than Earth or you can’t migrate humans away first? This is like those mathematical logic puzzles that get you to conclude that 1=2 because an interim step was a division by 0.

Semiapies 3y ago

How do things make it to the front page with just eight votes?