3 ms·

>.g. "it's better for one person to be tortured for fifty years than for an extremely large number of people to each get a speck of dust in their eye"), The nu

>.g. "it's better for one person to be tortured for fifty years than for an extremely large number of people to each get a speck of dust in their eye"), The number in question was 3^^^3 using Knuth's Up Arrow Notation. As he explains, that's a lot of people: (3^(3^(3^(... 7625597484987 times ...)))). If we decide that one person being tortured for 50 years is worth a quick blink by all those other people, then a still tiny amount of people (say, 7625597484987) being tortured would be worth everyone else blinking furiously for their entire lives. So it's not about deciding one person's fate. It's about being consistent, because if you make an exception for one person, that turns into several hundred billion, then that small sacrifice everyone else was to carry now has destroyed everyone else, too. (I think the issue is that people tend to round down "speck of dust" to 0, then multiply, then find of course any amount of torture isn't worth 0.) Personally, I take a more negative view and see the immense possibility for suffering as a reason that we should seek to destroy the entire multiverse. It's just not very nice otherwise. http://lesswrong.com/lw/kn/torture_vs_dust_specks/ http://lesswrong.com/lw/kn/torture_vs_dust_specks/

zyxley 12y ago
> I think the issue is that people tend to round down "speck of dust" to 0, then multiply, then find of course any amount of torture isn't worth 0. Why not round it down to 0? There's some level of discomfort in everyday life to start with that's effectively negligible for any normal person, simply because it's a prerequisite for interacting with the world (getting a raindrop in your eye, dealing with a minor wedgie, having an itchy nose, whatever).
- MichaelGG 12y ago
  If you round down before multiplying, then you get an invalid answer. First multiply it out. So in my example, torturing 7,625,597,484,987 people for 50 years, versus that many specks of dust for everyone else. (7 trillion is essentially 1, compared to 3^^^3, right?) 7 trillion specks of dust is enough to turn all those other lives into torture, as 7 trillion specks of dust is a few thousand a second, for a very long life. That's why you can't round down first. Eliezer's entire intent there is to illustrate that by default, we suck at scale, we're scope insensitive. Another way: If everyone in the US gave just a penny directly to a poor person, once in a lifetime, that person would be wealthy for life and be OK, right? A penny rounds down to zero, thus we can determine that the right action is to always give money directly to poor individuals, as this will cure poverty at a cost of nothing. You're right that if the only choice, ever, in the entire multiverse was this one 50 years or 3^^^3 specks of dust, then sure, you don't need to follow the rules/math. But we're never going to be faced with just a single decision. Instead, we should be consistent and follow the numbers to determine utility.
  - zyxley 12y ago
    > If you round down before multiplying, then you get an invalid answer. Your argument here is based on the premise that the subjectively-inflicted-pain of "a whole lot of specks of dust in a person's eye" can be treated as a multiple of "one speck of dust in a person's eye", which to me is the point of absurdity that makes the whole thought exercise fall apart.
    - Houshalter 12y ago
      That's just basic utilitarianism. What exactly are you objecting to? That 2 dust specks in 2 people's eyes isn't exactly twice as bad as 1 dust speck?
      - zyxley 12y ago
        > What exactly are you objecting to? That some arbitrarily minimal amount of subjective pain can be presumed to be actually distinguishable from the general background unpleasantness of being human in the first place. Edit: Rephrased to remove gibberish resulting from temporary brain/keyboard disconnect.
        Houshalter 12y ago
        It doesn't matter if it's "distinguishable". It's objectively worse. A pin prick might not be noticeable but a million is the worst torture imaginable.
        davidgerard 12y ago
        You're assuming that dust specks across different people can be added, and then asserting that as evidence that dust specks across different people can be added. The addition is ridiculous. Also, minmaxing is a thing.
        zyxley 12y ago
        For some reason HN won't let me reply to Houshalter's post, but... > objectively How can pain be objective? The universe doesn't give a shit that some lumpy sacks of meat dislike certain experiences, and exactly the same action can be unpleasant for some people but pleasant for others. (BDSM enthusiasts, for example, often actively seek out experiences that other people would consider painful.)
davidgerard 12y ago
In real-world AI, they tend to go for minmaxing, which avoids such absurdities. Minmaxing says "don't torture people, your dust speck has literally been defined to be insignificant."
- MichaelGG 12y ago
  Ok so now instead of one person being tortured for 50 years vs 3^^^3 people with dust specks, let's pose a new question: 7 trillion people tortured for 50 years vs 3^^^3 people with 7 trillion dust specks. Does that make the flaw more obvious? You can't round before multiplying. What am I misunderstanding?
  - davidgerard 12y ago
    That in real-life AI, they don't use simplistic utilitarianism. You're dodging the core of the problem, as stated above: "However, when pushed too far, those tools tend to break down—but the rationalist answer to that breakdown is all too often to embrace the model and discount the reality." https://news.ycombinator.com/item?id=9204442 https://news.ycombinator.com/item?id=9204442 Just because you love your model doesn't make its conclusions at extremes true.
rthomas6 12y ago
I've thought about this and concluded that this thought experiment ignores the subjective way people experience reality. If a dust speck fell in your own eye, you would round that experience down to zero. You wouldn't even remember the event an hour later. I think this subjective experience of reality should be taken into account. As for the part of the argument that dust specks in so many eyes would cause the death of a small fraction of the people, that's far from a given but for the sake of "steelmanning" let's assume that's true. At some point my own ethics place death as a higher utility outcome than prolonged suffering. How many deaths vs how much suffering? That's hard to quantify.

zyxley 12y ago

> I think the issue is that people tend to round down "speck of dust" to 0, then multiply, then find of course any amount of torture isn't worth 0. Why not round it down to 0? There's some level of discomfort in everyday life to start with that's effectively negligible for any normal person, simply because it's a prerequisite for interacting with the world (getting a raindrop in your eye, dealing with a minor wedgie, having an itchy nose, whatever).

MichaelGG 12y ago
If you round down before multiplying, then you get an invalid answer. First multiply it out. So in my example, torturing 7,625,597,484,987 people for 50 years, versus that many specks of dust for everyone else. (7 trillion is essentially 1, compared to 3^^^3, right?) 7 trillion specks of dust is enough to turn all those other lives into torture, as 7 trillion specks of dust is a few thousand a second, for a very long life. That's why you can't round down first. Eliezer's entire intent there is to illustrate that by default, we suck at scale, we're scope insensitive. Another way: If everyone in the US gave just a penny directly to a poor person, once in a lifetime, that person would be wealthy for life and be OK, right? A penny rounds down to zero, thus we can determine that the right action is to always give money directly to poor individuals, as this will cure poverty at a cost of nothing. You're right that if the only choice, ever, in the entire multiverse was this one 50 years or 3^^^3 specks of dust, then sure, you don't need to follow the rules/math. But we're never going to be faced with just a single decision. Instead, we should be consistent and follow the numbers to determine utility.
- zyxley 12y ago
  > If you round down before multiplying, then you get an invalid answer. Your argument here is based on the premise that the subjectively-inflicted-pain of "a whole lot of specks of dust in a person's eye" can be treated as a multiple of "one speck of dust in a person's eye", which to me is the point of absurdity that makes the whole thought exercise fall apart.
  - Houshalter 12y ago
    That's just basic utilitarianism. What exactly are you objecting to? That 2 dust specks in 2 people's eyes isn't exactly twice as bad as 1 dust speck?
    - zyxley 12y ago
      > What exactly are you objecting to? That some arbitrarily minimal amount of subjective pain can be presumed to be actually distinguishable from the general background unpleasantness of being human in the first place. Edit: Rephrased to remove gibberish resulting from temporary brain/keyboard disconnect.
      - Houshalter 12y ago
        It doesn't matter if it's "distinguishable". It's objectively worse. A pin prick might not be noticeable but a million is the worst torture imaginable.
        davidgerard 12y ago
        You're assuming that dust specks across different people can be added, and then asserting that as evidence that dust specks across different people can be added. The addition is ridiculous. Also, minmaxing is a thing.
      - zyxley 12y ago
        For some reason HN won't let me reply to Houshalter's post, but... > objectively How can pain be objective? The universe doesn't give a shit that some lumpy sacks of meat dislike certain experiences, and exactly the same action can be unpleasant for some people but pleasant for others. (BDSM enthusiasts, for example, often actively seek out experiences that other people would consider painful.)

davidgerard 12y ago

In real-world AI, they tend to go for minmaxing, which avoids such absurdities. Minmaxing says "don't torture people, your dust speck has literally been defined to be insignificant."

MichaelGG 12y ago
Ok so now instead of one person being tortured for 50 years vs 3^^^3 people with dust specks, let's pose a new question: 7 trillion people tortured for 50 years vs 3^^^3 people with 7 trillion dust specks. Does that make the flaw more obvious? You can't round before multiplying. What am I misunderstanding?
- davidgerard 12y ago
  That in real-life AI, they don't use simplistic utilitarianism. You're dodging the core of the problem, as stated above: "However, when pushed too far, those tools tend to break down—but the rationalist answer to that breakdown is all too often to embrace the model and discount the reality." https://news.ycombinator.com/item?id=9204442 https://news.ycombinator.com/item?id=9204442 Just because you love your model doesn't make its conclusions at extremes true.

rthomas6 12y ago

I've thought about this and concluded that this thought experiment ignores the subjective way people experience reality. If a dust speck fell in your own eye, you would round that experience down to zero. You wouldn't even remember the event an hour later. I think this subjective experience of reality should be taken into account. As for the part of the argument that dust specks in so many eyes would cause the death of a small fraction of the people, that's far from a given but for the sake of "steelmanning" let's assume that's true. At some point my own ethics place death as a higher utility outcome than prolonged suffering. How many deaths vs how much suffering? That's hard to quantify.