3 ms·

> precommit to not cooperating with acausal blackmail He knows that can't possibly work, right? Implicitly it assumes perfect invulnerability to any method of

> precommit to not cooperating with acausal blackmail He knows that can't possibly work, right? Implicitly it assumes perfect invulnerability to any method of coercion, exploitation, subversion, or suffering that can be invented by an intelligence sufficiently superhuman to have escaped its natal light cone. There may exist forms of life in this universe for which such an assumption is safe. Humanity circa 2024 seems most unlikely to be among them.

endtime 2y ago
Eliezer once told me that he thinks people aren't vegetarian because they don't think animals are sapient. And I tried to explain to him that actually most people aren't vegetarian because they don't think about it very much, and don't try to be rigorously ethical in any case, and that by far the most common response to ethical arguments is not "cows aren't sapient" but "you might be right but meat is delicious so I am going to keep eating it". I think EY is so surrounded by bright nerds that he has a hard time modeling average people. Though in this case, in his defense, average people will never hear about Roko's Basilisk.
- defrost 2y ago
  Despite, perhaps, all your experience to the contrary it's only a relatively recent change to a situation where "most people" have no association with the animals they eat for meat and thus can find themselves "not thinking about it very much". It's only within the past decade or so that the bulk of human population lives in an urban setting. Until that point most people did not and most people gone fishing, seen a carcass hanging in a butcher's shop, killed for food at least once, had a holiday on a farm if not worked on one or grown up farm adjacent. By most people, of course, I refer to globally. Throughout history vegetarianism was relatively rare save in vegatarian cultures (Hindi, et al) and in those cultures where it was rare people were all too aware of the animals they killed to eat. Many knew that pigs were smart and that dogs and cats interact with humans, etc. Eliezer was correct to think that people who killed to eat thought about their food animals differently but I suspect it had less to do with sapience and more to do with thinking animals to be of a lesser order, or there to be eaten and to be nutured so there would be more for the years to come. This is most evident in, sat, hunter societies, aboriginals and bushmen, who have extensive stories about animals, how they think, how they move and react, when they breed, how many can be taken, etc. They absolutely attribute a differing kind of thought, and they hunt them and try not to over tax the populations.
  - endtime 2y ago
    That's all fair, but the context of the conversation was the present day, not the aggregate of all human history.
    - defrost 2y ago
      People are or are not vegetarian mostly because of their parents and the culture in which they were raised. People who are not vegetarian but have never cared for or killed a farm animal were very likely (in most parts of the world) raised by people that have. Even in the USofA much of the present generations are not far removed from grandparents who owned farms | worked farms | hunted. The present day is a continuum from yesterday. Change can happen, but the current conditions are shaped by the prior conditions.
- tbrownaw 2y ago
  There's a standard response to a particular PETA campaign: "Meat is murder. Delicious, delicious murder.". It's a bit odd that someone would like to argue on the topic, but also either not have heard that or not recognize the ha-ha-only-serious nature of it.
  - digging 2y ago
    I believe most people would be fine with eating the meat of murdered humans, too, if it was sold on grocery store shelves for a few years. The power of normalization is immense. It sounds like Eliezer was stuck on a pretty wrong path in making that argument. But it's also an undated anecdote and it may be that he never said such a thing.
- Vecr 2y ago
  Yudkowsky's not a vegetarian though, is he? Not ideologically at least, unless he changed since 2015.
  - endtime 2y ago
    Not AFAIK, and IIRC (at least as of this conversation, which was probably around 2010) he doesn't think cows are sapient either.
    - throwanem 2y ago
      Has he met one? (I have and I still eat them, this isn't a loaded question; I would just be curious to know whether and what effect that would have on his personal ethic specifically.)
- lupire 2y ago
  This shows the difference between being "bright" and being "logical". Or being "wise" vs "intelligent". Being very good at an arbitary specific game isn't the same as being smart. Prrendit that the universe is the same as your game is not wise.
  - throwanem 2y ago
    I usually find better results describing this as the orthogonality of cleverness and wisdom, and avoiding the false assumption that one is preferable in excess.
- throwanem 2y ago
  > I think EY is so surrounded by bright nerds that he has a hard time modeling average people. On reflection, I could've inferred that from his crowd's need for a concept of "typical mind fallacy." I suppose I hadn't thought it all the way through. I'm in a weird spot on this, I think. I can follow most of the reasoning behind LW/EA/generally "Yudkowskyish" analysis and conclusions, but rarely find anything in them which I feel requires taking very seriously, due both to weak postulates too strongly favored, and to how those folks can't go to the corner store without building a moon rocket first. I recognize the evident delight in complexity for its own sake, and I do share it. But I also recognize it as something I grew far enough out of to recognize when it's inapplicable and (mostly!) avoid indulging it then. The thought can feel somewhat strange, because how I see those folks now palpably has much in common with how I myself was often seen in childhood, as the bright nerd I then was. (Both words were often used, not always with unequivocal approbation.) Given a different upbringing I might be solidly in the same cohort, if about as mediocre there as here. But from what I've seen of the results, there seems no substantive reason to regret the difference in outcome.
drdeca 2y ago
I think the key word here is acausal? How can it coerce you in a way that you can’t just, be committed to not cooperating with, without first having a causal influence on you? Acausal blackmail only works if one agent U predicts the likely future (or, otherwise not-yet-having-causal-influence) existence of another agent V, who would take actions so that if U’s actions aren’t in accordance with V’s preferences, then V’s actions will do harm to U(‘s interests) (eventually). But, this only works if U predicts the likely possible existence of V and V’s blackmail. If V is having a causal influence of U, in order to do the blackmail, that’s just ordinary coercion. And, if U doesn’t anticipate the existence (and preferences) of V, then U won’t cooperate with any such attempts at acausal blackmail. (… is “blackmail” really the right word? It isn’t like there’s a threat to reveal a secret, which I typically think of as central to the notion of blackmail.)
- khafra 2y ago
  Something can be "acausal," and still change the probability you assign to various outcomes in your future event space. The classic example is in the paper "Defeating Dr. Evil with self-locating belief": https://www.princeton.edu/~adame/papers/drevil/drevil.pdf https://www.princeton.edu/~adame/papers/drevil/drevil.pdf
  - throwanem 2y ago
    Oh, good grief. I don't agree with how the other nearby commenter said it, but I do agree with what they said, especially in light of the nearby context on Yudkowsky that is also novel to me. This all evinces a vast and vastly unbalanced excess of cleverness.
  - drdeca 2y ago
    Even if it would be rational to change the probabilities one assigns to one’s future event space, that doesn’t mean one can’t commit to not considering such reasons. Now, if it’s irrational to do so, then it’s irrational to do so, even though it is possible. But I’m not so sure it is irrational. If one is considering situations with things as powerful and oppositional as that, it seems like, unless one has a full solid theory of acausal trade ready and has shown that it is beneficial, that it is probably best to blanket refuse all acausal threats, so that they don’t influence what actually happens here.

endtime 2y ago

Eliezer once told me that he thinks people aren't vegetarian because they don't think animals are sapient. And I tried to explain to him that actually most people aren't vegetarian because they don't think about it very much, and don't try to be rigorously ethical in any case, and that by far the most common response to ethical arguments is not "cows aren't sapient" but "you might be right but meat is delicious so I am going to keep eating it". I think EY is so surrounded by bright nerds that he has a hard time modeling average people. Though in this case, in his defense, average people will never hear about Roko's Basilisk.

defrost 2y ago
Despite, perhaps, all your experience to the contrary it's only a relatively recent change to a situation where "most people" have no association with the animals they eat for meat and thus can find themselves "not thinking about it very much". It's only within the past decade or so that the bulk of human population lives in an urban setting. Until that point most people did not and most people gone fishing, seen a carcass hanging in a butcher's shop, killed for food at least once, had a holiday on a farm if not worked on one or grown up farm adjacent. By most people, of course, I refer to globally. Throughout history vegetarianism was relatively rare save in vegatarian cultures (Hindi, et al) and in those cultures where it was rare people were all too aware of the animals they killed to eat. Many knew that pigs were smart and that dogs and cats interact with humans, etc. Eliezer was correct to think that people who killed to eat thought about their food animals differently but I suspect it had less to do with sapience and more to do with thinking animals to be of a lesser order, or there to be eaten and to be nutured so there would be more for the years to come. This is most evident in, sat, hunter societies, aboriginals and bushmen, who have extensive stories about animals, how they think, how they move and react, when they breed, how many can be taken, etc. They absolutely attribute a differing kind of thought, and they hunt them and try not to over tax the populations.
- endtime 2y ago
  That's all fair, but the context of the conversation was the present day, not the aggregate of all human history.
  - defrost 2y ago
    People are or are not vegetarian mostly because of their parents and the culture in which they were raised. People who are not vegetarian but have never cared for or killed a farm animal were very likely (in most parts of the world) raised by people that have. Even in the USofA much of the present generations are not far removed from grandparents who owned farms | worked farms | hunted. The present day is a continuum from yesterday. Change can happen, but the current conditions are shaped by the prior conditions.
tbrownaw 2y ago
There's a standard response to a particular PETA campaign: "Meat is murder. Delicious, delicious murder.". It's a bit odd that someone would like to argue on the topic, but also either not have heard that or not recognize the ha-ha-only-serious nature of it.
- digging 2y ago
  I believe most people would be fine with eating the meat of murdered humans, too, if it was sold on grocery store shelves for a few years. The power of normalization is immense. It sounds like Eliezer was stuck on a pretty wrong path in making that argument. But it's also an undated anecdote and it may be that he never said such a thing.
Vecr 2y ago
Yudkowsky's not a vegetarian though, is he? Not ideologically at least, unless he changed since 2015.
- endtime 2y ago
  Not AFAIK, and IIRC (at least as of this conversation, which was probably around 2010) he doesn't think cows are sapient either.
  - throwanem 2y ago
    Has he met one? (I have and I still eat them, this isn't a loaded question; I would just be curious to know whether and what effect that would have on his personal ethic specifically.)
lupire 2y ago
This shows the difference between being "bright" and being "logical". Or being "wise" vs "intelligent". Being very good at an arbitary specific game isn't the same as being smart. Prrendit that the universe is the same as your game is not wise.
- throwanem 2y ago
  I usually find better results describing this as the orthogonality of cleverness and wisdom, and avoiding the false assumption that one is preferable in excess.
throwanem 2y ago
> I think EY is so surrounded by bright nerds that he has a hard time modeling average people. On reflection, I could've inferred that from his crowd's need for a concept of "typical mind fallacy." I suppose I hadn't thought it all the way through. I'm in a weird spot on this, I think. I can follow most of the reasoning behind LW/EA/generally "Yudkowskyish" analysis and conclusions, but rarely find anything in them which I feel requires taking very seriously, due both to weak postulates too strongly favored, and to how those folks can't go to the corner store without building a moon rocket first. I recognize the evident delight in complexity for its own sake, and I do share it. But I also recognize it as something I grew far enough out of to recognize when it's inapplicable and (mostly!) avoid indulging it then. The thought can feel somewhat strange, because how I see those folks now palpably has much in common with how I myself was often seen in childhood, as the bright nerd I then was. (Both words were often used, not always with unequivocal approbation.) Given a different upbringing I might be solidly in the same cohort, if about as mediocre there as here. But from what I've seen of the results, there seems no substantive reason to regret the difference in outcome.

drdeca 2y ago

I think the key word here is acausal? How can it coerce you in a way that you can’t just, be committed to not cooperating with, without first having a causal influence on you? Acausal blackmail only works if one agent U predicts the likely future (or, otherwise not-yet-having-causal-influence) existence of another agent V, who would take actions so that if U’s actions aren’t in accordance with V’s preferences, then V’s actions will do harm to U(‘s interests) (eventually). But, this only works if U predicts the likely possible existence of V and V’s blackmail. If V is having a causal influence of U, in order to do the blackmail, that’s just ordinary coercion. And, if U doesn’t anticipate the existence (and preferences) of V, then U won’t cooperate with any such attempts at acausal blackmail. (… is “blackmail” really the right word? It isn’t like there’s a threat to reveal a secret, which I typically think of as central to the notion of blackmail.)

khafra 2y ago
Something can be "acausal," and still change the probability you assign to various outcomes in your future event space. The classic example is in the paper "Defeating Dr. Evil with self-locating belief": https://www.princeton.edu/~adame/papers/drevil/drevil.pdf https://www.princeton.edu/~adame/papers/drevil/drevil.pdf
- throwanem 2y ago
  Oh, good grief. I don't agree with how the other nearby commenter said it, but I do agree with what they said, especially in light of the nearby context on Yudkowsky that is also novel to me. This all evinces a vast and vastly unbalanced excess of cleverness.
- drdeca 2y ago
  Even if it would be rational to change the probabilities one assigns to one’s future event space, that doesn’t mean one can’t commit to not considering such reasons. Now, if it’s irrational to do so, then it’s irrational to do so, even though it is possible. But I’m not so sure it is irrational. If one is considering situations with things as powerful and oppositional as that, it seems like, unless one has a full solid theory of acausal trade ready and has shown that it is beneficial, that it is probably best to blanket refuse all acausal threats, so that they don’t influence what actually happens here.