6 ms·

Rationality in service to what end, though? You can't just build some complicated instruments to record measurements of the ideal polity, the way you can refine

Rationality in service to what end, though? You can't just build some complicated instruments to record measurements of the ideal polity, the way you can refine measurements of the electron's mass. Most political disagreements are about values more than wonkish policymaking. But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it. This could describe cognitive error [1] or it could describe someone who's using deontological reasoning more than consequentialist reasoning. (And perhaps lacking the vocabulary to say "consequentialist arguments won't sway my values.") It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious. I have a vague sense that torturing suspects doesn't work to prevent terrorist attacks, but that's not the root of my opposition to torture. Maybe clever torturers could make it work more often than not. That wouldn't sway me, because torturing suspects is against my values. Documenting how well it can "work" won't make me reconsider. [1] Further food for thought: "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable?

catlifeonmars 6y ago
I believe that the author makes the basic assumption that rationality is the end.
renewiltord 6y ago
Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately. As for the rest, yes, I agree with you. Often, after having considered a great deal of evidence, I condense the finding into some strength of belief about something. Now, absent the original evidence I have a position. This is usually fine since I actually lack the ability to summon all evidence on demand. It's okay, though, since I (and I suspect most people intuitively) have both a likelihood notion and a certainty in likelihood notion analogous to a Confidence Interval and P-value, if you will. So I might hold the belief that GME is going to be between 120 and infinity on Jan 13 2022 and have a certainty in that belief that is like 10% but lose the evidence that made me think that. This is also sort of a failure mode for fast Aumann Agreement even in co-incentivized individuals - the fact that memory is short. So the problem is that my certainty holding is not well done. Thanks for the thought prompt.
- astrange 6y ago
  I've always found it odd that the people called "rationalists" are not rationalist. Talking about evidence and realistic priors all the time is good (they don't do enough evidence gathering though) but it is not rationalism, it is empiricism. Surely a rationalist would be into logic and actively refuse to read studies, like the Austrian economists did.
  - renewiltord 6y ago
    It's just a terminology thing. Sort of like how atheist actuaries still account for Acts of God.
  - spiderjerusalem 6y ago
    The word 'rational' is one of the most abused in western intellectual history. Hegel considered his system 'rational' and I think it is fairly self-evident that that has nothing to do with these internet 'rationalists'.
- rsj_hn 6y ago
  > Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately. That sounds like an extraordinary claim that would require extraordinary evidence. No doubt physics and engineering have made great advances, but that doesn't mean that every social science that dresses itself up in the language of rationalism will be able to make similar advances, or really any advance at all. In fact, there is a case it will lead to regression rather than advancement: Societies are very complex things made of people who respond to changing incentives. I don't believe anyone understands them very well, nor has any field in the social sciences established a track record of making highly accurate out of sample predictions, especially for the efficacy of interventions 10, 100, 1000 generations on. Even the simple question of "what keeps a society together?" is still a puzzling one, especially now that post-modern Western societies are disintegrating before our eyes and are failing to even biologically reproduce at replacement levels. And so we turn to values. At least with values, it is possible to gain experimental knowledge in the laboratory of history. Societies reproduce values and compete with each other as well as with a hostile environment, so long lived societies should have acquired value systems more conducive to social survival. And I don't think we can make much stronger claims than that. Certainly someone just imagining stuff or doing some small experiments is unlikely to come up with robust rules for organizing society that will stand the test of time better than what comes from the long lived traditions of successful societies.
drdeca 6y ago
Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values. So, for example, in the climate change example he gives, while it is of course appropriate to take into account possible biases of people promoting one position or another, and ways that those may result from the different values that the people with those biases, one should nonetheless endeavor to have a tendency to believe what is actually true about the factual matter, even if it were that the people with the wrong values were biased as an (indirect) result of those values, in a way that led them to be more likely to reach the correct conclusion about the empirical facts, than those with the good values. (Not to say that I think this is the case in this example. Just, hypothetically. Or, what do you call something that is like a hypothetical but without expressing any position about whether the thing is actually true?) In addition, while they may get tangled up and which is which may change for a given person over time (due to human thought being fuzzy and wobbly), I think it still makes sense to make a distinction between terminal values and instrumental values. Further, if one values something instrumentally, because one believes that it is instrumentally beneficial for one's terminal values, and it is demonstrably true that it actually does not benefit one's terminal values, then one should come to believe this truth, and so no longer instrumentally value the thing. I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model. People will sometimes understand their past behavior as arising from a cognitive error, and endeavor to address this. I suppose you could regard this as simply conflicting preferences, or preferences about preferences, but I don't think this is the best way to understand things.
- philipkglass 6y ago
  Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values. That's true, and I have been convinced of some of those things. For example, I am empirically convinced that mass shootings by spree killers are shocking but not a major issue of public health or safety in the US. Even though I believe that Democratic politics gives disproportionate attention to mass shootings, weirdly mirroring how Republican politics gives disproportionate attention to Islamist terrorism, I'm not going to say so in the company of many Democrats. Because another thing I have been empirically convinced of is that people build effective political coalitions by emphasizing common ground and judiciously overlooking their allies' faults. I want action on climate issues, so I'm not going to alienate my natural allies by lecturing them about their irrational fear of semi-automatic rifles. Effective politics is almost the thematic opposite of a personal quest to aggressively probe all things in search of Truth. I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model. It's a model that doesn't lend itself to mathematically grounded predictions. But is it actually not reasonable? Without access to another person's qualia it's really easy to misattribute whether their more surprising decisions are revealing a preference, demonstrating cognitive error, or servicing a fundamentally different value orientation. Like Hume's Problem of Induction I can't effectively grapple with it but I can't exactly dismiss it either.
throwawaysea 6y ago
I agree that a lot of political discussions boil down ultimately to values. But along the way people make irrational evaluations that could be improved or corrected, and that may lead to different politics even if the values remain stable. A good concrete example of political zealots being detached from reality is the George Floyd/BLM situation. Floyd had high levels of drugs in his system, COVID-19, and was complaining about breathing in the car before there was a knee on him. Chokeholds are also a very standard and typically non lethal form of getting a suspect under control. There’s no evidence of race playing a part in this incident (no evidence of motive) or it even being an instance of police brutality (standard and typically safe procedures were used), because of these complicating factors. The news media focus, social media virality, sustained legal protests, and sustained violent rioting have corrupted people’s ability to judge this situation rationally, to the point that people overestimate the number of unarmed black people killed by police by over 50x (https://www.lawenforcementtoday.com/poll-44-of-liberals-say-more-than-1k-unarmed-black-americans-killed-by-police-in-2019-hint-super-wrong/ https://www.lawenforcementtoday.com/poll-44-of-liberals-say-...). Better rational thinking from the general population might have changed the last 10 months, and also avoided the embarrassing and dangerous cycle of Minneapolis first defunding and then refunding their police department. Then there’s the possibility that people’s values are variable as well, but I’ve not put much thought into that. I suspect however that sustained rational thinking over a period of time can affect values and so it would change politics.
- smolder 6y ago
  Your post claims to describe a good example of political zealotry and then goes on to be one. To frame Floyd's killing by police as some routine accident is ignorant of the evidence made available to you, probably because you discounted such evidence as untrustworthy and biased. Guesses about how many die to police per year are irrelevant to whether the force used was excessive.
  - ggreer 6y ago
    I think his argument is that it’s a freak accident, not a routine one. If it were routine, the stats for unarmed people dying during arrest would be much higher.
deleted 6y ago
[deleted]
roenxi 6y ago
> It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious. I believe that too. There are some boxes so dangerous we need people not to open them even if they believe they should. However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence. We don't get sewage, skyscrapers and subways by following our instincts or because some people somehow made them values. Everyone wants good sewage systems. The best parts of society come from following evidence and logic as best we are able - including, I point out, the legal system.
- philipkglass 6y ago
  However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence. I don't think that you necessarily need to sway kids while they're young. 10 years ago there were no states in the US that had legalized recreational cannabis use for adults. Today there are 17, and I expect at least another 17 over the next decade. What changed? Attitudes shifted among all age groups. It didn't seem like a process of grandparents dying out and their grandchildren entering the voting booth. It didn't seem like an evidence based process either. I was well persuaded in the 1990s that the deleterious health effects and other negative externalities of cannabis use were no worse than those corresponding to alcohol. The historical example of Prohibition was more than enough evidence for me that fueling a black market was worse than letting adults impair themselves. I wonder why evidence didn't persuade people to change laws sooner, yet the shift happened far faster than generational replacement once it started. If anyone reading has a convincing model for how the cannabis attitude shift happened: does it only rationalize history, or does it also predict other things likely to change in the near future? There are a lot of ways we can turn past series of events into coherent stories. There are fewer that successfully foretell what happens in the next chapter.
bordercases 6y ago
> But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it. “We’re an empire now, and when we act, we create our own reality. And while you’re studying that reality — judiciously, as you will — we’ll act again, creating other new realities, which you can study too, and that’s how things will sort out. We’re history’s actors . . . and you, all of you, will be left to just study what we do.” - Karl Rove
ineptech 6y ago
> "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable? "I would still be against torture no matter what the research shows" is revealed preference, and "I'm against it, therefore it doesn't work, no matter what the research shows" is cognitive error. This is clearer if you suppose that I published a good, thorough study showing that torture works very well. In the former case, you could accept it as accurate but retain your opinion; in the latter case you would be forced to either change your stance or accuse me of [ineptitude, fraud, bias, secret sadism, etc].
- db579 6y ago
  I think this is often confused because "works" is poorly defined leading both sides to interpret it as: works == achieves my aims == matches my values
benchaney 6y ago
The cognitive error being discussed here relates to a systematic failures to make accurate descriptive statements. The distinction between deontological and consequentialist thinking is entirely prescriptive. Being able to make accurate descriptive statements and understanding the accuracy of descriptive statements are valuable tools regardless of if you are a consequentialist, a deontologist or adhere to some other moral framework altogether. I believe that torture is wrong and I also believe that torture doesn't work. I could theoretically be convinced that torture works, but I would still believe that it is wrong. If I were to insist that torture doesn't work in the face of strong evidence to the contrary because of my moral beliefs that would be a cognitive error. Regarding your footnote: They cannot be the same thing because preferences are prescriptive and errors are (at least as they are being discussed here) are descriptive.

catlifeonmars 6y ago

I believe that the author makes the basic assumption that rationality is the end.

renewiltord 6y ago

Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately. As for the rest, yes, I agree with you. Often, after having considered a great deal of evidence, I condense the finding into some strength of belief about something. Now, absent the original evidence I have a position. This is usually fine since I actually lack the ability to summon all evidence on demand. It's okay, though, since I (and I suspect most people intuitively) have both a likelihood notion and a certainty in likelihood notion analogous to a Confidence Interval and P-value, if you will. So I might hold the belief that GME is going to be between 120 and infinity on Jan 13 2022 and have a certainty in that belief that is like 10% but lose the evidence that made me think that. This is also sort of a failure mode for fast Aumann Agreement even in co-incentivized individuals - the fact that memory is short. So the problem is that my certainty holding is not well done. Thanks for the thought prompt.

astrange 6y ago
I've always found it odd that the people called "rationalists" are not rationalist. Talking about evidence and realistic priors all the time is good (they don't do enough evidence gathering though) but it is not rationalism, it is empiricism. Surely a rationalist would be into logic and actively refuse to read studies, like the Austrian economists did.
- renewiltord 6y ago
  It's just a terminology thing. Sort of like how atheist actuaries still account for Acts of God.
- spiderjerusalem 6y ago
  The word 'rational' is one of the most abused in western intellectual history. Hegel considered his system 'rational' and I think it is fairly self-evident that that has nothing to do with these internet 'rationalists'.
rsj_hn 6y ago
> Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately. That sounds like an extraordinary claim that would require extraordinary evidence. No doubt physics and engineering have made great advances, but that doesn't mean that every social science that dresses itself up in the language of rationalism will be able to make similar advances, or really any advance at all. In fact, there is a case it will lead to regression rather than advancement: Societies are very complex things made of people who respond to changing incentives. I don't believe anyone understands them very well, nor has any field in the social sciences established a track record of making highly accurate out of sample predictions, especially for the efficacy of interventions 10, 100, 1000 generations on. Even the simple question of "what keeps a society together?" is still a puzzling one, especially now that post-modern Western societies are disintegrating before our eyes and are failing to even biologically reproduce at replacement levels. And so we turn to values. At least with values, it is possible to gain experimental knowledge in the laboratory of history. Societies reproduce values and compete with each other as well as with a hostile environment, so long lived societies should have acquired value systems more conducive to social survival. And I don't think we can make much stronger claims than that. Certainly someone just imagining stuff or doing some small experiments is unlikely to come up with robust rules for organizing society that will stand the test of time better than what comes from the long lived traditions of successful societies.

drdeca 6y ago

Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values. So, for example, in the climate change example he gives, while it is of course appropriate to take into account possible biases of people promoting one position or another, and ways that those may result from the different values that the people with those biases, one should nonetheless endeavor to have a tendency to believe what is actually true about the factual matter, even if it were that the people with the wrong values were biased as an (indirect) result of those values, in a way that led them to be more likely to reach the correct conclusion about the empirical facts, than those with the good values. (Not to say that I think this is the case in this example. Just, hypothetically. Or, what do you call something that is like a hypothetical but without expressing any position about whether the thing is actually true?) In addition, while they may get tangled up and which is which may change for a given person over time (due to human thought being fuzzy and wobbly), I think it still makes sense to make a distinction between terminal values and instrumental values. Further, if one values something instrumentally, because one believes that it is instrumentally beneficial for one's terminal values, and it is demonstrably true that it actually does not benefit one's terminal values, then one should come to believe this truth, and so no longer instrumentally value the thing. I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model. People will sometimes understand their past behavior as arising from a cognitive error, and endeavor to address this. I suppose you could regard this as simply conflicting preferences, or preferences about preferences, but I don't think this is the best way to understand things.

philipkglass 6y ago
Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values. That's true, and I have been convinced of some of those things. For example, I am empirically convinced that mass shootings by spree killers are shocking but not a major issue of public health or safety in the US. Even though I believe that Democratic politics gives disproportionate attention to mass shootings, weirdly mirroring how Republican politics gives disproportionate attention to Islamist terrorism, I'm not going to say so in the company of many Democrats. Because another thing I have been empirically convinced of is that people build effective political coalitions by emphasizing common ground and judiciously overlooking their allies' faults. I want action on climate issues, so I'm not going to alienate my natural allies by lecturing them about their irrational fear of semi-automatic rifles. Effective politics is almost the thematic opposite of a personal quest to aggressively probe all things in search of Truth. I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model. It's a model that doesn't lend itself to mathematically grounded predictions. But is it actually not reasonable? Without access to another person's qualia it's really easy to misattribute whether their more surprising decisions are revealing a preference, demonstrating cognitive error, or servicing a fundamentally different value orientation. Like Hume's Problem of Induction I can't effectively grapple with it but I can't exactly dismiss it either.

throwawaysea 6y ago

I agree that a lot of political discussions boil down ultimately to values. But along the way people make irrational evaluations that could be improved or corrected, and that may lead to different politics even if the values remain stable. A good concrete example of political zealots being detached from reality is the George Floyd/BLM situation. Floyd had high levels of drugs in his system, COVID-19, and was complaining about breathing in the car before there was a knee on him. Chokeholds are also a very standard and typically non lethal form of getting a suspect under control. There’s no evidence of race playing a part in this incident (no evidence of motive) or it even being an instance of police brutality (standard and typically safe procedures were used), because of these complicating factors. The news media focus, social media virality, sustained legal protests, and sustained violent rioting have corrupted people’s ability to judge this situation rationally, to the point that people overestimate the number of unarmed black people killed by police by over 50x (https://www.lawenforcementtoday.com/poll-44-of-liberals-say-more-than-1k-unarmed-black-americans-killed-by-police-in-2019-hint-super-wrong/ https://www.lawenforcementtoday.com/poll-44-of-liberals-say-...). Better rational thinking from the general population might have changed the last 10 months, and also avoided the embarrassing and dangerous cycle of Minneapolis first defunding and then refunding their police department. Then there’s the possibility that people’s values are variable as well, but I’ve not put much thought into that. I suspect however that sustained rational thinking over a period of time can affect values and so it would change politics.

smolder 6y ago
Your post claims to describe a good example of political zealotry and then goes on to be one. To frame Floyd's killing by police as some routine accident is ignorant of the evidence made available to you, probably because you discounted such evidence as untrustworthy and biased. Guesses about how many die to police per year are irrelevant to whether the force used was excessive.
- ggreer 6y ago
  I think his argument is that it’s a freak accident, not a routine one. If it were routine, the stats for unarmed people dying during arrest would be much higher.

deleted 6y ago

[deleted]

roenxi 6y ago

> It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious. I believe that too. There are some boxes so dangerous we need people not to open them even if they believe they should. However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence. We don't get sewage, skyscrapers and subways by following our instincts or because some people somehow made them values. Everyone wants good sewage systems. The best parts of society come from following evidence and logic as best we are able - including, I point out, the legal system.

philipkglass 6y ago
However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence. I don't think that you necessarily need to sway kids while they're young. 10 years ago there were no states in the US that had legalized recreational cannabis use for adults. Today there are 17, and I expect at least another 17 over the next decade. What changed? Attitudes shifted among all age groups. It didn't seem like a process of grandparents dying out and their grandchildren entering the voting booth. It didn't seem like an evidence based process either. I was well persuaded in the 1990s that the deleterious health effects and other negative externalities of cannabis use were no worse than those corresponding to alcohol. The historical example of Prohibition was more than enough evidence for me that fueling a black market was worse than letting adults impair themselves. I wonder why evidence didn't persuade people to change laws sooner, yet the shift happened far faster than generational replacement once it started. If anyone reading has a convincing model for how the cannabis attitude shift happened: does it only rationalize history, or does it also predict other things likely to change in the near future? There are a lot of ways we can turn past series of events into coherent stories. There are fewer that successfully foretell what happens in the next chapter.

bordercases 6y ago

> But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it. “We’re an empire now, and when we act, we create our own reality. And while you’re studying that reality — judiciously, as you will — we’ll act again, creating other new realities, which you can study too, and that’s how things will sort out. We’re history’s actors . . . and you, all of you, will be left to just study what we do.” - Karl Rove

ineptech 6y ago

> "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable? "I would still be against torture no matter what the research shows" is revealed preference, and "I'm against it, therefore it doesn't work, no matter what the research shows" is cognitive error. This is clearer if you suppose that I published a good, thorough study showing that torture works very well. In the former case, you could accept it as accurate but retain your opinion; in the latter case you would be forced to either change your stance or accuse me of [ineptitude, fraud, bias, secret sadism, etc].

db579 6y ago
I think this is often confused because "works" is poorly defined leading both sides to interpret it as: works == achieves my aims == matches my values

benchaney 6y ago

The cognitive error being discussed here relates to a systematic failures to make accurate descriptive statements. The distinction between deontological and consequentialist thinking is entirely prescriptive. Being able to make accurate descriptive statements and understanding the accuracy of descriptive statements are valuable tools regardless of if you are a consequentialist, a deontologist or adhere to some other moral framework altogether. I believe that torture is wrong and I also believe that torture doesn't work. I could theoretically be convinced that torture works, but I would still believe that it is wrong. If I were to insist that torture doesn't work in the face of strong evidence to the contrary because of my moral beliefs that would be a cognitive error. Regarding your footnote: They cannot be the same thing because preferences are prescriptive and errors are (at least as they are being discussed here) are descriptive.