3 ms·

What Are We Arguing About When We Argue About Rationality?

1 points by zdw 5y ago

PaulHoule 5y ago
There is being rational and then there is 'rationalism'. Don Rumsfeld liked to to talk about the 'known unknowns' (uncertainty built into your model) and the 'unknown unknowns' (your model is just wrong.) This problem turns up all the time when you are modeling the safety of a risky system like a nuclear power plant or the space shuttle. The earliest model of a serious nuclear accident at an LWR was that the pressure vessel was going to burst, people understand pressure vessels really well and can design one with lots of margin that the probability of a failure is minuscule. Trouble is that early model was wrong and the actual initiating event that's most common is that the power goes out, you can't circulate water, and the core becomes uncovered. That's what happened at Fukushima. The "A.I. Safety" discussion that the rationalists keep having feels a lot like that except it is the other way around, I think they are using broken models to talk up a big risk that isn't really there. (GPT-3 gets held back not to protect the world from it but so we don't see the emperor has no clothes.) I think they valorize "intelligence" too much. I think you could improve certain parameters of intelligence and not have a breakthrough to something qualitatively better than us because we are hemmed in by limitations like Godel's theorems, Ashby's law, etc. Another problem I have with rationalists is the problem I have with Reason magazine, that is the concept of "Reason" is a weapon that can be used against people who are accused of being unreasonable. (According to Reason magazine you are unreasonable if you think people should be able to join labor unions or that cars should be required to have seat belts.) For instance normal institutions that claim to be meritocratic are biased in all sorts of ways. If you hung out with the rationalists and were vociferous about their approach to A.I. safety being bogus and thought that Yudkowsky is not one of the great thinkers of the 21st century you would eventually get driven out. Success has a huge amount to do with luck and mirroring other people's narcissism. I find the whole discussion tiring: for instance it looks like Biden is appointing somebody good to the supreme court who happens to be a black women but also has a unique background for a supreme court justice that adds something to the court, on the other hand when the nation is in crisis in so many ways it seems that there are higher priorities than picking a black women for the supreme court and if one was being cynical one might say this is a way to "kill two birds with one stone" and take advantage of intersectionality to pretend you're remedying two injustices instead of remedying just one. The (basically) conservative idea that it's a holy obligation to pick the job candidate who has 95.2 units of estimated merit over the one that has 95.1 estimated units of merit is irrational because of both the known and unknown unknowns. When it comes time to do the job the actual qualities requires will deviate from the model -- I think of a professor who is a great guy and very smart and able but was in a field that became unfashionable in the time he was in the tenure process. Or someone like Volodymyr Zelenskyy who was doing awful in public opinion until he found his voice as a wartime leader. Matt Taibbi in particular expresses this kind of cold-blooded rage that demands an accounting of irrational grievances down to two or three decimal places which seems compelling to his readers and probably helps some feel superior to their foes but really it is not a place where you're going to solve problems and get us out of our cultural funk.
- jfengel 5y ago
  I could make the argument against "rationality" a lot better than the strawman the author picked. "Rationalism" is, too often, a way of insisting that one's model of the world is so self-evidently true that any objections to it are of necessity irrational. They often like to insist on simplified models, and insist that those simplifications must be fair because they're simple, while denying the ways in which they're constructed to reinforce their own preferences. Economics is one especially popular place for that. Magazines like Reason will insist on "supply and demand" to explain everything, and simply ignore when reality fails to bear it out. Supply and demand is first-week-of-the-first-year economics. It shouldn't come as a surprise that it's no more effective than the frictionless massless pulleys taught to equivalent physics students. Which is to say, not ineffective, but nobody would try to run a real economy or a real engine with just that. Rationalists aren't often very good even at the purely tangible stuff, much less the intangible stuff. Especially when that intangible stuff is things like the experiences and points of view of people who have been excluded because they've specifically defined those things as "intangible" and thus not subject to their rationalism. In general, it's helpful to be skeptical of self-complimentary labels like "rationalism". Nobody sees themselves as irrational, so what is it they're actually setting themselves up in opposition to?

1 points by zdw 5y ago

There is being rational and then there is 'rationalism'. Don Rumsfeld liked to to talk about the 'known unknowns' (uncertainty built into your model) and the 'unknown unknowns' (your model is just wrong.) This problem turns up all the time when you are modeling the safety of a risky system like a nuclear power plant or the space shuttle. The earliest model of a serious nuclear accident at an LWR was that the pressure vessel was going to burst, people understand pressure vessels really well and can design one with lots of margin that the probability of a failure is minuscule. Trouble is that early model was wrong and the actual initiating event that's most common is that the power goes out, you can't circulate water, and the core becomes uncovered. That's what happened at Fukushima. The "A.I. Safety" discussion that the rationalists keep having feels a lot like that except it is the other way around, I think they are using broken models to talk up a big risk that isn't really there. (GPT-3 gets held back not to protect the world from it but so we don't see the emperor has no clothes.) I think they valorize "intelligence" too much. I think you could improve certain parameters of intelligence and not have a breakthrough to something qualitatively better than us because we are hemmed in by limitations like Godel's theorems, Ashby's law, etc. Another problem I have with rationalists is the problem I have with Reason magazine, that is the concept of "Reason" is a weapon that can be used against people who are accused of being unreasonable. (According to Reason magazine you are unreasonable if you think people should be able to join labor unions or that cars should be required to have seat belts.) For instance normal institutions that claim to be meritocratic are biased in all sorts of ways. If you hung out with the rationalists and were vociferous about their approach to A.I. safety being bogus and thought that Yudkowsky is not one of the great thinkers of the 21st century you would eventually get driven out. Success has a huge amount to do with luck and mirroring other people's narcissism. I find the whole discussion tiring: for instance it looks like Biden is appointing somebody good to the supreme court who happens to be a black women but also has a unique background for a supreme court justice that adds something to the court, on the other hand when the nation is in crisis in so many ways it seems that there are higher priorities than picking a black women for the supreme court and if one was being cynical one might say this is a way to "kill two birds with one stone" and take advantage of intersectionality to pretend you're remedying two injustices instead of remedying just one. The (basically) conservative idea that it's a holy obligation to pick the job candidate who has 95.2 units of estimated merit over the one that has 95.1 estimated units of merit is irrational because of both the known and unknown unknowns. When it comes time to do the job the actual qualities requires will deviate from the model -- I think of a professor who is a great guy and very smart and able but was in a field that became unfashionable in the time he was in the tenure process. Or someone like Volodymyr Zelenskyy who was doing awful in public opinion until he found his voice as a wartime leader. Matt Taibbi in particular expresses this kind of cold-blooded rage that demands an accounting of irrational grievances down to two or three decimal places which seems compelling to his readers and probably helps some feel superior to their foes but really it is not a place where you're going to solve problems and get us out of our cultural funk.

jfengel 5y ago
I could make the argument against "rationality" a lot better than the strawman the author picked. "Rationalism" is, too often, a way of insisting that one's model of the world is so self-evidently true that any objections to it are of necessity irrational. They often like to insist on simplified models, and insist that those simplifications must be fair because they're simple, while denying the ways in which they're constructed to reinforce their own preferences. Economics is one especially popular place for that. Magazines like Reason will insist on "supply and demand" to explain everything, and simply ignore when reality fails to bear it out. Supply and demand is first-week-of-the-first-year economics. It shouldn't come as a surprise that it's no more effective than the frictionless massless pulleys taught to equivalent physics students. Which is to say, not ineffective, but nobody would try to run a real economy or a real engine with just that. Rationalists aren't often very good even at the purely tangible stuff, much less the intangible stuff. Especially when that intangible stuff is things like the experiences and points of view of people who have been excluded because they've specifically defined those things as "intangible" and thus not subject to their rationalism. In general, it's helpful to be skeptical of self-complimentary labels like "rationalism". Nobody sees themselves as irrational, so what is it they're actually setting themselves up in opposition to?