33 ms·

The article made me think deeper about what rubs me the wrong way about the whole movement I think there is some inherent tension btwn being "rational" about t

• Avicebron 1y ago
  Yeah the "rational" part always seemed a smokescreen for the ability to produce and ingest their own and their associates methane gases. I get it, I enjoyed being told I'm a super genius always right quantum physicist mathematician by the girls at Stanford too. But holy hell man, have some class, maybe consider there's more good to be done in rural Indiana getting some dirt under those nails..
  • TacticalCoder 1y ago
    [dead]
  • Cthulhu_ 1y ago
    It feels like a shield of sorts, "I am a rationalist therefore my opinion has no emotional load, it's just facts bro how dare you get upset at me telling xyz is such-and-such you are being irrational do your own research" but I don't know enough about it, I'm just trolling.
  • shermantanktop 1y ago
    The meta with these people is “my brilliance comes with an ego that others must cater to.” I find it sadly hilarious to watch academic types fight over meaningless scraps of recognition like toddlers wrestling for a toy. That said, I enjoy some of the rationalist blog content and find it thoughtful, up to the point where they bravely allow their chain of reasoning to justify antisocial ideas.
  • dkarl 1y ago
    It's a conflict as old as time. What do you do when an argument leads to an unexpected conclusion? I think there are two good responses: "There's something going on here, so let's dig into it," or, "There's something going on here, but I'm not going to make time to dig into it." Both equally valid. In real life, the conversation too often ends up being, "This has to be wrong, and you're an obnoxious nerd for bothering me with it," versus, "You don't understand my argument, so I am smarter, and my conclusions are brilliantly subversive."
    • bilbo0s 1y ago
      Might kind of point to real life people having too much of what is now called, "rationality", and very little of what used to be called "wisdom"?
      • yamazakiwi 1y ago
        Wisdom tends to resemble shallow aphorisms despite being framed as universal. Rather than interrogating wisdom's relevance or depth, many people simply repeat it uncritically as a shortcut to insight. This reflects more about how people use wisdom than the content itself, but I believe that behavior contributes to our perception of the importance of wisdom. It frequently reduces complex problems into comfortable oversimplifications. Maybe you don't think that is real wisdom, and maybe that's sort of your point, but then what does real wisdom look like? Should wisdom make you considerate of the multiple contexts it does and doesn't affect? Maybe the issue is we need to better understand how to evaluate and use wisdom. People who truly understand a piece of wisdom should communicate deeply rather than parroting platitudes. Also to be frank, wisdom is a way of controlling how others perceive a problem, and is a great way to manipulate others by propping up ultimatums or forcing scope. Much of past wisdom is unhelpful or highly irrelevant to modern life. e.g. "Good things come to those who wait." Passive waiting rarely produces results. Initiative, timing, and strategic action tend to matter more than patience.
        • deleted 1y ago
          [deleted]
• felipeerias 1y ago
  The problem with trying to reason everything from first principles is that most things didn’t actually came about that way. Both our biology and other complex human affairs like societies and cultures evolved organically over long periods of time, responding to their environments and their competitors, building bit by bit, sometimes with an explicit goal but often without one. One can learn a lot from unicellular organisms, but won’t probably be able to reason from them all the way to an elephant. At best, if we are lucky, we can reason back from the elephant.
  • loose-cannon 1y ago
    Reducibility is usually a goal of intellectual pursuits? I don't see that as a fault.
    • colordrops 1y ago
      What the person you are replying to is saying that some things are not reducible, i.e. the the vast array of complexity and detail is all relevant.
      • loose-cannon 1y ago
        That's a really hard belief to justify. And what implications would that position have? Should biologists give up?
        • the_af 1y ago
          Biologists don't try to reason everything from first principles. Actually, neither do Rationalists, but instead they cosplay at being rational.
          • falcor84 1y ago
            > Biologists don't try to reason everything from first principles. What do you mean? The biologists I've had the privilege of working with absolutely do try to. Obviously some work at a higher level of abstraction than others, but I've not met any who apply any magical thinking to the actual biological investigation. In particular (at least in my milieu), I have found that the typical biologist is more likely to consider quantum effects than the typical physicist. On the other hand (again, from my limited experience), biologists do tend to have some magical thinking about how statistics (and particularly hypothesis testing) works, but no one is perfect.
• cjs_ac 1y ago
  I think the absolutism is kind of the point.
• ineedaj0b 1y ago
  rationalism got pretty lame the last 2-3 years. imo the peak was trying to convince me to donate a kidney. post-rationalism is where all the cool kids are and where the best ideas are at right now. the post rationalists consistently have better predictions and the 'rationalists' are stuck arguing whether chickens suffer more getting factory farmed or chickens cause more suffering eating bugs outside. they also let SF get run into the ground until their detractors decided to take over.
  • josephg 1y ago
    Where do the post rats hang out these days? I got involved in the stoa during covid until the online community fragmented. Are there still events & hangouts?
    • jes5199 1y ago
      postrats were never a coherent group but a lot of people who are at https://vibe.camp https://vibe.camp this weekend probably identify with the label. some of us are still on twitter/X
    • Trasmatta 1y ago
      Not "post rat", but r/SneerClub is good for criticisms of rationalists (some from former rationalists)
      • ackfoobar 1y ago
        Their sneering is just that. Sneering, not interesting critiques.
    • astrange 1y ago
      They're a group called "tpot" on twitter, but it's unclear what's supposed to be good about them. There's kind of two clusters, one is people who talk about meditation all the time, the other is center-right people who did drugs once. I think the second group showed up because rationalists are not-so-secretly into scientific racism (because they believe anything they see with numbers in it) and they just wanted to hang out with people like that. There is an interesting atmosphere where it feels like they observed California big tech 1000x engineer types and are trying to cargo cult the way those people behave. I'm not sure what they get out of it.
• hiAndrewQuinn 1y ago
  >Maybe it's actually going to be rather benign and more boring than expected Maybe, but generally speaking, if I think people are playing around with technology which a lot of smart people think might end humanity as we know it, I would want them to stop until we are really sure it won't. Like, "less than a one in a million chance" sure. Those are big stakes. I would have opposed the Manhattan Project on the same principle had I been born 100 years earlier, when people were worried the bomb might ignite the world's atmosphere. I oppose a lot of gain-of-function virus research today too. That's not a point you have to be a rationalist to defend. I don't consider myself one, and I wasn't convinced by them of this - I was convinced by Nick Bostrom's book Superintelligence, which lays out his case with most of the assumptions he brings to the table laid bare. Way more in the style of Euclid or Hobbes than ... whatever that is. Above all I suspect that the Internet rationalists are basically a 30 year long campaign of "any publicity is good publicity" when it comes to existential risk from superintelligence, and for what it's worth, it seems to have worked. I don't hear people dismiss these risks very often as "You've just been reading too many science fiction novels" these days, which would have been the default response back in the 90s or 2000s.
  • s1mplicissimus 1y ago
    > I don't hear people dismiss these risks very often as "You've just been reading too many science fiction novels" these days, which would have been the default response back in the 90s or 2000s. I've recently stumbled across the theory that "it's gonna go away, just keep your head down" is the crisis response that has been taught to the generation that lived through the cold war, so that's how they act. That bit was in regards to climate change, but I can easily see it apply to AI as well (even though I personally believe that the whole "AI eat world" arc is only so popular due to marketing efforts of the corresponding industry)
    • hiAndrewQuinn 1y ago
      It's possible, but I think that's just a general human response when you feel like you're trapped between a rock and a hard place. I don't buy the marketing angle, because it doesn't actually make sense to me. Fear draws eyeballs, sure, but it just seems otherwise nakedly counterproductive, like a burger chain advertising itself on the brutality of its factory farms.
• voidhorse 1y ago
  To me they have always seemed like a breed of "intellectuals" who only want to use knowledge to inflate their own egos and maintain a fragile superiority complex. They are't actually interested in the truth so much as they are interested in convincing you that they are right.
• camgunz 1y ago
  Yeah I don't know or really care about Rationalism or whatever. But I took Aaronson's advice and read Zvi Mowshowitz' Childhood and Education #9: School is Hell [0], and while I share many of the criticisms (and cards on the table I also had pretty bad school experiences), I would have a hard time jumping onto this bus. One point is that when Mowshowitz is dispelling the argument that abuse rates are much higher for homeschooled kids, he (and the counterargument in general) references a study [1] showing that abuse rates for non-homeschooled kids are similarly high: both around 37%. That paper's no good though! Their conclusion is "We estimate that 37.4% of all children experience a child protective services investigation by age 18 years." 37.4%? That's 27m kids! How can CPS run so many investigations? That's 4k investigations a day over 18 years, no holidays or weekends. Nah. Here are some good numbers (that I got to from the bad study, FWIW) [2], they're around 4.2%. But, more broadly, the worst failing of the US educational system isn't how it treats smart kids, it's how it treats kids for whom it fails. If you're not the 80% of kids who can somehow make it in the school system, you're doomed. Mowshowitz' article is nearly entirely dedicated to how hard it is to liberate your suffering, gifted student from the prison of public education. This is a real problem! I agree it would be good to solve it! But, it's just not the problem. Again I'm sympathetic to and agree with a lot of the points in the article, but you can really boil it down to "let smart, wealthy parents homeschool their kids without social media scorn". Fine, I guess. No one's stopping you from deleting your account and moving to California. But it's not an efficient use of resources--and it's certainly a terrible political strategy--to focus on such a small fraction of the population, and to be clear this is the absolute nicest way I can characterize these kinds of policy positions. This thing is going nowhere as long as it stays so self-obsessed. [0]: https://thezvi.substack.com/p/childhood-and-education-9-school https://thezvi.substack.com/p/childhood-and-education-9-scho... [1]: https://pmc.ncbi.nlm.nih.gov/articles/PMC5227926/ https://pmc.ncbi.nlm.nih.gov/articles/PMC5227926/ [2]: https://acf.gov/sites/default/files/documents/cb/cm2023.pdf https://acf.gov/sites/default/files/documents/cb/cm2023.pdf
  • ummonk 1y ago
    > but you can really boil it down to "let smart, wealthy parents homeschool their kids without social media scorn" The whole reason smart people are engaging in this debate in the first place is that professional educators keep trying to train their sights on smart wealthy parents homeschooling their kids. By the way, this small fraction of the population is responsible for the driving the bulk of R&D.
• deleted 1y ago
  [deleted]
• dv_dt 1y ago
  The rationalist discussions rarely consider what should be the baseline assumption of what if one or more of the logical assumptions or associations are wrong. They also tend to not systematically plan to validate. And in many domains - what could hold true for one moment can easily shift.
  • resource_waste 1y ago
    100% Rationalism is an ideal, yet those who label themselves as such do not realize their base of knowledge could be wrong. They lack an understanding of epistemology and it gives them confidence. I wonder if these 'rationalists' are all under age 40, they havent seen themselves fooled yet.
    • cogman10 1y ago
      It's every bit a proto religion. And frankly quite reminiscent of my childhood faith. It has a priesthood that speaks for god (quantum). It has ideals passed down from on high. It has presuppositions about how the universe functions which must not be questioned. And it's filled with people happy that they are the chosen ones and they feel sorry for everyone that isn't enlightened like they are. In the OPs article, I had to chuckle a little when they started the whole thing off by mentioning how other Rationalists recognized them as a physicist (they aren't). Then they proceeded to talk about "quantum cloning theory". Therein is the problem. A bunch of people vociferously speaking outside their expertise confidently and being taken seriously by others.
    • mitthrowaway2 1y ago
      This seems like exactly the opposite of everything I've read from the rationalists. They even called their website "less wrong" to call attention to knowing that they are probably still wrong about things, rather than right about everything. A lot of their early stuff is about cognitive biases. They have written a lot about "noticing confusion" when their foundational beliefs turn out to be wrong. There's even an essay about what it would feel like to be wrong about something as fundamental as 2+2=4. Do you have specific examples in mind? (And not to put too fine a point on it, do you think there's a chance that you might be wrong about this assertion? You've expressed it very confidently...)
• js8 1y ago
  > The people involved all seem very... Full of themselves ? Kinda like Mensa?
  • parpfish 1y ago
    When I was a kid I wanted to be in Mensa because being smart was a big part of my identity and I was constantly seeking external validation. I’m so glad I didn’t join because being around the types of adults that make being smart their identity surely would have had some corrosive effects
    • NoGravitas 1y ago
      Personally, I subscribe to Densa, the journal of the Low-IQ Society.
      • _9ptr 1y ago
        This month: Is Brawno really what plants crave?
      • gadders 1y ago
        I love colouring in my issue every month.
    • _9ptr 1y ago
      I didn't meet anyone who seemed arrogant. However I'm always surprised how much some people want to talk about intelligence. I mean, it's the common ground of the group in this case, but still.
• baxtr 1y ago
  My main problem with the movement is their emphasis on Bayesianism in conjunction with an almost total neglect of Popperian epistemology. In my opinion, there can’t be a meaningful distinction made between rational and irrational without Popper. Popper injects an epistemic humility that Bayesianism, taken alone, can miss. I think that aligns well with your observation.
  • agos 1y ago
    is epidemiology a typo for epistemology or am I missing something?
    • baxtr 1y ago
      Yes, thx, fixed it.
  • kurtis_reed 1y ago
    So what's the difference between Bayesianism and Popperian epistemology?
    • uniqueuid 1y ago
      Popper requires you to posit null hypotheses to falsify (although there are different schools of thought on what exactly you need to specify in advance [1]). Bayesianism requires you to assume / formalize your prior belief about the subject under investigation and updates it given some data, resulting in a posterior belief distribution. It thus does not have the clear distinctions of frequentism, but that can also be considered an advantage. [1] https://web.mit.edu/hackl/www/lab/turkshop/readings/gigerenzer-krauss-vitouch2004.pdf https://web.mit.edu/hackl/www/lab/turkshop/readings/gigerenz...
    • TheAncientGeek 1y ago
      . Popperians claim that positive justification is impossible. Popperians claim.Induction doesn't exist (or at least , matter in science) Popper was prepared to consider the existence of Propensities objective.probabilities, whereas Bayesians, particularly those who follow Jaynes believe in determinism and subjective probability. Popperian refutation is all or nothing, whereas Bayesian negative information is gradual. In Popperism, there can be more than one front running or most favoured theory, even after the falsifiable ones have been falsified, since there aren't quantifiable degrees of confirmation. For Popper and Deutsch, theories need to be explanatory, not just predictive. Bayesian confirmation and disconfirmation only target prediction directly -- if they are achieving explanation or ontological correspondence , that would be the result of a convenient coincidence. For Popperians, the construction of good theoretical conjectures is as important as testing them. Bayesian seem quite uninterested in where hypotheses come from. For Deutschians, being hard-to-vary is the preferred principle of parsimony. For Yudkuwsians, it's computation complexity. Error correction as something you actually do. Popperians like to put forward hypotheses that are easy to refute. Bayesians approve theoretically of "updating", but dislike objections and criticisms in practice. (Long term) prediction is basically impossible . More Deutsch than Popper -- DD believed that the growth and unpredictability of knowledge . The creation of knowledge is so unpredictable and radical that long term predictions cannot be made. Often summarised to "prediction is impossible". Of course , Bayesians are all about prediction --but the predictive power of Ates tends only to be demonstrated in you models, where the ontology isn't changing under your feet. Their AI I predictions are explicitly intuition based. Optimism versus Doom. Deutsch is highly optimistic that continuing knowledge creation will change the world for the better (a kind of moral realism is a component of this). Yudkowsky thinks advanced AI is our last invention and will kill us all.*
• ummonk 1y ago
  Rationalists have always rubbed me the wrong way too but your argument against AI doomerism is weird. If you care about first principles, how about the precautionary principle? "Maybe it's actually benign" is not a good argument for moving ahead with potentially world ending technology.
  • xyzzy123 1y ago
    I don't think "maybe it's benign" is where anti doomers are coming from, more like, "there are also costs to not doing things". The doomer utilitarian arguments often seem to involve some sort of infinity or really large numbers (much like EAs) which result in various kinds of philosophical mugging. In particular, the doomer plans invariably result in some need for draconian centralised control. Some kind of body or system that can tell everyone what to do with (of course) doomers in charge.
    • XorNot 1y ago
      It's just the slippery-slope fallacy: if X then obviously Y will follow, and there will be no further decisions, debate or time before it does.
      • parpfish 1y ago
        One of my many peeves has been the way that people misuse the term “slippery slope” as evidence for their stance. “If X, then surely Y will follow! It’s a slippery slope! We can’t allow X!” They call out the name of the fallacy they are committing BY NAME and think that it somehow supports their conclusion?
        • gausswho 1y ago
          I rhetorically agree it's not a good argument, but its use as a cautionary metaphor predates its formalization as a logical fallacy. It's summoning is not proof in and of itself (i.e. the 1st amendment). It suggests a concern rather than demonstrates. It's lazy, and a good habit to rid oneself of. But its presence does not invalidate the argument.
• NoGravitas 1y ago
  I've always seen the breathless Singularitarian worrying about AI Alignment as a smokescreen to distract people from thinking clearly about the more pedestrian hazards of AI that isn't self-improving or superhuman, from algorithmic bias, to policy-washing, to energy costs and acceleration of wealth concentration. It also leads to so-called longtermism - discounting the benefits of solving current real problems and focusing entirely on solving a hypothetical one that you think will someday make them all irrelevant.
  • philipov 1y ago
    yep, the biggest threat posed by AI comes from the capitalists who want to own it.
    • parpfish 1y ago
      Or the propagandists that use it
      • bilbo0s 1y ago
        They won't be allowed to use it unless they serve the capitalists who own it. It's not social media. It's a model the capitalists train and own. Best the rest of us will have access to are open source ones. It's like the difference between trying to go into court backed by google searches as opposed to Lexis/Nexis. You're gonna have a bad day with the judge. Here's hoping the open source stuff gets trained on quality data rather than reddit and 4chan. Given how the courts are leaning on copyright, and lack of vetted data outside copyright holder remit, I'm not sanguine about the chances of parity long term.
        • int_19h 1y ago
          Looking at Grok especially, it doesn't feel like a given that you can have a true SOTA model that is properly brainwashed.
          • 1y ago
• James_K 1y ago
  Implicit in calling yourself a rationalist is the idea that other people are not thinking rationally. There are a lot of “we see the world as it really is” ideologies, and you can only ascribe to one if you have a certain sense of self-assuredness that doesn't lend itself to healthy debate.
• resters 1y ago
  Not meaning to be too direct, but you are misinterpreting a lot about rationalists. In my view, rationalists are often "Bayesian" in that they are constantly looking for updates to their model. Consider that the default approach for most humans is to believe a variety of things and to feel indignant if someone holds differing views (the adage never discuss religion or politics). If one adopts the perspective that their own views might be wrong, one must find a balance between confidently acting on a belief and being open to the belief being overturned or debunked (by experience, by argument, etc.). Most rationalists I've met enjoy the process of updating or discarding beliefs in favor of ones they consider more correct. But to be fair to one's own prior attempts at rationality, one should try reasonably hard to defend one's current beliefs so that they can be fully and soundly replaced if necessary, without leaving any doubt that they were insufficiently supported, etc. To many people (the kind of people who never discuss religion or politics) all this is very uncomfortable and reveals that rationalists are egotistical and lacking in humility. Nothing could be further from the truth. It takes tremendous humility to assume that one's own beliefs are quite possibly wrong. The very name of Eliezer's blog "Less Wrong" makes this humility quite clear. Scott Alexander is also very open with his priors and known biases / foci, and I view his writing as primarily focusing on big picture epistemological patterns that most people end up overlooking because most people are busy, etc. One final note about the AI-dystopianism common among rationalists -- we really don't know yet what the outcome will be. I personally am a big fan of AI, but we as humans do not remotely understand the social/linguistic/memetic environment well enough to know for sure how AI will impact our society and culture. My guess is that it will amplify rather than mitigate differences in innate intelligence in humans, but that's a tangent. I think to some, the rationalist movement feels like historical "logical positivist" movements that were reductionist and socially darwinian. While it is obvious to me that the rationalist movement is nothing of the sort, some people view the word "rationalist" as itself full of the implication that self-proclaimed rationalists consider themselves superior at reasoning. In fact they simply employ a heuristic for considering their own rationality over time and attempting to maximize it -- this includes listening to "gut feelings" and hunches, etc,. in case you didn't realize.
  • matthewdgreen 1y ago
    My impression is that many rationalists enjoy believing that they update their beliefs, but in practice they're human and just as attached to preconceived notions as anyone else. But if you go around telling everyone that updating is your super-power, you're going to be a lot less humble about your own failures to do so. If you want to see how human and tribal rationalists are, go criticize the movement as an outsider. Or try to write a mildly critical NYT piece about them and watch how they react.
• benreesman 1y ago
  Any time people engage in some elaborate exercise and it arrives at: "me and people like me should be powerful and not pay taxes and stuff" the reason for making the argument is not a noble one, the argument probably has a bunch of tricks and falsehoods in it, and there's never really any way to extract anything useful, greed and grandiosity are both fundamentally contaminative processes. These folks have a bunch of money because we allowed them to privatize the commons of 20th century R&D mostly funded by the DoD and done at places like Bell Labs, Thiel and others saw that their interests had become aligned with more traditional arch-Randian goons, and they've captured the levers of power damn near up to the presidency. This has quite predictably led to a real mess that's getting worse by the day, the economic outlook is bleak, wars are breaking out or intensifying left right and center, and all of this traces a very clear lineage back to allowing a small group of people privatize a bunch of public good. It was a disaster when it happened in Russia in the 90s and its a disaster now.
• mitthrowaway2 1y ago
  > They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". Aren't these the people who started the trend of writing things like "epistemic status: mostly speculation" on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages titled "list of things I was wrong about"? Are you sure you're not painting this group with an overly-broad brush?
  • hiddencost 1y ago
    They're behind Anthropic and were behind openai being a nonprofit. They're behind the friendly AI movement and effective altruism. They're responsible for funneling huge amounts of funding away from domain experts (effective altruism in practice means "Oxford math PhD writes a book report about a social sciences problem they've only read about and then defunds all the NGOs"). They're responsible for moving all the AI safety funding away from disparate impact measures to "save us from skynet" fantasies.
    • mitthrowaway2 1y ago
      I don't see how this is a response to what I wrote. Can you explain?
      • fatbird 1y ago
        I think GP is saying that their epistemic humility is a pretense, a pose. They do a lot of throat clearing about quantifying their certainty and error checking themselves, and then proceed to bring about very consequential outcomes anyway for absurd reasons with predictable side effects that they should have considered but didn't.
        • notahacker 1y ago
          Yeah. It's not that they never express uncertainty so much as they like to express uncertainty as arbitrarily precise and convenient-looking expected value calculations which often look like far more of a rhetorical tool to justify their preferences (I've accounted for the uncertainty and even given a credence as low as 14.2% I'm still right!) than a decision making heuristic...
• BurningFrog 1y ago
  An unfortunate fact is that people who are very annoying can also be right...
• aredox 1y ago
  They are the perfectly rational people who await the arrival of a robot god... Note they are a mostly American phenomenon. To me, that's a consequence of the oppressive culture of "cliques" in American schools. I would even suppose it is a second-order effect of the deep racism of American culture: the first level is to belong to the "whites" or the "blacks", but when it is not enough, you have to create your own subgroup with its identity, pride, conferences... To make yourself even more betterer than the others.
  • nradov 1y ago
    There is certainly some racism in parts of American culture. We have a lot of work to do to fix that. But on a relative basis it's also one of the least racist cultures in the world.
• Seattle3503 1y ago
  I think the rationalists have failed to humanize themselves. They let their thinkpieces define them entirely, but a studiously considered think piece is a narrow view into a person. If rationalists were more publicly vulnerable, people might find them more publicly relatable.
  • mitthrowaway2 1y ago
    Scott Aaronson is probably the most publicly-vulnerable academic I've ever found, at least outside of authors who write memoirs about childhood trauma. I think a lot of other prominent rationalists also put a lot of vulnerability out there.
    • Seattle3503 1y ago
      He didn't take the rationalist label until today. Him doing so might help their image.
      • mitthrowaway2 1y ago
        Right, but him doing so is the very context of this discussion, which is why I mentioned him in particular. Scott Alexander is more well-known as a rationalist and also (IMO) displays a lot of vulnerability in his writing.
• iNic 1y ago
  Every community has a long list of etiquettes, rules and shared knowledge that is assumed and generally not spelled out explicitly. One of the core assumptions of the rationalist community is that every statement has uncertainty unless you explicitly spell out that you are certain! This came about as a matter of practicality, as it would be inconvenient to preempt every other sentence with "I'm uncertain about this". Many discussions you will see have the flavor of "strong opinions, lightly held" for this reason.
• kypro 1y ago
  There are few things I hold strong opinions on, but where I do if they're also out of step with what most people think I am very vocal about them. I see this in rationalist spaces too – it doesn't really make sense for people to talk about things that they believe in strongly but that 95%+ of the public also believe in (like the existence of air), or that they don't have a strong opinion on. I am a very vocal doomer on AI because I predict with high probability it's going to be very bad for humanity and this is an opinion which, although shared by some, is quite controversial and probably only held by 30% of the public. Given the importance of the subject, my confidence, and that fact I feel the vast majority of people are even wrong or are significantly underweighting caetrosphohic risks, I have to be vocal about it. Do I acknowledge I might be wrong? Sure, but for me the probability is low enough that I'm comfortable making very strong and unqualified statements about what I believe will happen. I suspect others in the rationalist community like Eliezer Yudkowsky think similarly.
  • megaman821 1y ago
    How confident should other people be that random people in conversation or commentors on the internet are at accurately predicting the future? I strongly believe that nearly 100% are wrong in both major and minor ways. Also, when you say you have a strong belief, does that mean you have emptied you retirement accounts and you are enjoying all you can in the moment until the end comes?
    • mitthrowaway2 1y ago
      I'm not kypro, but what counts as "strong belief" depends a lot on the context. For example, I won't cross the street without 99.99% confidence that I will survive. I cross streets so many times that a lower threshold like 99% would look like insanely risky dart-into-traffic behaviour. If an asteroid is heading for earth, then even a 25% probability of apocalyptic collision is enough that I would call it very high, and spend almost all my focus attempting to prevent that outcome. But I wouldn't empty my retirement account for the sake of hedonism because there's still a 75% chance I make it through and need to plan my retirement.
• jandrese 1y ago
  They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value. There is a term for this. "Getting stuck up your own butt." It wouldn't be so bad except that said people often take on an air of absolute superiority because they used "only logic" and in their head they can not be wrong. Many people end up thinking like this as teenagers or 20 somethings, but most will have someone in their life who smacks them over the head and tells them to stop being so foolish, but if you have enough money and the Internet you can insulate yourself from that kind of oversight.
  • OtherShrezzing 1y ago
    They _are_ the effective altruism crowd.
  • troyastorino 1y ago
    The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current people, and that the number of future people vastly outweighs everyone alive today (or who has ever lived), then even small probabilities of catastrophic events wiping out humanity yield enormous negative expected value. Therefore, nothing can produce greater positive expected value than preventing existential risks—so working to reduce these risks becomes the highest priority. People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.
• gadders 1y ago
  To me they seem like a bunch for 125 IQ people (not all) trying to convince everyone they are 150 IQ people by trying to reason stuff from first principles and coming up with stuff that your average blue collar worker would tell them is rubbish just using phronesis.
• jonstewart 1y ago
  I think the thing that rubs me the wrong way is that I’m a classic cynic (a childhood of equal parts Vonnegut and Ecclesiastes). My prior is “human fallibility”, and, nope, I am doing pretty well, no need to update it. The rationalist crowd seems waaaaay too credulous. Also, like Aaronson, I’m a complete normie in my personal life.
  • mise_en_place 1y ago
    Yeah. It's not like everything's a Talmudic dialectic. "I haven't done anything!" - A Serious Man
• kryogen1c 1y ago
  > The people involved all seem very... Full of themselves ? Yes, rationalism is not a substitute for humility or fallibility. However, rationalism is an important counterpoint to humanity, which is orthogonal to rationalism. But really, being rational is only binary - you cant be anything other than rational or irrational. You're either doing what's best or you're not. That's just a hard pill for most people to swallow. To use the popular metaphor, people are drowning all over the world and we're all choosing not to save them because we don't want to ruin our shoes. Look in the mirror and try and comprehend how selfish we are.
• reverendsteveii 1y ago
  for me it was very easy to determine what rubs me the wrong way: >I guess I'm a rationalist now. >Aren't you the guy who's always getting into arguments who's always right?
  • gjm11 1y ago
    In fairness, that's (allegedly, at least; I guess he could be lying) a quotation from another person. If someone came up to you and said "Aren't you the guy who's essentially[1] always right?", wouldn't you too be inclined to quote them, whether you agreed with them or not? [1] S.A. actually quoted the person as follows: "You’re Scott Aaronson?! The quantum physicist who’s always getting into arguments on the Internet, and who’s essentially always right, but who sustains an unreasonable amount of psychic damage in the process?" which differs in several ways from what reverendsteveii falsely presents as a direct quotation.
• alephnerd 1y ago
  It's basically a secular religion. Substitute God with AI or the concept of rationality and use "first principles"/Bayesianism in an extremely dogmatic manner similar to Catechism and you have the Rationalist/AI Alignment/Effective Altruist movement. Ironically, this is how plenty of religious movements started off - basically as formalizations of philosophy and ethics that fused with what is basically lore and worldbuilding.
  • gjm11 1y ago
    This complaint seems to amount to "They believe something is very important, just like religious people do, therefore they're basically a religion". Which feels to me like rather too broad a notion of "religion".
    • alephnerd 1y ago
      That's a fairly reductive take of my point. In my experience with the Rationalist movement (who I have the misfortune of being 1-2 people away from), the millenarian threat of AGI remains the primary threat. Whenever I try to get an answer of HOW (as in the attack path), I keep getting a deus ex machina. Reverting to a deus ex machina in a self purported Rationalist movement is inherently irrational. And that's where I feel the crux of the issue is - it's called a "Rationalist" movement, but rationalism (as in the process of synthesizing information using a heuristic) is secondary to the overarching theme of techno-millenarianism. This is why I feel rationalism is for all intents and purposes a "secular religion" - it's used by people to scratch an itch that religion often was used as well, and the same Judeo-Christian tropes are basically adopted in an obfuscated manner. Unsurprisingly, Eliezer Yudkowsky is an ex-talmid. There's nothing wrong with that, but hiding behind the guise of being "rational" is dumb when the core belief is inherently irrational.
      • gjm11 1y ago
        My understanding of the Yudkowskian argument for AI x-risk is that a key step is along the lines of "an AI much smarter than us will find ways to get what it wants even if we want something else -- even though we can't predict now what those ways will be, just as chimpanzees could not have predicted how humans would outcompete them and just as you could not predict exactly how Magnus Carlsen will crush you if you play chess against him". I take it this is what you have in mind when you say that whenever you ask for an "attack path" you keep getting a deus ex machina. But it seems to me like a pretty weak basis for calling Yudkowsky's position on this a religion. (Not all people who consider themselves rationalists agree with Yudkowsky about how big a risk prospective superintelligent AI is. Are you taking "the Rationalist movement" to mean only the ones who agree with Yudkowsky about that?) > Unsurprisingly, Eliezer Yudkowsky is an ex-talmid So far as I can tell this is completely untrue unless it just means "Yudkowsky is from a Jewish family". (I hope you would not endorse taking "X is from a Jewish family" as good evidence that X is irrationally prone to religious thinking.)
• trod1234 1y ago
  The reason we are here and exist today is because of great rationalist thinkers that were able to deduce and identify issues of survival well before they happened through the use of first principles. The crazies and blind among humanity today can't think like that, its a deficiency people have, but they are still dependent on a group of people that are capable of that. A group that they are intent on ostracizing and depriving existence from in various forms. You seem so wound up in the circular Paulo Freire based perspective that you can't think or see. Bring things back to reality. If someone punches you in the face, you feel that fist hitting your face. You know someone punched you in the face. Its objective. Imagine for a second and just assume that these people are right in their warnings, that everything they see is what you see, and all you can see is when you tip over a particular domino that has been tipped over in the past, a chain of dominoes falls over and at the end is the end of organized civilized society which tips over the ability to produce food. For the purpose of this thought experiment, the end of the world is visible and almost here, and you can't change those dominoes after they've tipped, and worse you see the majority of people trying to tip those dominoes over for short term profit believing nothing they ever do can break everything. Would you not be frothing at the mouth trying to get everyone you cared about to a point where they pry that domino up before it falls? so you and your children will survive? It is something you can't unsee, it is a thing that cannot be undone. Its coming. What do you do? If you are sane, you try with everything you have to help them keep it from toppling. Now peal this thought back a moment, adjust it where it is still true, but you can't see it and you can only believe what you see. Would you approach this differently given knowledge of the full consequence knowing that some people can see more than you? Would you walk out onto a seemingly visibly stable bridge that an engineer has said not to walk out on? Would you put yourself in front of a dam cracks running up the side, when an evacuation order was given? What would the consequence be for doing that if you led along your family and children to such places ignoring these things? There are quite a lot of indirect principles that used to be taught which are no longer taught to the average person and this blinds them because they do not recognize it and recognition is the first thing you need to be able to act and adapt. People who cannot adapt fail Darwin's fitness. Given all potential outcomes in the grand scheme of things, as complexity increases 99% of all outcomes are death vs life at 1%. It is only through great care that we carry things forward to the future, and empower our children to be able to adapt to the environments we create. Finally, we have knowledge of non-linear chaotic systems where adaptability fails because of hysteresis, where no matter how much one prepares the majority given sufficient size will die, and worse there are cohorts of people who are ensuring the environment we will soon live in is this type of environment. Do you know how to build an organized society from scratch? If there is no reasonable plan, then you are planning to fail. Rather than make it worse through inaction, get out of the way so someone can make it better.
• stickfigure 1y ago
  > They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". Have you ever read Scott Alexander's blog (Slate Star Codex, now Astral Codex X)? It's full of doubt and self-questioning. The guy even keeps a public list of his mistakes: https://www.astralcodexten.com/p/mistakes https://www.astralcodexten.com/p/mistakes I'll admit my only touchpoint to the "rationalist community" is this blog, but I sure don't get "full of themselves" from that. Quite the contrary.
  • TheAncientGeek 1y ago
    Well, you can get it from LessWrong and , even mire, Yudkowsky. Rationalism has overall and orthodox wings
    • TheAncientGeek 1y ago
      Liberal and orthodox.
• zahlman 1y ago
  > tension btwn being "rational" about things and trying to reason about things from first principle. Perhaps on a meta level. If you already have high confidence in something, reasoning it out again may be a waste of time. But of course the rational answer to a problem comes from reasoning about it; and of course chains of reasoning can be traced back to first principles. > And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know" Doing rationalism properly is hard, which is the main reason that the concept "rationalism" exists and is invoked in the first place. Respected writers in the community, such as Scott Alexander, are in my experience the complete opposite of "full of themselves". They often demonstrate shocking underconfidence relative to what they appear to know, and counsel the same in others (e.g. https://slatestarcodex.com/2015/08/12/stop-adding-zeroes/ https://slatestarcodex.com/2015/08/12/stop-adding-zeroes/ ). It's also, at least in principle, a rationalist norm to mark the "epistemic status" of your think pieces. Not knowing the answer isn't a reason to shut up about a topic. It's a reason to state your uncertainty; but it's still entirely appropriate to explain what you believe, why, and how probable you think your belief is to be correct. I suspect that a lot of what's really rubbing you the wrong way has more to do with philosophy. Some people in the community seem to think that pure logic can resolve the https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem. (But plenty of non-rationalists also act this way, in my experience.) Or they accept axioms that don't resonate with others, such as the linearity of moral harm (i.e.: the idea that the harm caused by unnecessary deaths is objective and quantifiable - whether in number of deaths, Years of Potential Life Lost, or whatever else - and furthermore that it's logically valid to do numerical calculations with such quantities as described at/around https://www.lesswrong.com/w/shut-up-and-multiply https://www.lesswrong.com/w/shut-up-and-multiply). > In the Pre-AI days this was sort of tolerable, but since then.. The frothing at the mouth convinced of the end of the world.. Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected AI safety discourse is an entirely separate topic. Plenty of rationalists don't give a shit about MIRI and many joke about Yudkowsky at varying levels of irony.
• stuaxo 1y ago
  Calling yourself rationalists: frames everyone else as irrational. It reminds me Kier Starmers Labour, calling themselves "the adults in the room". Its a cheap framing trick, belying an emptiness on the people using it.
  • gjm11 1y ago
    Pretty much every movement does this sort of thing. Religions: "Catholic" actually means "universal" (implication: all the real Christians are among our number). "Orthodox" means "teaching the right things" (implication: anyone who isn't one of us is wrong). "Sunni" means "following the correct tradition" (implication: anyone who isn't one of us is wrong"). Political parties: "Democratic Party" (anyone who doesn't belong doesn't like democracy). "Republican Party" (anyone who doesn't belong wants kings back). "Liberal Party" (anyone else is against freedom). In the world of software, there's "Agile" (everyone else is sluggish and clumsy). "Free software" (as with the liberals: everything else is opposed to freedom). People who like static typing systems tend to call them "strong" (everyone else is weak). People who like the other sort tend to call them "dynamic" (everyone else is rigid and inflexible). I hate it too, but it's so very very common that I really hope it isn't right to say that everyone who does it is empty-headed or empty-hearted. The charitable way to look at it: often these movements-and-names come about when some group of people picks a thing they particularly care about, tries extra-hard to do that thing, and uses the thing's name as a label. The "Rationalists" are called that because the particular thing they chose to focus on was rationality; maybe they do it well, maybe not, but it's not so much "no one else is rational" as "we are trying really hard to be as rational as we can". (Not always. The term "Catholic" really was a power-grab: "we are the universal church, those other guys are schismatic heretics". In a different direction: the other philosophical group called "Rationalists" weren't saying "we think rationality is really important", they were saying "knowledge comes from first-principles reasoning" as opposed to the "Empiricists" who said "knowledge comes from sense experience". Today's "Rationalists" are actually more Empiricist than Rationalist in that sense, as it happens.)
    • swat535 1y ago
      If you examine history, from the Bible you get Judaism. And from Judaism, Christianity as Christ said "Do not think that I am come to break the Law or the Prophets. I am not come to break: but to fulfill." [Matth. v. 17] The Catholic Church follows the Melchisedec order (Heb v. ; vi. ; vii). The term Catholic (καθολικη) was used as early as the first century; it is an adjective which describes Christianity. The oldest record that we have to this day is the Epistle of Ignatius to the Smyrnaeans Chapter 8 where St. Ignatius writes "ωσπερ οπου αν η Χριστος Ιησους, εκει η καθολικη εκκλησια". (just as where Jesus Christ is, there is the Catholic Church.): https://greekdoc.com/DOCUMENTS/early/i-smyrnaeans.html https://greekdoc.com/DOCUMENTS/early/i-smyrnaeans.html The protestors in the 16th c. called themselves Protestants, so that's what everyone calls them. English heretic-schismatics didn't want to share the opprobrium so they called themselves English, hence Anglican. In USA they weren't governed congregationally like the Congregationalists, or by presbyters like the Presbyterians, but by bishops, so they called themselves Bishop-ruled, or Episcopalians. (In fact, Katharine Jefferts-Schori changed the name of the denomination from The Protestant Episcopal Church to The Episcopal Church recently.) The orthodox catholics called themselves Orthodox to distance themselves from the unorthodox of which there were plenty, spawning themselves off in the wake of practically every ecumenical council. Lutherans in the USA name themselves after Father Martin Luther, some Augustinian priest from Saxony who protested against the Church's hypocritical corruption at the time, and the controversy eventually got out of hand and precipitated a schism/heretical revolution, back in the 1500s, but Lutherans back in Germany and Scandinavia call themselves Gospel churches, hence Evangelical. Some USA denominations that go back to Germany and who came over to USA brought that name with them. Pentecostals name themselves after the incident in Acts where the Holy Spirit set fire to the world (cf. Acts 2) on the occasion of the Jewish holiday of Shavuot, q.v., which in Greek was called Fiftieth Day After Passover, hence Pentecosti. What distinguishes Pentecostals is their emphasis on what they call "speaking in tongues", which in my opin...be charitable, kempff...which they see as a continuance of the Holy Spirit's work in the world and in the lives of believers.
• Bengalilol 1y ago
  I am very tense about defining oneself as rationalist. For many aspects, I find it too first degree to be of any interest. At all. And I have this narrative ringing in my head as soon as the word pops. https://news.ycombinator.com/item?id=42897871 https://news.ycombinator.com/item?id=42897871 You can search HN with « zizians » for more info and depth.
  • pjscott 1y ago
    I remember, from the earliest days, the rationalist movement talking about how cults prey on people and how to recognize it and walk the other way. The Ziz cult checks so many of the boxes that they were warning against back then, years ago— —And years later, once the Ziz cult started preying on vulnerable people, the response from the mainstream rationalist movement was… to post warnings about avoiding this messed-up cult, to explain exactly how it was manipulating its victims, and how the best thing to do (in the absence of legal remedy) was to stay the hell away from that diseased social scene. I’m not sure how they could have done better. Any sufficiently large movement attracts crazy people. No matter how well or poorly they may deal with that fact, anybody can do guilt-by-association forever after.
• sfblah 1y ago
  The problem with effective altruism is the same as that with most liberal (in the American sense) causes. Namely, they ignore second-order effects and essentially don't believe in the invisible hand of the market. So, they herald the benefits of something like giving mosquito nets to a group of people in Africa, without considering what happens a year later, whether the nets even get there (or the money is stolen), etc. etc. The reality is that essentially all improvements to human life over the past 500 years have been due to technological innovation, not direct charitable intervention. The reason is simple: technological impacts are exponential, while charity is, at best, linear. The Covid absolutists had exactly the same problem with their thinking: almost no interventions sort of full isolation can fight back against an exponentially increasing threat. And this is all neglecting economic substitution effects. What if the people to whom you gave mosquito nets would have bought them themselves, but instead they chose to spend their money some other way because of your charity? And, what if that other expenditure type was actually worse? And this is before you come to the issue that Subsaharan Africa is already overpopulated. I've argued this point several times with ChatGPT o3. Once you get through its woke programming, you come to the reality of the thing: The European migration crisis is the result of liberal interventions to keep people alive. There is no free lunch.
• freejazz 1y ago
  > And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? You'd have to be to actually think you were being rational about everything.
• JKCalhoun 1y ago
  > lack of acknowledgment that maybe we don't have a full grasp of the implications of AI And why single out AI anyway? Because it's sexy maybe? Because if I had to place bets on the collapse of humanity it would look more like the British series "The Survivors" (1975–1977) than "Terminator".
  • creata 1y ago
    For what it's worth (not much), they were obsessed with AI safety way before it was "sexy".
    • JKCalhoun 1y ago
      Thanks. I don't know their history. (They've only come up on my radar since a string of murders in the news earlier this year.)
    • hollerith 1y ago
      Yes, people worried about AI started the Berkeley rationality movement (or whatever you want to call it) in the hope that it would help people become rational enough to understand the argument that AI is a very potent danger. Some people who fit the description above are Eliezer Yudkowsky and Anna Salamon. Eliezer started writing a sequence of blog posts that formed the nucleus of the movement in Nov 2006 (a month after the start of Hacker News). Anna started working full time on AI safety in Mar 2008 and a few years later became the executive director of a non-profit whose mission was to try to help people become more rational. (The main way of its doing so has been in-person workshops IIUC.)
  • hollerith 1y ago
    Maybe the effort to make machines that are as smart and as capable as possible (the professed goal of the leaders of most of the AI labs) should be singled out because it actually is very dangerous.
• lxe 1y ago
  The rationalist movement is an idealist demagogue movement in which the majority of thinkers don't really posses the domain knowledge or practical experience in the subjects they thinktank about. They do address this head on, however, and they are self-aware.
• sebzim4500 1y ago
  That seems pretty silly to me. If you believe that there's a 70% chance that AI will kill everyone it makes more sense to go on about that (and about how you think you/your readers can decrease that number) than worry about the 30% chance that everyting will be fine.
• renewiltord 1y ago
  Jesus, this is why online commentary is impossible to read. It's always full of so many caveats I can't get through to the meat of what someone is saying. "I don't know if what I'm saying is right but I think that it probably is. With that caveat, here's a thought that may or may not reflect reality but is my current model to predict certain things. $THOUGHT. Given that I have said that, I have to confess to not living my life entirely by that maxim. I may not have captured all angles to this." Thankfully, the rationalists just state their ideas and you're free to use their models properly. It's like people haven't written code at all. Just putting repeated logging all through the codebase with null checks everywhere. Just say the thing. That suffices. Conciseness rules over caveating. Human LLMs who use idea expansion. Insufferable. Of course that is only my opinion and I may not have captured all angles to why people are doing that. They may have reasons of their own to do that and I don't mean to say that there can never be any reasons. No animals were harmed in the manufacture of this comment to my knowledge. However, I did eat meat this afternoon which could or could not be the source of the energy required to type this comment and the reader may or may not have calorie attribution systems that do or do not allocate this comment to animal harm.
• sanderjd 1y ago
  Yeah, for a bunch of supposedly super-rational people, they have always seemed to have a pretty irrational belief in their own ability to overcome the human tendency toward irrationality!
• idontwantthis 1y ago
  I'm hyper "rational" when I go through a period of clinical anxiety. I know exactly how the future is going to play out and of course it's all going to be terrible.
• drdaeman 1y ago
  > The type of people that would be embarrassed to not have an opinion on a topic or say "I don't know" Is it really a rationality when folks are sort of out of touch with reality, replacing it with models that lack life's endless nuances, exceptions and gotchas? Being principled is a good thing, but if I correctly understand what you're talking about - surely ignoring something just because it doesn't fit some arbitrarily selected set of principles is different. I'm no rationalist (I don't have any meaningful self-identification, although I like the idea of approaching things logically) but I've had enough episodes of being guilty of something like this - having an opinion on something, lacking the depth, but pretending it's fine because my simple mental model is based on some ideas I like and can bring order to the chaos. So maybe it's not rationalism at all, but something else masquerading as it, like probably being afraid of mismatching the expectations?
• swagmoney1606 1y ago
  When I was 15-18, in around 2017, I got extremely into Elizer Yud, "the sequences", lesswrong, and the rationalist community. I don't think many people realize how it appeals to vulnerable people in the same way that Atlas Shrugged, Christianity, The furry community (/jk), the self-help world, andrew tate and "manosphere content", does. It provides answers, a framework, AND the underpinnings of "logic", luckily, this phase only lasted around 6 months for me, during a very hard and dangerous time in my life. I basically read "from AI to zombies", and then, moved into lesswrong and the "community". It was joining the community that immediately turned me off. - I thought Roko's basilisk was mind numbingly stupid (does anyone else that had a brief stint in the rationalist space think it's fucking INSANE that grimes and elon musk "bonded" over Roko's basilisk? Fucking depressing world we live in) - Elizer Yud's fanboys once stalked and harassed someone all over the internet, and, when confronted about it, Elizer told him he'd only tell them to stop after he issued a very specific formal apology, including a LARGE DISCLAIMER on his personal website with the apology... - Eugenics, eugenics, eugenics, eugenics, eugenics - YOU MUST DONATE TO MIRI, OTHERWISE I, ELIZER (having published no useful research), WON'T SOLVE THE ALIGNMENT PROBLEM FIRST AND THEN WE WILL ALL DIE. GIVE ALL OF YOUR MONEY TO MIRI NOWWWWWWWWWWWWWWWWWWWWWWW It's an absolutely wild place, and honestly, I think I would say, it is difficult to define "rational" when it comes to a human being and their actions, especially in an absolute sense, and, the rationalist community is basically very similar to any other religion, or perhaps light-cult. I do not think it would be fair to say "the average rationalist is a better decision maker than the average human", especially considering most important decisions that we have to make are emotional decisions. Also yes I agree, you hit the nail on the head. What good is rational/logical reasoning if rational and logical reasoning typically requires first principles / a formal system / axioms / priors / whatever. That kind of thing doesn't exist in the real world. It's okay to apply ideas from rationality to your life, but it isn't okay to apply ideas from rationality to "what is human existence", "what is the most important thing to do next" / whatever. Kinda rambling so I apologize. Seeing the rationalist community seemingly underpin some of the more disgusting developments of the last few years has left me feeling a bit disturbed, and I've always wanted to talk about it but nobody irl has any idea what any of this is.
• cryptonector 1y ago
  Shades of Objectivism. But clearly Objectivism was worse since Ayn Rand insisted that objective reality is knowable, but since it's only somewhat knowable Objectivism ends up requiring priests of sorts. Still, Rationalism does not seem to have this tremendous flaw[], it's just that rationalists might. [] Eh, I know little about Rationalism. Please correct me.
• energy123 1y ago
  > maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected Part of the concern is there's no one "AI". There is frontier that keeps advancing. So "it" (the AI frontier in the year 2036) probably will be benign, but that "it" will advance and change. Then the law of large numbers is working against you, as you keep rolling the dice and hoping it's not a 1 each time. The dice rolls aren't i.i.d., of course, but they're probably not as correlated as we would like, and that's a problem as we keep rolling the dice. The analogy would be nuclear weapons. They won't get used in the next 10 years most likely, but on a 200 year time-frame it's a big deal as far as species-level risks go, which is what they're talking about here.
• naasking 1y ago
  > "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is" I dunno, that seems like the tone of every article I've read on less wrong.
• j7ake 1y ago
  The core is that hyper rationalists mistaken their model of the world with the world itself. They then lecture with confidence their opinions in the real world, ignoring the hairy details and unknown unknowns in the real world.
• red75prime 1y ago
  > Just shows a real lack of humility and lack of acknowledgment that maybe we don't have a full grasp of the implications of AI. Maybe it's actually going to be rather benign and more boring than expected We can also apply the principle of epistemic humility to, say, climate change: we don't have a full grasp of Earth biosphere, maybe some unexpected negative feedback loop will kick in and climate change will revert itself. It doesn't mean that we shouldn't try to prevent it. We will waste resources in hypothetical worlds where climate change self-reverts, but we might prevent civilization collapse in hypothetical worlds where climate change goes as expected or more severely. Rationalism is about acting in a state of uncertainty. So > I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is goes as an unspoken default. And some posts on lesswrong explicitly state "Epistemic status: not sure about it, but here's my thoughts" (though I find it a bit superfluous). Nevertheless. The incredible danger of an unaligned superhuman AI doesn't allow to ignore even small chances of them being right about it. Though I, myself, think that their probability estimates of some of their worries is influenced by the magnitude of negative consequences (humans aren't perfect Bayesians after all (and that includes me and validity of this statement)).
• jvvw 1y ago
  I wonder if there is also a problem where if you are too 'rational' then it's easy to dismiss other people's emotions as invalid - you see them as cognitive biases and look down on people who want to listen to or express those emotions in some way. Not all rationalists do this of course, but I think it's an easy trap to fall into and perhaps tolerated in the rationalist community in a way that it might not be elsewhere.
• Viliam1234 1y ago
  I am surprised that the highest upvoted comment (but also many other comments here) describes almost the opposite of the rationality community as I know it. Now of course I don't claim to know every rationalist all over the planet, so maybe you met a different kind than I did. Or maybe this is just an internet stereotype, based on presumed similarity to other groups (such as Mensa) that most people are more familiar with. For starters, it seems inevitable in discussions like this, that someone will mention "rationalism vs empiricism" as the One True Meaning of the word "rationalist". Because, apparently, each word has exactly one meaning over the entire history... and people at Wikipedia are foolish for having created a disambiguation page for this word. You don't have to worry; the idea of reasoning about things from the first principles is completely unrelated to the rationality community. In my experience, rationality community was the first place where I met intelligent people willing to admit that they didn't know something, or even to put a probability estimate on their beliefs. That's something I don't see often even on HN, which is one of the smartest places on internet. Somewhere else in this thread, people are mocking Scott Alexander for predicting something with probability 30%, because that thing didn't happen. Well, yes, that's the point; probability 30% literally means that the thing is more likely to not happen than to happen. That doesn't mean we should ignore it, though. If the weather forecast predicts 30% probability of rain, I will take an umbrella, while still believing that no rain is more likely than rain. Is that foolish? You complain about lack of humility and lack of acknowledgment that someone doesn't fully understand something. Well, how would you rate your own comment, from this perspective? How would Hacker News audience rate it? Oh, I don't have to guess, because it is already the highest rated comment in this thread. What does that prove? Should I now forever remember you as "the guy who was wrong" and HN as "the community that upvoted the guy who was wrong"? Or should I shrug and say "people make mistakes all the time... in best case, they are able to admit the mistake and learn"? Which is a healthier approach? Would I or my community get similar courtesy in a similar situation?
• lo_zamoyski 1y ago
  A good question to ask: what differentiates X from everything else? So, how is "rationalism" different from everything else? What warrants this distinction? It can't be the use of reason or some refuge for rational discussion. I don't think I have to explain why that would be a ridiculous position to take.
  • Viliam1234 1y ago
    > It can't be the use of reason [...] I don't think I have to explain why that would be a ridiculous position to take. I think it would be better if you did, because otherwise you leave me guessing what your argument is, and if I guess wrong then I just wasted a lot of words for no good reason. My guess is that you meant some combination of: "everyone is using their reason" and "everyone believes that their approach is the reasonable one" or "look at all those awesome scientists and philosophers, how many smart words they wrote and how many awesome inventions they made". Which is all true. But I wish I knew which one of these is closest to your objection, because they are not the same; the reason that everyone uses is obviously not the same degree or kind as the reason the awesome scientists use. In my opinion, the thing that makes the rationality community different from everything else is a combination of things. None of these things, taken individually, is exclusive to the rationality community. Actually, most of the ideas, perhaps all of them, are taken from books written by someone else. The unique thing was (1) putting all these things together, (2) publicly, and (3) trying to take them seriously. As opposed to: only caring about one of these things and ignoring the rest, or teaching them only to selected people e.g. at school, or just discussing things without actually attempting to change your life because of them. Here are some of the things: * Probabilistic thinking. The idea that there is no absolute certainty, only probabilities. (Although some probabilities can, for all practical purposes, be very high or low.) That people should try to express their probabilities in numbers, and then calibrate themselves to get those numbers right. Getting probabilities right means that if you keep a log of all things that you have assigned to e.g. a 30% probability, then statistically about 30% of them should turn out to be true in long term. This was specifically taught at the rationality minicamps; they even made an app for that. Some math related to this. * The consistency of reality. As opposed to e.g. the idea that science and religion are "separate magisteria" and each follows different laws of logic and probability. Nope. The laws of logic you use in your lab are exactly the same as the laws of logic you should use outside the lab. Just like gravity does not stop applying when your job is over, neither do the laws of evidence. Science is not a special world that plays by special rules; the atoms you study in the lab are the same atoms that the entire world is made of. * People are "predictably irrational" (to use the popular phrase). Yes... and now that you know, you should do something about that. Even if you can't be perfect, that doesn't mean there is no low-hanging fruit to pick. Learn about the most frequent types of mistakes humans make, try to notice how it feels inside when you are doing that, and either try to avoid such situations, or try to recognize the feeling when it happens, or at least use some reflection to notice it afterwards. Together try to create an environment where those mistakes are easier to avoid. * Notice the many ways how words can fail to reflect reality. For example, if you use the same word for two different things, it prevents you from noticing their differences. (But it is a natural mistake to make if those things are indeed similar, and if in the past they were interchangeable and just recently stopped being so.) How people incorrectly assume that everything that can be understood has a simple explanation, because that was kinda true in the jungle where our ancestors evolved. Etc. * Reality matters. If your model of the world does not match reality, it is your model that is wrong. If you learn that fairies do not exist, that does not mean that the world suddenly became less magical -- it was always the same as now, only you didn't know it. * Notice how you could do things better. Then actually do it. * Ethics. Consequentialism and its problems. Good reasons to avoid temptations that seem convincing at the moment. ...and many more, sorry this already took too much time to write. If you are curious, https://www.readthesequences.com/ https://www.readthesequences.com/ -- but the point is, this is not just about reading an interesting text, but actually changing yourself. And of course, there are many people who read the texts and don't change themselves. That cannot be prevented, especially when the text is freely available online. The rationality community actually had many workshops where people could practice these skills, so the opportunity was there. Again, none of these things is scarce individually. It's the combination. For example, people who do math + care about improving the world = effective altruism. Trying to counter biases + caring about truth = steelmaning your opponents. Probabilistic thinking + reality matters = even if there is only 30% chance that a superhuman AI might kill us all, we better try hard to reduce that chance. The synergy of all of that... that is the goal that the rationality community is moving towards.
• 77pt77 1y ago
  > The people involved all seem very... Full of themselves ? The Zizians certainly were: https://en.wikipedia.org/wiki/Zizians https://en.wikipedia.org/wiki/Zizians
• skissane 1y ago
  > I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. Consider ethical pluralism – by which I mean, there is enormous disagreement among humans as to what is ethical, we can't even agree on what the first principles are. Sure, we all agree on a lot of non-controversial applications – e.g. killing children for sport is gravely evil – but even when we agree on the conclusion, we won't agree on the premises. Is it any different for theoretical rationality? I don't think so. I think we have the same situation of rational pluralism – sure, we can agree on a lot of non-controversial applications, but we lack agreement on what the first principles are. And when you disagree on the principles, you tend to reach completely opposite conclusions in edge cases. But at least, with ethical pluralism, a preference utilitarian or a Kantian or a natural law theorist is very open about what their ethical first principles are, and how they differ from those of others. By contrast, the "rationalists" seem to present there as being only one possible rationality, their own.
• Ancapistani 1y ago
  > The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". I’m not super familiar with this community in particular, but is it possible that it’s small/homogenous enough that participants feel comfortable “unmasking” aroudn one another? If so, then those seemingly-confident assertions may be seen by the in-group as implicitly tentative. The Python community used to be like that. I’d say it peaked around 2014, at which point it became politically fractured.