3 ms·

Truth be told, I don't think I fully understand all of the language of the original post, being unfamiliar with the infosec community and not having time to rea

Truth be told, I don't think I fully understand all of the language of the original post, being unfamiliar with the infosec community and not having time to read all the linked articles, but I do have a bit of familiarity with postmodern thought. That said, the post uses some strange vocabulary that probably has additional connotations in the space, so it's possible I'm talking in a completely different direction than OP. >I mean... this "fluidity" is literally what the bayesian stuff is for. Probabilistic reasoning isn't in opposition to "fuzzy boundaries", it's a formalization of it. From this description, "fluidity" isn't something new or superior, it's just informal baby steps in the direction of what later became Bayesian modeling. I might just note here, because I might be wrong, but I'm not sure this is the same thing. I believe OP is trying to describe a limitation with language and its assumption of static meaning. Consider, for instance, trying to model gender probabilistically - it doesn't quite work. If that's not to your taste, then how the distinction between the political left and right seems obvious in aggregate, but 1) transcends a fully comprehensive definition in practice, and even if you define it 2) people tend to not fall neatly into one category or another on every issue. Even political borders between states have a sort of fuzzy history. I recommend Peter Sahlins' Boundaries, which goes into the border between France and Spain, but I won't beleaguer that point. But basically trying to set probabilistic distinctions is somewhat like trying to measure a coastline - things become almost infinitely detailed the more you investigate it, and the map looks less and less like the territory. Even if you say, modeled political affiliation w/ a statistical model and simply assigned a confidence score to a predicted affiliation, what it seems that the post is claiming is that the Infosec community seems more willing to critique the underlying definitions of, say, "left" and "right". Obviously, you could counter and say that operationally these definitions work well enough, and in practice you'd most likely be right, but the willingness to recognize that definitions are not historically or socially static is probably what's being meant by "the infosec community may be ahead of society at large in understanding change, nuance, and complexity. The world is filled with conflicts over boundaries and meanings". I'm not going to go as far as to say there's a Sausseurian or Derridean conception of language here, but if that's the case then it's a different sort of uncertainty than probablistic as well. That said, I skimmed the linked "fluidity" post and really hated it. Maybe there's a coherent thought buried in the thesaurus soup but even then the self-aggrandizing tone puts me off.

TeMPOraL 4y ago
My point is that "rationalistic" approach wielding probabilistic reasoning will get the cases in your examples right. It's precisely the rationalists who I'd expect to point out that "gender" and "political affiliation" are ill-defined categories, which is why they resist useful formalization. "The willingness to recognize that definitions are not historically or socially static" is something I learned from reading rationalist pieces. Meta-point: perhaps "rationalism" and "fluidity" are themselves ill-defined categories. Perhaps the authors only ever met rationalist groups that are extremely limited in their understanding and mostly just have a crush on mathy-sounding language - and so the authors invent "fluidity", which to the rationalist groups I interacted with would be just "basics of thinking straight, explained for 5 years old". > But basically trying to set probabilistic distinctions is somewhat like trying to measure a coastline - things become almost infinitely detailed the more you investigate it, and the map looks less and less like the territory. I know those examples, precisely because of the probability-wielding rationalists, who point at such cases as curiosities, and show how to put numbers on them correctly. "All models are wrong, but some are useful." I think the authors may be complaining about the people who forgot about the "all models are wrong" part, but they themselves seem to forget the "some are useful" part. Throwing hands up in the air and saying "it always depends" isn't a solution. BTW. on a bit of a tangent, one of your examples: > If that's not to your taste, then how the distinction between the political left and right seems obvious in aggregate, but (...) I still can't imagine to whom this distinction seems "obvious in aggregate". I never found it obvious. Nor useful for anything other than cultivating hate and shutting your own thinking off. This is not a "look at me, I transcend boundaries" remark - it's just if anything is obvious, it's that political categories do not map well to how people think about issues, much less to any sort of correct or optimal beliefs. Rather, they work as attractors in the political spectacle. (This has become painfully pronounced in recent times, as people are happy to round you off to "far-${otherside}" just because you disagree with them on something.)
- rjknight 4y ago
  It's possibly that Chapman is arguing against a straw-man of Yudkowskian rationality, but it's also the case that many people really do seem to believe in something like that straw-man, and so it's necessary to argue against it anyway. It's true that in any attempt to orient oneself to the world, probabilities will be involved. Some of these are modelled implicitly and autonomously by your perceptual system, some are experienced as intuitions, and some are modelled explicitly as part of a problem-solving task, and some are considered reflectively. In the problem-solving scenario, we are taking a certain model of the situation as a given, and assigning probabilities based on that. Sometimes that's good: our model is accurate and gives us things that are relatively easy to assign probabilities to. In some situations, like casino gambling, the map/territory distinction is almost nil. We still have some problems, though: not all maps are accurate, and some accurate maps don't make it easy to assign probabilities. We can still do it - we can always assign some probability - but we might not be getting very good results any more. Changing our map requires, in Chapman's terms, changing our "stance": our idea about what the hell is going on in the situation, including our ideas about who we are in relation to it. Roughly speaking, stances are cached sets of techniques, assumptions and perspectives which make certain actions and perceptions available to us. They are associated with schools of practice, and have evolved for suitability for certain situations. We may have reason to believe that a given stance is correct for a situation, but our reason for thinking so is not that the stance gives us access to an explicit model of probabilities. A lot of this is actually pretty obvious, and we do it all the time. The feeling of "oh, this is one of those situations, I know what to do now" is familiar to most of us. To the extent that a stance gives us a probabilistic model to work with, it is by updating our perceptual and intuitive senses of what to do and what kind of result to expect. "Meta-rationality", in Chapman's terms, is the skill of maintaining a constant awareness of one's stance, the possibility of changing it, and some well-trained intuition about when to do so. Introspection on these stance-associated intuitions can be very valuable, but this is a reflective practice, requiring a much greater intensity of attention than merely acting on them (and this is generally not compatible with acting on them in real time, c.f "choking" in sports). It is more "computationally" (and hence metabolically) expensive, and is generally something one would do only after gaining familiarity with the practice. One cannot use a priori probabilistic reasoning to imagine what one would do in a situation if one were, say, skilled in the art of kung fu - one can only understand the implications of being good at kung fu by actually becoming good at it (see Agnes Callard's book Aspiration[1] on this point). After which point, it may or may not be good for your kung fu practice to examine it with the tools of probabilistic rationality. I imagine Chapman would argue that, for many stances and their associated practices, it isn't really worth it, but you'd have to ask him. Of course, we can certainly describe the entire process using probabilities: my decision to treat this as a kung fu problem rather than a diplomatic problem ultimately rests on some model I have of the likely outcomes. But this is merely descriptive: I am not "shutting up and calculating" in order to make the decision, merely rationalising it after the fact. Perhaps this does not seem particularly insightful to you, and I would agree that perhaps it should not - you may well have figured this out for yourself, and experienced rationalists may be familiar with all of it. Most of these ideas are within the grasp of thoughtful teenagers. And yet, at the same time, there really are people who act as if they believe that knowledge of probability theory will enable them to make, in near-real-time, optimal decisions even in unfamiliar domains. And most of us, if we're honest, forget just how much of a difference a change in stance can make to our perception of a situation. Many of us are naive realists who have trouble accepting that our very ideas of what is "real" in a situation is mentally constructed and dependent on our sense of what is relevant. It's not coincidental that Chapman's writings are often a re-working of certain Buddhist treatments of how our view constructs our reality. [1]: https://academic.oup.com/book/4936 https://academic.oup.com/book/4936

TeMPOraL 4y ago

My point is that "rationalistic" approach wielding probabilistic reasoning will get the cases in your examples right. It's precisely the rationalists who I'd expect to point out that "gender" and "political affiliation" are ill-defined categories, which is why they resist useful formalization. "The willingness to recognize that definitions are not historically or socially static" is something I learned from reading rationalist pieces. Meta-point: perhaps "rationalism" and "fluidity" are themselves ill-defined categories. Perhaps the authors only ever met rationalist groups that are extremely limited in their understanding and mostly just have a crush on mathy-sounding language - and so the authors invent "fluidity", which to the rationalist groups I interacted with would be just "basics of thinking straight, explained for 5 years old". > But basically trying to set probabilistic distinctions is somewhat like trying to measure a coastline - things become almost infinitely detailed the more you investigate it, and the map looks less and less like the territory. I know those examples, precisely because of the probability-wielding rationalists, who point at such cases as curiosities, and show how to put numbers on them correctly. "All models are wrong, but some are useful." I think the authors may be complaining about the people who forgot about the "all models are wrong" part, but they themselves seem to forget the "some are useful" part. Throwing hands up in the air and saying "it always depends" isn't a solution. BTW. on a bit of a tangent, one of your examples: > If that's not to your taste, then how the distinction between the political left and right seems obvious in aggregate, but (...) I still can't imagine to whom this distinction seems "obvious in aggregate". I never found it obvious. Nor useful for anything other than cultivating hate and shutting your own thinking off. This is not a "look at me, I transcend boundaries" remark - it's just if anything is obvious, it's that political categories do not map well to how people think about issues, much less to any sort of correct or optimal beliefs. Rather, they work as attractors in the political spectacle. (This has become painfully pronounced in recent times, as people are happy to round you off to "far-${otherside}" just because you disagree with them on something.)

rjknight 4y ago
It's possibly that Chapman is arguing against a straw-man of Yudkowskian rationality, but it's also the case that many people really do seem to believe in something like that straw-man, and so it's necessary to argue against it anyway. It's true that in any attempt to orient oneself to the world, probabilities will be involved. Some of these are modelled implicitly and autonomously by your perceptual system, some are experienced as intuitions, and some are modelled explicitly as part of a problem-solving task, and some are considered reflectively. In the problem-solving scenario, we are taking a certain model of the situation as a given, and assigning probabilities based on that. Sometimes that's good: our model is accurate and gives us things that are relatively easy to assign probabilities to. In some situations, like casino gambling, the map/territory distinction is almost nil. We still have some problems, though: not all maps are accurate, and some accurate maps don't make it easy to assign probabilities. We can still do it - we can always assign some probability - but we might not be getting very good results any more. Changing our map requires, in Chapman's terms, changing our "stance": our idea about what the hell is going on in the situation, including our ideas about who we are in relation to it. Roughly speaking, stances are cached sets of techniques, assumptions and perspectives which make certain actions and perceptions available to us. They are associated with schools of practice, and have evolved for suitability for certain situations. We may have reason to believe that a given stance is correct for a situation, but our reason for thinking so is not that the stance gives us access to an explicit model of probabilities. A lot of this is actually pretty obvious, and we do it all the time. The feeling of "oh, this is one of those situations, I know what to do now" is familiar to most of us. To the extent that a stance gives us a probabilistic model to work with, it is by updating our perceptual and intuitive senses of what to do and what kind of result to expect. "Meta-rationality", in Chapman's terms, is the skill of maintaining a constant awareness of one's stance, the possibility of changing it, and some well-trained intuition about when to do so. Introspection on these stance-associated intuitions can be very valuable, but this is a reflective practice, requiring a much greater intensity of attention than merely acting on them (and this is generally not compatible with acting on them in real time, c.f "choking" in sports). It is more "computationally" (and hence metabolically) expensive, and is generally something one would do only after gaining familiarity with the practice. One cannot use a priori probabilistic reasoning to imagine what one would do in a situation if one were, say, skilled in the art of kung fu - one can only understand the implications of being good at kung fu by actually becoming good at it (see Agnes Callard's book Aspiration[1] on this point). After which point, it may or may not be good for your kung fu practice to examine it with the tools of probabilistic rationality. I imagine Chapman would argue that, for many stances and their associated practices, it isn't really worth it, but you'd have to ask him. Of course, we can certainly describe the entire process using probabilities: my decision to treat this as a kung fu problem rather than a diplomatic problem ultimately rests on some model I have of the likely outcomes. But this is merely descriptive: I am not "shutting up and calculating" in order to make the decision, merely rationalising it after the fact. Perhaps this does not seem particularly insightful to you, and I would agree that perhaps it should not - you may well have figured this out for yourself, and experienced rationalists may be familiar with all of it. Most of these ideas are within the grasp of thoughtful teenagers. And yet, at the same time, there really are people who act as if they believe that knowledge of probability theory will enable them to make, in near-real-time, optimal decisions even in unfamiliar domains. And most of us, if we're honest, forget just how much of a difference a change in stance can make to our perception of a situation. Many of us are naive realists who have trouble accepting that our very ideas of what is "real" in a situation is mentally constructed and dependent on our sense of what is relevant. It's not coincidental that Chapman's writings are often a re-working of certain Buddhist treatments of how our view constructs our reality. [1]: https://academic.oup.com/book/4936 https://academic.oup.com/book/4936