4 ms·

In the context of my comment you’re thinking very small. Yes, providing more options for users can make an application less secure. But that force to make it se

In the context of my comment you’re thinking very small. Yes, providing more options for users can make an application less secure. But that force to make it secure is the very force I’m describing which locks down people’s options, their ability to self-express, and their options to create personal and societal value out of their actions. Every choice a programmer makes about how to model the world is a choice that is forced upon every user of their application. Many programmers lean left politically, but the same type of “fascistic” qualities they complain about in political leaders seems to go unquestioned when making a program, which dispassionately divides the world into rigid categories.

FreakLegion 2y ago
Technological homogeneity also creates single points of failure, as we saw this week with CrowdStrike. Heterogeneity, on the other hand, can (not will, but can) improve security outcomes. I remember making a comment years ago about opening PDFs in PDF.js instead of Acrobat (12 years, it turns out [1] -- PDF.js had just started shipping in Firefox). At the time, Acrobat was the standard and widely exploited, but obviously none of those exploits did anything in the browser. Still, homogeneity isn't always bad. If I were in, say, the cyber insurance racket, I'd make any company not using U2F or FIDO2 pay up. Not all choices are good choices. 1. https://news.ycombinator.com/item?id=4033804 https://news.ycombinator.com/item?id=4033804
polynox 2y ago
It is not the "dispassionate dividing of the world into rigid categories" necessary in order to implement software that is morally or ethically problematic, it is the imposition of those rigid categories upon society at large, perhaps by itself or perhaps merely just the conflation of the separation between the categories that gets incorporated into the design as a subconscious or even conscious choice to reify the categorical differences that is the problem. It is not the knowledge itself that is the problem, it is how that knowledge gets incorporated into an overly simplistic and heavy-handed approach to act. Indeed that is the very thesis of the Seeing like a State book. To take the first example of scientific forestry that replaced the local control over the land in which the categories were the species of plants as an example, it was not the factual knowledge about the species of plants, which is an observation about the true nature of reality, it was the decision to use that fact about the distinction between the plants and decide to separate the plants geographically to uniform monoculture plots rather than allowing them to be mixed together (and, in the process, in fact destroying things that they did not understand, the ecosystem) that is argued to have been the cause of the scientific forestry failure. The plots were just fine before and after the knowledge about what plants were where and what the different types of plants there were became known; it was the act of separating the plants into the monoculture plots. The knowledge was a necessary prerequisite to do so, but it was not sufficient, and I don't know if I could believe that accurate knowledge of actual pre-existing categories that correspond to the actual universe could by itself be wrong. Certain plants really are different than other plants, species actually exist. What you decide to do with that information is up to you, but I don't know if I could support the suppression of the acquisition of knowledge merely because it might (but also might not) be used in ways we disagree. If they had merely studied the plants, that would not have been wrong. If they had created a map of where the plants were, that also would not have been wrong in my view. It was when they decided to change reality that the moral implications attach, in my view. They could have mapped out the plants and divided the map into categories: "there are oak trees here and not here", or "there are strawberry bushes here, and not here", and that is "dividing the world into categories" (namely: presence or absence of certain plants in certain locations), but that is merely the true expression of factual reality. It was how they decided as a separate step, "let's make all the strawberry bushes be in the same location" which incorporated many smaller actions like "let's move this strawberry bush from here to here" or "let's remove all the plants we have not specifically identified, as we (falsely) believe that they are not essential" in which reality was actually changed, and thus the moral or ethical implication attaches in my view. Those who act have a duty to act responsibly, to do no harm, and so on, and the duty scales with the scope of the action, and perhaps there is a scope of action which by itself is too large, the standard or duty of care too impossible to satisfy due to the sheer complexity, for any one person or collection of people to rightfully possess what would be obligatory in my view to have the permission to act. I would also not view the mere publication of software as action per se either. In your example, it is the utilization of the marketing channels to push the software onto users with or without their knowledge, to exercise editorial control over what software may or may not be used on one's supposedly own device, and so on that the duty applies. If someone publishes a program that divides the world into categories, but no one uses the program, it doesn't matter how wrong the program is. The program can have a field for skin color that is literally black and white for the purpose of calibrating an algorithm that divides pixels into skin-or-not-skin, for example to cut out the background, and yet if no one actually uses, much less is forced to use the program then the mechanism of distinguishing between skin-and-not-skin being incorrect and not for example a scalar or some other model does not harm anyone. It's just an overly simplistic program, a piece of text, an incomplete idea, a reflection of a categorical distinction that does not exist in reality but only in the mind of the programmer and in the software, but if no one ever uses the software let alone is forced to use it, who possibly can be harmed by it?
- sho 2y ago
  The "scientific management" was just one example out of many, and I disagree its lessons form the thesis of the book. The thesis as I understood it was that states seek legibility of domains under their jurisdiction, and the example of the forests was demonstrating the bad outcomes arising from seeking that legibility through misguided means. While unfortunate, that was an implementation error that could - and indeed was - corrected. I'd say a much better illustrative example from the book was the introduction of last names. In that case, faced with the illegible identity of the people under its rule, a whole new construct was invented, imposed and pertains to this day. This one really opened my eyes. The whole reason we have last names at all is because our society just sort of invented them, hundreds of years ago, for its own convenience, and it was so successful we all just accept it as the way things are. Now that is seeing - and acting - like a state!
  - polynox 2y ago
    I agree and you are correct that concept of legibility was central, however going from the text now I believe that James takes specific issue with joint combination of both the exercise of state authority as well as the blindness that is implied by high modernism. Starting on page 88: > I believe that many of the most tragic episodes of state development in the late nineteenth and twentieth centuries originate in a particularly pernicious combination of three elements. The first is the aspiration to the administrative ordering of nature and society, ... The second element is the unrestrained use of the power of the modern state as an instrument for achieving those designs. The third element is a weakened or prostrate civil society that lacks the capacity to resist those plans. I believe my comment still stands, in responding to not the book but the author of the comment, in that framework I'm describing, assuming the thesis of the book is true, then the second element is not met by the mere publication of software. Rather it is the foisting of such software upon users that is the act of power or other covert means by which choice is restricted such as controlling the bounds of publication or else the centralizing nature of capital investment in software that by sheer bad luck it turns out that perhaps we just end up with a few pseudo monopolies because software as a capital expense is ... well, expensive to develop.
  - MichaelZuo 2y ago
jagged-chisel 2y ago
> In the context of my comment you’re thinking very small. I am sorry that you failed to acknowledge how my comment is applicable at scale. > [T]hat force to make it secure is the very force I’m describing which locks down people’s options, their ability to self-express, and their options to create personal and societal value out of their actions. Yes. You see how it applies. The initial attack was uncalled for.
mrbombastic 2y ago
I find this a bit hyperbolic, a program is a tool, code is a means to build that tool, restricting the feature set of a tool is not fascistic it is a design choice that is often for the benefit of the user, although I will concede certainly not always. If the user doesn’t feel your tool gives them the right options they can pick another tool. is making a hammer that hits nails fascistic because the user could have used the raw materials to make a screwdriver? Or maybe we shouldn’t build any more programs, your favorite programing language already can edit your photos! Good luck grandma!

FreakLegion 2y ago

Technological homogeneity also creates single points of failure, as we saw this week with CrowdStrike. Heterogeneity, on the other hand, can (not will, but can) improve security outcomes. I remember making a comment years ago about opening PDFs in PDF.js instead of Acrobat (12 years, it turns out [1] -- PDF.js had just started shipping in Firefox). At the time, Acrobat was the standard and widely exploited, but obviously none of those exploits did anything in the browser. Still, homogeneity isn't always bad. If I were in, say, the cyber insurance racket, I'd make any company not using U2F or FIDO2 pay up. Not all choices are good choices. 1. https://news.ycombinator.com/item?id=4033804 https://news.ycombinator.com/item?id=4033804

polynox 2y ago

It is not the "dispassionate dividing of the world into rigid categories" necessary in order to implement software that is morally or ethically problematic, it is the imposition of those rigid categories upon society at large, perhaps by itself or perhaps merely just the conflation of the separation between the categories that gets incorporated into the design as a subconscious or even conscious choice to reify the categorical differences that is the problem. It is not the knowledge itself that is the problem, it is how that knowledge gets incorporated into an overly simplistic and heavy-handed approach to act. Indeed that is the very thesis of the Seeing like a State book. To take the first example of scientific forestry that replaced the local control over the land in which the categories were the species of plants as an example, it was not the factual knowledge about the species of plants, which is an observation about the true nature of reality, it was the decision to use that fact about the distinction between the plants and decide to separate the plants geographically to uniform monoculture plots rather than allowing them to be mixed together (and, in the process, in fact destroying things that they did not understand, the ecosystem) that is argued to have been the cause of the scientific forestry failure. The plots were just fine before and after the knowledge about what plants were where and what the different types of plants there were became known; it was the act of separating the plants into the monoculture plots. The knowledge was a necessary prerequisite to do so, but it was not sufficient, and I don't know if I could believe that accurate knowledge of actual pre-existing categories that correspond to the actual universe could by itself be wrong. Certain plants really are different than other plants, species actually exist. What you decide to do with that information is up to you, but I don't know if I could support the suppression of the acquisition of knowledge merely because it might (but also might not) be used in ways we disagree. If they had merely studied the plants, that would not have been wrong. If they had created a map of where the plants were, that also would not have been wrong in my view. It was when they decided to change reality that the moral implications attach, in my view. They could have mapped out the plants and divided the map into categories: "there are oak trees here and not here", or "there are strawberry bushes here, and not here", and that is "dividing the world into categories" (namely: presence or absence of certain plants in certain locations), but that is merely the true expression of factual reality. It was how they decided as a separate step, "let's make all the strawberry bushes be in the same location" which incorporated many smaller actions like "let's move this strawberry bush from here to here" or "let's remove all the plants we have not specifically identified, as we (falsely) believe that they are not essential" in which reality was actually changed, and thus the moral or ethical implication attaches in my view. Those who act have a duty to act responsibly, to do no harm, and so on, and the duty scales with the scope of the action, and perhaps there is a scope of action which by itself is too large, the standard or duty of care too impossible to satisfy due to the sheer complexity, for any one person or collection of people to rightfully possess what would be obligatory in my view to have the permission to act. I would also not view the mere publication of software as action per se either. In your example, it is the utilization of the marketing channels to push the software onto users with or without their knowledge, to exercise editorial control over what software may or may not be used on one's supposedly own device, and so on that the duty applies. If someone publishes a program that divides the world into categories, but no one uses the program, it doesn't matter how wrong the program is. The program can have a field for skin color that is literally black and white for the purpose of calibrating an algorithm that divides pixels into skin-or-not-skin, for example to cut out the background, and yet if no one actually uses, much less is forced to use the program then the mechanism of distinguishing between skin-and-not-skin being incorrect and not for example a scalar or some other model does not harm anyone. It's just an overly simplistic program, a piece of text, an incomplete idea, a reflection of a categorical distinction that does not exist in reality but only in the mind of the programmer and in the software, but if no one ever uses the software let alone is forced to use it, who possibly can be harmed by it?

sho 2y ago
The "scientific management" was just one example out of many, and I disagree its lessons form the thesis of the book. The thesis as I understood it was that states seek legibility of domains under their jurisdiction, and the example of the forests was demonstrating the bad outcomes arising from seeking that legibility through misguided means. While unfortunate, that was an implementation error that could - and indeed was - corrected. I'd say a much better illustrative example from the book was the introduction of last names. In that case, faced with the illegible identity of the people under its rule, a whole new construct was invented, imposed and pertains to this day. This one really opened my eyes. The whole reason we have last names at all is because our society just sort of invented them, hundreds of years ago, for its own convenience, and it was so successful we all just accept it as the way things are. Now that is seeing - and acting - like a state!
- polynox 2y ago
  I agree and you are correct that concept of legibility was central, however going from the text now I believe that James takes specific issue with joint combination of both the exercise of state authority as well as the blindness that is implied by high modernism. Starting on page 88: > I believe that many of the most tragic episodes of state development in the late nineteenth and twentieth centuries originate in a particularly pernicious combination of three elements. The first is the aspiration to the administrative ordering of nature and society, ... The second element is the unrestrained use of the power of the modern state as an instrument for achieving those designs. The third element is a weakened or prostrate civil society that lacks the capacity to resist those plans. I believe my comment still stands, in responding to not the book but the author of the comment, in that framework I'm describing, assuming the thesis of the book is true, then the second element is not met by the mere publication of software. Rather it is the foisting of such software upon users that is the act of power or other covert means by which choice is restricted such as controlling the bounds of publication or else the centralizing nature of capital investment in software that by sheer bad luck it turns out that perhaps we just end up with a few pseudo monopolies because software as a capital expense is ... well, expensive to develop.
- MichaelZuo 2y ago

jagged-chisel 2y ago

> In the context of my comment you’re thinking very small. I am sorry that you failed to acknowledge how my comment is applicable at scale. > [T]hat force to make it secure is the very force I’m describing which locks down people’s options, their ability to self-express, and their options to create personal and societal value out of their actions. Yes. You see how it applies. The initial attack was uncalled for.

mrbombastic 2y ago

I find this a bit hyperbolic, a program is a tool, code is a means to build that tool, restricting the feature set of a tool is not fascistic it is a design choice that is often for the benefit of the user, although I will concede certainly not always. If the user doesn’t feel your tool gives them the right options they can pick another tool. is making a hammer that hits nails fascistic because the user could have used the raw materials to make a screwdriver? Or maybe we shouldn’t build any more programs, your favorite programing language already can edit your photos! Good luck grandma!