6 ms·

When Algorithms Rule, Values Can Wither

63 points by otoburb 4y ago

aebtebeten 4y ago
From The Onion I recently learned about psychopathy[0], one[1] of the constituents of which is meanness. > Meanness entails deficient empathy, lack of affiliative capacity, contempt toward others, predatory exploitativeness, and empowerment through cruelty or destructiveness. I would not be surprised if any algorithm selected to be effective would wind up externalising its costs on others, hence displaying meanness. [0] https://www.theonion.com/things-all-cats-do-that-prove-they-are-psychopaths-1849592141/slides/2 https://www.theonion.com/things-all-cats-do-that-prove-they-... [1] the other aspects are Disinhibition and Boldness. Comparing to Aristotle's Virtues, do we have reasonable matches between: Boldness : excessive Fortitude Disinhibition : insufficient Temperance Meanness : Wisdom (knowing how to act for one's own benefit) without Justice (knowing how to act for everyone's benefit) ?
hprotagonist 4y ago
I don't know how to communicate this, or even if it is possible. But the question of justice has concerned me greatly of late. And I say to any creature who may be listening, there can be no justice so long as laws are absolute. Even life itself is an exercise in exceptions. -- Jean-Luc Picard
- kayodelycaon 4y ago
  From Star Trek: The Next Generation, Season 1, Episode 7 “Justice”. :)
- lostmsu 4y ago
  There's an ultimate justice machine with absolute laws though: the universe itself.
  - hprotagonist 4y ago
    i’m not convinced by that framing for many reasons, but it’s worth noting that so far it seems that more than a few “laws of the universe” end with the phrase “empirical observations will fall along the following probability distribution”, which is not how we usually understand “absolute”.
wolverine876 4y ago
I think the same may apply when people are trained and experienced in algorithms (and other technology) and not in values (and other aspects of humanities). The extreme corruption I perceive in SV could be, IMHO, partly a consequence. Most issues in SV, such as free speech, fraud, AI's impact on society, disinformation and misinformation, labor rights, the concentration of wealth and power, the role of government, mob rule online, narcissism and megalomania in leaders, the corruption of power, etc. etc. are mainly humanities issues. Many in technology have disparaged and avoided humanities education - itself an act of basic egocentric bias and a lack of skepticism - and it shows in the outcomes.
- echosha 4y ago
  This is absolutely true. The typical tech CEO is a guy who got one B in a college history course and decided the whole subject was just so far beneath his genius as to be completely unworthy of study. And so... we get what we've currently got. Paul Graham, for his flaws, was at least a painter.
  - barry-cotter 4y ago
    I know, right? Peter Thiel is so much more enlightened, with his philosophy and law degrees. Or Zuckerberg with his deep interest in psychology and fluency in Latin and devotion to the Stoics.
    - wolverine876 4y ago
      Why are these argumentative and empty statements the best we can do on HN?
- amadeuspagel 4y ago
  Alex Karp has a PhD in philosophy. So there's at least one company that we can trust to navigate these issues.
  - AngriestLettuce 4y ago
joe_the_user 4y ago
Interest in the possibilities afforded by algorithms and big data continues to blossom as early adopters gain benefits from AI systems that automate decisions as varied as making customer recommendations, screening job applicants, detecting fraud, and optimizing logistical routes. No doubt optimizing logistical routes is a problem for algorithm and has been studied since, idk, mid 1920s? With the invention of linear programming? Or maybe later but much earlier than the current wave. In contrast, certain screen job applicants is done aggressively now, no doubt but there's not evidence imo that scanning does more than reduce interviewer workload. And product recommendations is one of those constantly talked-of and generally bullshitty applications that average person can verify the ineffectiveness of. Which is to say, perhaps algorithms today wind-up being used because they allow the meta-corp act a blind juggernaut, that this is the primary advantage the approach has, with the secondary advantage being justifying that operation and "greater efficiency" sometimes results but is more often an excuse.
amadeuspagel 4y ago
Where Paywalls Rule ... Socrates once had to admit that he did not understand the issue of names, because he was not able to afford the 50 drachma course on the matter. I'm now similarly embarrassed. I do not understand what it means for algorithms to rule (are laws algorithms? are procedures?) or for values to wither. I can only ask whether the fundamental problem in the robodebt scandal was not the shift in the burden of proof.
- OkayPhysicist 4y ago
  Laws are not algorithms, because a human legal system has wiggle room. You can negotiate, persuade, and the system is capable of dealing with unanticipated input. An algorithm is a strict series of steps, executed based on pre-determined allowable inputs. They are preferable to the ones defining them because A) they can be automated and B) they deflect blame: "It's not my fault, it's just policy" or "it's just the way the program works". They thus act as a proxy for the ruling class to operate through. "Values" are ideas that we, as a society, "value". Compassion, understanding, etc. These tend to involve nuance, which a system designed in advance and applied blindly to a given circumstance inherently lacks. Thus, in case you aren't simply feigning ignorance, the title could be expanded to: "When individuals with existing power over others design rigid systems to be applied indiscriminately to those beneath them for both convenience and to redirect the perception of responsibility, it leads to inhumane consequences."
  - amadeuspagel 4y ago
    > Laws are not algorithms, because a human legal system has wiggle room. Is this "wiggle room" the fundamental difference between laws and algorithms, so fundamental that we have to fear values withering when algorithms rule, but not when laws rule? Is it the reason the ruling class can't operate through laws, only through algorithms? Does it prevent humans executing policies from saying that "it's not my fault, it's just policy"?
    - JumpCrisscross 4y ago
      > this "wiggle room" the fundamental difference between laws and algorithms, so fundamental that we have to fear values withering when algorithms rule, but not when laws rule? Yes. No rule is clairvoyant. All exceptions cannot be anticipated; exceptions have to be adapted to or stomped out. The law aims to do the former. Algorithms deliver the latter. > the ruling class can't operate through laws, only through algorithms Dictators’ decrees are closer to algorithms than law. They’re absolute in a way laws are not. To the degree they diverge, it’s in the enforcement, which is a sloppier version of the law.
armchairhacker 4y ago
This is just bureaucracy, which has existed long before AI. Ok, it's not just bureaucracy, because the mistakes that AI makes to favor efficiency are different than those made by a large chain of overworked and sometimes corrupt people. But the root issue is not AI, it's the badly-implemented policies that the AI is designed to carry out. Look at any other strained system (food stamps, DHS, IRS, not government but Google and Amazon customer support), we have real humans carrying out bad policies and it's not much better. Most people really do have morals and don't like hurting others, but those morals only go so far when you're part of an underfunded system, apathetic from burnout, and you literally don't have the resources or ability to make the morally-correct choice. All this debate on how AI lacks empathy and common sense place way too much value on humans who also lack empathy and common sense, and while an AI programmed to do something unethical won't hesitate, history and experiments show that some humans won't either. We definitely should be learning and understanding the limits of AI and its hold on ethics, and not delegate policies like laws and income to 100% automated systems. But AI and improved efficiency really can help a lot of government institutions. Especially when the AI takes over the mundane, and the people in charge can handle the more difficult cases. Because then those people can spend more of their "human" empathy and decision-making better.
- javajosh 4y ago
  >This is just bureaucracy... I don't think so. Software-powered decisions do share a feature with any bureaucracy: the inscrutability of it. Both the IRS and the Google datacenter are black boxes that are enormously powerful, automated, and dangerous. The ham-handed way in which they behave can sometimes be traced to the virtue of simplicity: do it the same way every time, without regard to context. Any programmer recognizes this as the simplest possible program! And if you decide to build in more "policy wiggle room", this often has a perverse effect of being immediately and ruthlessly exploited in ways that are difficult or impossible to enforce. The new thing with software decisions is the unaccountably for responsible humans. It's like the corporate veil, but much worse. This one is..real, physical. A human can say not only didn't they perform the action (the same is true using a human underling) but now they can say a) I didn't set the policy, It did, and b) I don't know where It runs, or how to change It or stop It. I imagine it will be quite the fashion for hyper wealthy c-suite to delegate to an AI, enjoy life and the continued high remuneration and decrease of accountability and liability. The other thing is that, in general, software accelerates complexity and never reduces it. We live in a truly science-fiction era where we have problems that our society can cheaply reproduce devices (chips) with 10^10 microscopic states, and we forbid each other from looking, or even knowing, what those states are. We buy and sell these tiny machines, and we don't really know what's inside them. We connect them to the internet! There is just so much space for things to hide, it's frightening. Now consider a typical bureaucracy tends to do more poorly the more complex its inner workings; now add thousands, millions of computers, with 10 generations of programmers blood soaked code (note: programmer generations are like 5 years long).
voidhorse 4y ago
In my opinion, the problem is much bigger than just a class of algorithms (AI) and is largely a side-effect of late-capitalism. The primary issue is that software systems are inherently rule-driven (algorithmic) and that encoded rules are just that--encoded, fixed--for some amount of time during a release cycle, a system will operate in a fixed way. Private companies are the ones designing and developing these systems, and as far as private companies are concerned the only value they care about is profit. Ethics hardly enters into discussion when it comes to the extraction of the maximum capital possible. As long as there's no pre-established juridical ruling against it, the capitalist is not going to eliminate any approach for ethical reasons. Worse, because the algorithm is the product, these rules are carefully guarded and as the product grows in popularity it is not only the experience, but the algorithmic mode of reasoning itself that begins to dominate--people are complacent about, e.g. horrendous privacy practices because an ethics of digital privacy was never a question to begin with and as soon as companies rolled out products built on this absence it became normalized. Algorithms in themselves do not make values wither. A society in which anything other than monetary value was already never given consideration establishes an environment in which values wither, whether or not that erosion is realized by digital or mechanical means. It all comes back to capital baby. In other words, algorithms are one means of realizing the obfuscation an a-ethical modus operandi the grand abstraction of near unbridled capital already permits.

63 points by otoburb 4y ago

From The Onion I recently learned about psychopathy[0], one[1] of the constituents of which is meanness. > Meanness entails deficient empathy, lack of affiliative capacity, contempt toward others, predatory exploitativeness, and empowerment through cruelty or destructiveness. I would not be surprised if any algorithm selected to be effective would wind up externalising its costs on others, hence displaying meanness. [0] https://www.theonion.com/things-all-cats-do-that-prove-they-are-psychopaths-1849592141/slides/2 https://www.theonion.com/things-all-cats-do-that-prove-they-... [1] the other aspects are Disinhibition and Boldness. Comparing to Aristotle's Virtues, do we have reasonable matches between: Boldness : excessive Fortitude Disinhibition : insufficient Temperance Meanness : Wisdom (knowing how to act for one's own benefit) without Justice (knowing how to act for everyone's benefit) ?

hprotagonist 4y ago

I don't know how to communicate this, or even if it is possible. But the question of justice has concerned me greatly of late. And I say to any creature who may be listening, there can be no justice so long as laws are absolute. Even life itself is an exercise in exceptions. -- Jean-Luc Picard

kayodelycaon 4y ago
From Star Trek: The Next Generation, Season 1, Episode 7 “Justice”. :)
lostmsu 4y ago
There's an ultimate justice machine with absolute laws though: the universe itself.
- hprotagonist 4y ago
  i’m not convinced by that framing for many reasons, but it’s worth noting that so far it seems that more than a few “laws of the universe” end with the phrase “empirical observations will fall along the following probability distribution”, which is not how we usually understand “absolute”.

wolverine876 4y ago

I think the same may apply when people are trained and experienced in algorithms (and other technology) and not in values (and other aspects of humanities). The extreme corruption I perceive in SV could be, IMHO, partly a consequence. Most issues in SV, such as free speech, fraud, AI's impact on society, disinformation and misinformation, labor rights, the concentration of wealth and power, the role of government, mob rule online, narcissism and megalomania in leaders, the corruption of power, etc. etc. are mainly humanities issues. Many in technology have disparaged and avoided humanities education - itself an act of basic egocentric bias and a lack of skepticism - and it shows in the outcomes.

echosha 4y ago
This is absolutely true. The typical tech CEO is a guy who got one B in a college history course and decided the whole subject was just so far beneath his genius as to be completely unworthy of study. And so... we get what we've currently got. Paul Graham, for his flaws, was at least a painter.
- barry-cotter 4y ago
  I know, right? Peter Thiel is so much more enlightened, with his philosophy and law degrees. Or Zuckerberg with his deep interest in psychology and fluency in Latin and devotion to the Stoics.
  - wolverine876 4y ago
    Why are these argumentative and empty statements the best we can do on HN?
amadeuspagel 4y ago
Alex Karp has a PhD in philosophy. So there's at least one company that we can trust to navigate these issues.
- AngriestLettuce 4y ago

joe_the_user 4y ago

Interest in the possibilities afforded by algorithms and big data continues to blossom as early adopters gain benefits from AI systems that automate decisions as varied as making customer recommendations, screening job applicants, detecting fraud, and optimizing logistical routes. No doubt optimizing logistical routes is a problem for algorithm and has been studied since, idk, mid 1920s? With the invention of linear programming? Or maybe later but much earlier than the current wave. In contrast, certain screen job applicants is done aggressively now, no doubt but there's not evidence imo that scanning does more than reduce interviewer workload. And product recommendations is one of those constantly talked-of and generally bullshitty applications that average person can verify the ineffectiveness of. Which is to say, perhaps algorithms today wind-up being used because they allow the meta-corp act a blind juggernaut, that this is the primary advantage the approach has, with the secondary advantage being justifying that operation and "greater efficiency" sometimes results but is more often an excuse.

amadeuspagel 4y ago

Where Paywalls Rule ... Socrates once had to admit that he did not understand the issue of names, because he was not able to afford the 50 drachma course on the matter. I'm now similarly embarrassed. I do not understand what it means for algorithms to rule (are laws algorithms? are procedures?) or for values to wither. I can only ask whether the fundamental problem in the robodebt scandal was not the shift in the burden of proof.

OkayPhysicist 4y ago
Laws are not algorithms, because a human legal system has wiggle room. You can negotiate, persuade, and the system is capable of dealing with unanticipated input. An algorithm is a strict series of steps, executed based on pre-determined allowable inputs. They are preferable to the ones defining them because A) they can be automated and B) they deflect blame: "It's not my fault, it's just policy" or "it's just the way the program works". They thus act as a proxy for the ruling class to operate through. "Values" are ideas that we, as a society, "value". Compassion, understanding, etc. These tend to involve nuance, which a system designed in advance and applied blindly to a given circumstance inherently lacks. Thus, in case you aren't simply feigning ignorance, the title could be expanded to: "When individuals with existing power over others design rigid systems to be applied indiscriminately to those beneath them for both convenience and to redirect the perception of responsibility, it leads to inhumane consequences."
- amadeuspagel 4y ago
  > Laws are not algorithms, because a human legal system has wiggle room. Is this "wiggle room" the fundamental difference between laws and algorithms, so fundamental that we have to fear values withering when algorithms rule, but not when laws rule? Is it the reason the ruling class can't operate through laws, only through algorithms? Does it prevent humans executing policies from saying that "it's not my fault, it's just policy"?
  - JumpCrisscross 4y ago
    > this "wiggle room" the fundamental difference between laws and algorithms, so fundamental that we have to fear values withering when algorithms rule, but not when laws rule? Yes. No rule is clairvoyant. All exceptions cannot be anticipated; exceptions have to be adapted to or stomped out. The law aims to do the former. Algorithms deliver the latter. > the ruling class can't operate through laws, only through algorithms Dictators’ decrees are closer to algorithms than law. They’re absolute in a way laws are not. To the degree they diverge, it’s in the enforcement, which is a sloppier version of the law.

armchairhacker 4y ago

This is just bureaucracy, which has existed long before AI. Ok, it's not just bureaucracy, because the mistakes that AI makes to favor efficiency are different than those made by a large chain of overworked and sometimes corrupt people. But the root issue is not AI, it's the badly-implemented policies that the AI is designed to carry out. Look at any other strained system (food stamps, DHS, IRS, not government but Google and Amazon customer support), we have real humans carrying out bad policies and it's not much better. Most people really do have morals and don't like hurting others, but those morals only go so far when you're part of an underfunded system, apathetic from burnout, and you literally don't have the resources or ability to make the morally-correct choice. All this debate on how AI lacks empathy and common sense place way too much value on humans who also lack empathy and common sense, and while an AI programmed to do something unethical won't hesitate, history and experiments show that some humans won't either. We definitely should be learning and understanding the limits of AI and its hold on ethics, and not delegate policies like laws and income to 100% automated systems. But AI and improved efficiency really can help a lot of government institutions. Especially when the AI takes over the mundane, and the people in charge can handle the more difficult cases. Because then those people can spend more of their "human" empathy and decision-making better.

javajosh 4y ago
>This is just bureaucracy... I don't think so. Software-powered decisions do share a feature with any bureaucracy: the inscrutability of it. Both the IRS and the Google datacenter are black boxes that are enormously powerful, automated, and dangerous. The ham-handed way in which they behave can sometimes be traced to the virtue of simplicity: do it the same way every time, without regard to context. Any programmer recognizes this as the simplest possible program! And if you decide to build in more "policy wiggle room", this often has a perverse effect of being immediately and ruthlessly exploited in ways that are difficult or impossible to enforce. The new thing with software decisions is the unaccountably for responsible humans. It's like the corporate veil, but much worse. This one is..real, physical. A human can say not only didn't they perform the action (the same is true using a human underling) but now they can say a) I didn't set the policy, It did, and b) I don't know where It runs, or how to change It or stop It. I imagine it will be quite the fashion for hyper wealthy c-suite to delegate to an AI, enjoy life and the continued high remuneration and decrease of accountability and liability. The other thing is that, in general, software accelerates complexity and never reduces it. We live in a truly science-fiction era where we have problems that our society can cheaply reproduce devices (chips) with 10^10 microscopic states, and we forbid each other from looking, or even knowing, what those states are. We buy and sell these tiny machines, and we don't really know what's inside them. We connect them to the internet! There is just so much space for things to hide, it's frightening. Now consider a typical bureaucracy tends to do more poorly the more complex its inner workings; now add thousands, millions of computers, with 10 generations of programmers blood soaked code (note: programmer generations are like 5 years long).

voidhorse 4y ago

In my opinion, the problem is much bigger than just a class of algorithms (AI) and is largely a side-effect of late-capitalism. The primary issue is that software systems are inherently rule-driven (algorithmic) and that encoded rules are just that--encoded, fixed--for some amount of time during a release cycle, a system will operate in a fixed way. Private companies are the ones designing and developing these systems, and as far as private companies are concerned the only value they care about is profit. Ethics hardly enters into discussion when it comes to the extraction of the maximum capital possible. As long as there's no pre-established juridical ruling against it, the capitalist is not going to eliminate any approach for ethical reasons. Worse, because the algorithm is the product, these rules are carefully guarded and as the product grows in popularity it is not only the experience, but the algorithmic mode of reasoning itself that begins to dominate--people are complacent about, e.g. horrendous privacy practices because an ethics of digital privacy was never a question to begin with and as soon as companies rolled out products built on this absence it became normalized. Algorithms in themselves do not make values wither. A society in which anything other than monetary value was already never given consideration establishes an environment in which values wither, whether or not that erosion is realized by digital or mechanical means. It all comes back to capital baby. In other words, algorithms are one means of realizing the obfuscation an a-ethical modus operandi the grand abstraction of near unbridled capital already permits.