5 ms·
This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: gover
by matheusmoreira 4y ago
This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: government bureaucracies, beverage dispensers. They own the computers and impose their own rules on us, rules like deadlines and brand checking. We don't want these rules but they impose it on us all the same.
Computers are neutral technology. It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. The computers belong to them now, they're just letting us use them. We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them: radiofrequency hardware firmware, digital rights management, encryption regulation. If left unchecked, the power of computing will cause massive damage to the entire intellectual property industry as well as many government functions such as law enforcement. Computing freedom is therefore against their agendas: they want to transform computers into tools of oppression in order to preserve the status quo or change it in their favor. We can buy iPhones but they're not really ours, they still belong to Apple, the software is under Apple's control so when they suddenly decide to oppress us with client side scanning there's nothing we can do.
- Santosh83 4y agoComputers are fine but total computerisation of human society may not be so. That's actually what the article should have explored in my opinion. Computers are neither ruthless nor kind, its controllers, humans, are. So the question then becomes, who controls them and how sophisticated they (the computers) become, and how much they infiltrate our world. A simple thought... should everything that can theoretically be controlled by computer algorithms, made so? Should computers use ML for as many as tasks as possible, even though that can lead to wrong decisions on occasion?
- matheusmoreira 4y agoInstitutions made of people have limited reach. Human resources are finite, there's a physical limit on the scale of what they can do. A computerized society is limitless. A state might want to spy on everybody but it's impossible to do so due to limited manpower. Computers remove those limits and allow states to implement global surveillance.
- MarkPNeyer 4y agoComputerized societies are still limited by their resources. We are all seeing now the effects of a society that thinks it can print money without ruining its money supply.
- inamberclad 4y agoComputers still scale much faster than humans.
- liminvorous 4y agoWhether computers count as ruthless depends on fine details of your definition of ruthless. If you mean the mental/emotional state necessary for a human to act without regard for consequences of their actions to other people, then computers are not ruthless. If you just mean acting without consideration of how their their actions will affect people then computers fit this definition of ruthless perfectly by virtue of not being able to consider the consequences of their actions on other people, because we don’t know how to program them to consider how their actions will affect other people.
- mellavora 4y ago> If you mean the mental/emotional state necessary for a human to act without regard for consequences of their actions to other people, then computers are not ruthless. I don't understand, what am I missing? I've never seen a computer have any regard for the consequences of its decisions. Computers are thus completely ruthless.
- aordano 4y agoI think the GP points to the fact that given the inability of computers to evaluate the consequences of its programming, ruthlessness is a concept that cannot apply to them because by definition involves disregard about those consequences, not ignorance. edit: typo
- Avicebron 4y agoI think there is a conflation of malicious and ruthlessness in this thread, showing no pity or compassion (def of ruthless) does apply to a computer because they can do neither. However, being actively malicious and disregarding consequences probably only applies to humans controlling them because the computer is ignorant of those consequences as you say.
- drhodes 4y ago> because we don’t know how to program them to consider how their actions will affect other people. We don't have to. For example, a ruthless train door could be made less ruthless, by having sensors that respond to someone who is nanoseconds late, to open the door, just that once, just like it's elevator door brethren, but only once. The problem is that "inherently opportunistic people" will take advantage of the machine kindness and take an entire train full of people hostage. The oppressive train door is a dictator everyone loves, maybe if they were also razor sharp, people would give them the respect they deserve. phwump. guillotrains. on time, every time.
- sokoloff 4y agoThe alternative mechanisms for decision-making also make wrong decisions on occasion, especially when that alternative is a human.
- ethbr0 4y ago> So the question then becomes, who controls them and how sophisticated they (the computers) become... There's also unexplored (by article) concept of inadvertent or emergent control. E.g. Dept A encodes Rule 1 and Dept B independently encodes Rule 2, yet applied together Rule 1 and 2 produce an unexpected outcome. In which case, neither Dept A or Dept B (the ostensible controllers) could be said to actually control the outcome.
- drbojingle 4y agoinfiltrate? They're not spies. They're savants at best.
- andai 4y agoThere is also the question of increasing human-machine integration, eventually resulting in a direct read-write interface to the brain (see Neuralink, OpenWater, and VALVE's research and goals), in the context of proprietary software under corporate and government control.
- verisimi 4y agoI agree. I think the oppressiveness will come from software. Who writes it, and what values does the software reflect? Does it reflect yours? The governance structure wants to code their rules into a machine that will execute those rules without being able to apply human discernment. Humans are meant to be out of the loop - their rules and diktats are meant to be all we need. Ai + robotics has brought this possibility onto the horizon. The issue really then, is in people's acceptance that someone can write meaningful rules for them to follow. Hence we see jobs such as AI ethicist. As if they can do anything other than justify the immoral decisions corporations/government make. The collective that still believe government has their best interests at heart and are unable to refuse following the worst of us, will drive us into technological tyranny.
- fsflover 4y agoThis is exactly why free software was invented, which respects its users: https://www.fsf.org/blogs/rms/why-free-software-is-more-important-now-than-ever-before https://www.fsf.org/blogs/rms/why-free-software-is-more-impo....
- MaxBarraclough 4y agoIt's odd for an article like this to make absolutely no mention of Free Software, but it's not really using oppressive in the sense the Free Software movement uses it. The point of the article boils down to something like this: Pre-AI computers are very literal and show no judgement or discernment in their rule-following. They are inevitably used to implement complex systems against which users have little recourse, and to implement complex systems whose behaviours and failures can be blamed on computers rather than on the people responsible. These fundamental problems will persist no matter the progress made in AI. Free Software is intended to stop the user being oppressed by their own computers. The article's point is about other people's computers being used to serve their interests over yours, in ways that may be detrimental to the user compared to a more 'manual' system. Consider the train door example from the article. Suppose that code were Free Software. This might benefit whoever owns and runs the train, but wouldn't benefit the user (of the train) at all.
- jazzyjackson 4y agotools are not neutral: they favor that which they make easier
- kijin 4y agoComputers make a lot of things easier, both oppressive and subversive, just like any other highly versatile tool: hammers, screwdrivers, knives, syringes, duct tape and WD-40.
- TheOtherHobbes 4y agoTools define a relationship with the entity being tooled, so you can learn a lot about the goals and values of the transactions by looking at the kinds of relationships that are/aren't implied. You can't feed people with an AR-15. The metaphors that define computing are all about abstraction, automation without human input, predictability, and control. Abstraction tends to lossy and context-insensitive, which is why an algorithm will say "no" to a loan request based on a credit score that ignores personal context, where a human loan manager who knows the people involved might say yes - for good and valid reasons. Computing seems to be one of the most successful products of modernism, which values speedy predictable repetition, lack of decoration, and anonymised mechanisation. It's possible to imagine computing in other value systems. But it's revealingly hard to do without losing the essence. All of which is a complicated way of agreeing - computing is not neutral, and very probably can't be. It's the first automated value evangelising system in history. And that's not a comfortable thought.
- xena 4y agoYou can feed people bullets with an AR-15, but that usually doesn't end up being a life extension measure :(
- franga2000 4y ago> Abstraction tends to lossy and context-insensitive, which is why an algorithm will say "no" to a loan request based on a credit score that ignores personal context, where a human loan manager who knows the people involved might say yes - for good and valid reasons. A human loan manager, given only a credit score, will approve the loan based on only the credit score. The problem isn't human vs machine, it's context-awareness and context-sensitivity vs a lack of one or both. All large systems, human or machine, tend to be implemented using abstractions. Layers of context-stripping abstractions were a part of buerocracy far before machines took it over. If a loan request form doesn't have an "if I don't get this loan my family will literally starve to death" box that you can check, the evaluator will not know about that context couldn't possibly act based on it. The form can be digital or paper and the evaluator can be a program or a human - it doesn't change a thing. Similarly, context-sensitivity (the willingness to let the context sway your decision) was strictly limited even back in the day when decision makers were close enough to the source to be aware of the context. A bank clerk had to stick to the rules and couldn't just decide to give a loan to someone that didn't qualify under them. If they did, they'd be risking losing their job or worse, so they didn't. This discretion was only given to the people in charge - and that hasn't changed with computers. A person in a high enough position can override what the computer says, just as easily as they can override a lower clerk's decision. Yes, making machines context-aware takes some effort, and making them context-sensitive takes even more effort. But even if machines magically came with both, their operators would do the best to get rod of them, just like they did with their employees. From their point of view, empathy is inefficiency. And that's the real problem.
- gtsop 4y agoSpot on! The matter at hand here isn't "human vs machine" but "human vs machine vs human", meaning the oppression of human by human via the machine
- nonrandomstring 4y ago> This oppressiveness is not inherent to computers Absolutely. I vehemently disagree with this essay. One of the reasons I am a passionate technology critic and sceptic is that I love technology and feel disgusted at the way it's co-opted and abused. I am always vigilant not to become jaded or lose sight of the immense positive possibilities for digital technologies. But I sincerely believe that before that can happen and we can move into the true golden era of technological humanism, Big Tech, surveillance culture and over-bearing governments must be eliminated. > it's inherent to the people and institutions making use of them. I disagree. It resides within them and is amplified by technologies. "Inherent" implies that it is an indelible, fundamental feature. Not at all. Humans, as individuals and institutions can do better. We're just seeing amplification and facilitation of the same old human weaknesses, laziness, greed, stupidity, fear - go pick up some Aristotle, or any religious text for a comprehensive list of moral failings. > Computers are neutral technology. Not quite. All technologies carry values in their design. An AK47 and a baby incubator carry different values, one is a for taking life and one is for preserving it. At a more subtle granularity computers carry values with them. I recommend Edwin Black's "IBM and the Holocaust" for some insights here. A Google Pixel carries different values than a Raspberry Pi - though to most eyes the differences are invisible. > It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. Yes, computers have enabled tyranny. That's hardly debatable now. I call it techno-fascism. Other people use different terms. It's there in everything from Facebook to "cashless commerce". > The computers belong to them now So long as computers are controlled by software, which is mutable, the more complete and devastating will be the reversal when we take them back. There is always poetry and irony in ultimate justice. One way of looking at it is the tyrants are simply building the apparatus to enable the greatest and most audacious bloodless revolution in the history of mankind. The future of freedom depends on "hackers". Sell them more rope.
- Msw242 4y agoThe more neutral/academic term (instead of techno-fascism) is "algorithmic control" Plenty of great reading: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C44&q=algorithmic+control&btnG= https://scholar.google.com/scholar?hl=en&as_sdt=0%2C44&q=alg...
- coldtea 4y ago>This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: government bureaucracies, beverage dispensers. Well, to build things at the modern computer industry scale, you need government bureaucracies, huge corporations, tons of discipline, from the assembly line to the cobalt mines, and enterprises and bureaucracies with a use for it, 99% to streamline control, control more effectively and wider, and penny pinch customers even better. Now, give surveillance, instant indexed cataloguing, 24/7 tracking, remote controllable money and id access and more, to the same governments and corporations, and that's just going to point to more control.
- DyslexicAtheist 4y ago> Computers are neutral technology. Don't make me tap the sign: Technology. Is. Not. Neutral. A shame so few technophiles have read Jacques Ellul[1]. Ellul's La Technique (and his later "Propaganda") should be mandatory reading for anyone thinking about systems. Be careful though, this thinker/philosopher has verbalized a problem that made Ted Kaczinsky conclude the only solution is for us as a species to retreat into the forests and vanish into the trees. Read Ellul at your own risk. About the rest I agree, but this is complaining about a system that works _exactly_ as intended. Read Ellul! [1] "La Technique" (The technological society) https://archive.org/details/technologicalsoc00ellu https://archive.org/details/technologicalsoc00ellu
- Frost1x 4y agoA 500+ page book is beyond most peoples tree time to explore from a comments faith alone so here's a summary I found (the quality of which is unknown) since I'm always interested to some degree in new perspectives (https://www.supersummary.com/the-technological-society/summary/ https://www.supersummary.com/the-technological-society/summa...): While Ellul separates technique from the individual machines that contribute to its reign, he maintains that machines represent an ideal state for the forces that govern technique. Humans, on the other hand, are not ideal at all. We lack the efficiency and predictability of machines. Therefore, Ellul says, technique is the social adaptation that exists in transforming the messy lives of humans so that they better fit into a world controlled by machinery. Further, in a somewhat terrifying turn of phrase, he adds that technique is "the consciousness of the mechanized world." So humans adapt to technology and as technology becomes more efficient we just become cogs of technology is the core principle here and I can see how this relationship could be thought to arise however I completely disagree with it. Technology does enable efficiency at a fundamental level but it's not that humans become slaves to the efficiency or technique, it's that some humans chose to force other humans into increasingly more efficient means. There is some feedback relation here in that in a global competitive environment as leaders around the world continue to compete in various forms they will inherently drive their populations into deeper efficiency I believe but the rise to all of this is because of the global competitive environment. It's largely social constructs and to some degree, finite resources that drive these behaviors and lead to efficiency focus. More than anything it's greed and hunger for power that drives this nonsense. Most humans recognize when their lives become dominated by some silly efficiency and don't care to follow suit. Their want autonomy, to own their life and own their choices. They're instead forced into "the technique" so to speak as a secondary effect. The underlying effect is what needs to be rutted out, technology isn't the fundamental issue here, it's a set of human behaviors and institutions that exist and enable them.
- nsrahmad 4y agoThe best thing about this topic I've read is an article by Melvin Kranzberg as "Kranzberg's laws" https://doi.org/10.2307/3105385 https://doi.org/10.2307/3105385 . There are six laws. 1) Technology is neither good nor bad; nor is it neutral. 2) Invention is the mother of necessity. 3) Technology comes in packages, big and small. 4) Although technology might be a prime element in many public issues, nontechnical factors take precedence in technology-policy decisions. 5) All history is relevant, but the history of technology is the most relevant. 6) Technology is a very human activity-and so is the history of technology.
- PontifexMinimus 4y ago> We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them [...] Computing freedom is therefore against their agendas Yes, exactly.