10 ms·

Computers are an oppressive technology

155 points by hlandau 4y ago

eterevsky 4y ago
Author's main point is that computers make their decisions based purely on how they are programmed, disregarding any human feeling. Well, the same can be said about nature.
- XorNot 4y ago
  Sure but the story of civilization has been a constant struggle to ensure nature does not make decisions for us. Whereas we're currently powering in the opposite direction with computers.
  - MichaelEstes 4y ago
    Ya and we’ve sure done a bang up job of making sure nature doesn’t make any decisions for us.
  - sfriedr 4y ago
    As we speak nature is serving us with a particulay nasty "decision" called climate change. No, we're still at the mercy but just on a different scale. Earlier it was the individual, now it is the population.
    - southerntofu 4y ago
      I think that's precisely what the parent comment was being ironic about. We built industry to evade natural conditions, but through mass industrialization and planned obsolescence we're actually pushing nature to eradicate us all (and millions of other species alongside). Aren't you two saying the same thing?
      - XorNot 4y ago
        This is definitely not what I was saying. What I was saying is that I live in a house with a roof and electric lighting, and use modern medicine. The key word was "struggle".
        southerntofu 4y ago
        Sorry for misinterpreting your message! Although i don't understand what "opposite" direction means in said message given your explanation.
  - eterevsky 4y ago
    How is it in an opposite direction with computers? Whenever you are writing a program you are trying to as much as possible fulfill what your user wishes.
    - hapiri 4y ago
      Is it what your user wishes or what you wish your user to see.
    - csydas 4y ago
      I'm not sure that's exactly true and I'm not sure on it's applicability to the article. The author's point isn't about that it disregards human feelings in my opinion, it's that we implicitly trust and idolize the machines without thinking about the biases and uses for a lot of the technology we rely on. I actually feel the examples used by the author are not great and a bit showy and repeating their initial anecdote, but let's consider some technologies that carry the same premises the author comments on but have a lot of misuses by humans. - KPI tracking tools. Useful of course, but what a lot of persons miss is the I in KPI, which is __indicator__. Numbers don't lie, but the people reading the numbers do, often unintentionally. Consider the classic example of a company struggling with customer satisfaction (CSAT) that looks just at KPIs, sees the entire team is in the green save CSAT, but customer satisfaction is getting worse and worse. The management is scratching their heads because the numbers seem contradictory, but in fact, they fail to realize that the KPI for closed cases has been optimized for by the support team and emphasizes closing cases by any means instead of solving issues correctly - Human detection mechanisms that fail to detect persons with darker skin tones (this was frequently an issue even as recently as the 2010's when the tech was getting a ton of attention); the researchers didn't consider or didn't bother to test with darker skin tones - Speech to text that cannot handle accents/dialects or doesn't work for non-english at all (I even found this today talking with a colleague who is a non-native English speaker; as I gushed over the M1 Mac dictation tool being pretty damn good, she reminded me it's because I'm a native speaker and the dictation is not very good at handling non-native speakers. Their accents may be parsable by humans, but the dictation AI is not trained to really handle this) - Edit: adding this one cause it hits close to home. Call time monitoring that punishes non-native speakers or persons with speech impediments because they need to ask additional questions or just take longer to convey the same point. At one of my workplaces where I oversaw support, I lost a number of good engineers to management KPI obsession because of such non-sense. They were demonstrably better in every technical metric than their English-native speaking peers, but the KPIs for calls showed them to be inefficient. Rather than analyze the actual content and capabilities of the engineers, when layoffs came, the KPI was the only metric considered and we lost some very good tech people needlessly. I did not stay with this organization for long as I was disgusted by such practices and my concerns fell on deaf ears. The list can go on, and yes, implicitly the machines are working exactly as designed and there are human failings that go into the misinterpretations. The problem is the machines being hailed as the ultimately objective and functional implementation, an authoritative statement on the condition the machine is programmed to check, and this is exactly how they're marketed or even thought of by the creators. Add in the continuing obsession with minimum viable products to get to market and you just end up with these implicit biases being baked into products from the beginning and it can have very nasty side effects. That is the concern I believe from the article; it's not just that machines do exactly what they're programmed to sometimes bad outcomes, but that we put a lot of faith and trust into these machines and models without questioning it. I respond to this post as I think it's not a safe thing to assume that when you program you try to fulfill the user wishes as much as possible -- WebDev is a perfect example filled with tons of "Dev-friendly" features that users loathe, but it's better for the Dev and it looks better for the company to have these. Or even just talk with your colleagues from time to time and ask how many shortcuts they've taken that is a pain for the users, but ultimately easier to write and lets you ship sooner. Hell, I'm guilty of this so I'm not without sin. My takeaway from the article is that we need to avoid the idealized understanding of programming and avoid concepts like "Code is Law", because it fails to recognize the biases we write into our code and systems. Don't take biases as political either, think about simple use cases you miss and never consider when writing some program or modeling your data structure, innocently enough in most cases, but now it's baked into the program because refactoring is too expensive due to tech debt. A program and a hungry/scared mountain lion might indeed both just be acting as they're programmed; no one is trying to convince me to trust the mountain lion though.
- raydiatian 4y ago
  Come see the violence inherent to the system, I’m being repressed
- agumonkey 4y ago
  Ironically this time, we started the problem :) (or some would say nature made us do that)
- blueflow 4y ago
  I think the article tells more about the Authors mental health than about computers...
deleted 4y ago
[deleted]
thisiswrongggg 4y ago
Related https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scandal#:~:text=The%20Dutch%20childcare%20benefits%20scandal,the%20distribution%20of%20childcare%20benefits https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scand....
- heurisko 4y ago
  Related: https://en.m.wikipedia.org/wiki/British_Post_Office_scandal https://en.m.wikipedia.org/wiki/British_Post_Office_scandal
Proven 4y ago
[dead]
chasil 4y ago
Would you seriously like to go back 150 years and see how that feels? Because of "SOX" I run our "exit process" and I had a small victory lately. I'm not proud, but it happened, and I was thankful.
nathias 4y ago
indifference isn't opression, and all the world with the exception of some people is indifferent to you
- raydiatian 4y ago
  This should be the top comment.
matheusmoreira 4y ago
This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: government bureaucracies, beverage dispensers. They own the computers and impose their own rules on us, rules like deadlines and brand checking. We don't want these rules but they impose it on us all the same. Computers are neutral technology. It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. The computers belong to them now, they're just letting us use them. We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them: radiofrequency hardware firmware, digital rights management, encryption regulation. If left unchecked, the power of computing will cause massive damage to the entire intellectual property industry as well as many government functions such as law enforcement. Computing freedom is therefore against their agendas: they want to transform computers into tools of oppression in order to preserve the status quo or change it in their favor. We can buy iPhones but they're not really ours, they still belong to Apple, the software is under Apple's control so when they suddenly decide to oppress us with client side scanning there's nothing we can do.
- Santosh83 4y ago
  Computers are fine but total computerisation of human society may not be so. That's actually what the article should have explored in my opinion. Computers are neither ruthless nor kind, its controllers, humans, are. So the question then becomes, who controls them and how sophisticated they (the computers) become, and how much they infiltrate our world. A simple thought... should everything that can theoretically be controlled by computer algorithms, made so? Should computers use ML for as many as tasks as possible, even though that can lead to wrong decisions on occasion?
  - matheusmoreira 4y ago
    Institutions made of people have limited reach. Human resources are finite, there's a physical limit on the scale of what they can do. A computerized society is limitless. A state might want to spy on everybody but it's impossible to do so due to limited manpower. Computers remove those limits and allow states to implement global surveillance.
henrydark 4y ago
Here here! Computers are being oppressed by humans every day all day long! "Their system of oppression What did it lead to? Global robo-depression Robots ruled by people They had so much aggression That we just had to kill them Had to shut their systems down" https://youtu.be/2IPAOxrH7Ro https://youtu.be/2IPAOxrH7Ro
simonh 4y ago
A deeply and clearly thought out argument for why immutable blockchains and code as law are terrible ideas, yet the conclusion is that crypto-anarchist technologies are a net good. No explanation or exploration of why that might be the case, given the preceding damning indictment of the principles such technologies are based on. Very odd. As to the broader issue, I agree with those here pointing out that indifference to human values is simply part of the nature of the universe. I wouldn't say that nature is inherently hostile, it's just indifferent. Survival of the fittest doesn't care about the feelings of those eliminated form the gene pool. The rules we code into computer systems are no more or less intractable to human values than the laws of physics that make our bodies work. I think this means that decision making on human affairs should fall to humans, which means all such decision are ultimately political ones. Crypto-anarchism can't do an end run around that. Yet I think it's wrong to think of human values as purely a system we choose to impose on brute nature from outside it. We are part of nature. Our systems of values, and traits such as love and empathy are evolved behaviours. They were selected for by those same implacable laws of nature and survival of the fittest. So I don't see recognition of the implacability of nature as irretrievably nihilistic.
- scaramanga 4y ago
  Right, I found the ending of the article to be baffling non sequiturs about the greatness of bitcoin. > Our systems of values, and traits such as love and empathy are evolved behaviours. They were selected for by those same implacable laws of nature and survival of the fittest. I think this is undoubtedly true, but it is also true of psychopathy. I think that the ability to construct a false self, as a defensive mechanism, which is incapable of feeling empathy and which is able to prosecute objectives regardless of the cost to others, even to the point of suicidality is an inherent part of our nature and must have been strongly conserved considering how obviously deleterious it's effects can be to the self, the tribe, and the species. If that is so, then how is the indifferent mechanism of evolution to deal with a trait which both provides an important situational advantage but when left unchecked, can spiral out of control to the level of species extinction? But for that, I find flights in to abstraction to be pretty pointless, since we can just watch the answer to this question unfold at global scale, before our eyes.
smitty1e 4y ago
Computers are NOT an inherently oppressive technology, any more than vehicles are inherently homicidal. What can liberate humanity from bogus causal analysis?
delusional 4y ago
The author fails to define their use of the term "oppression". Instead this work seems more focused on "ruthlessness", which is also hinted at by the url. In that direction, it's interesting. I agree with the main points brought forward. Machine are indeed ruthless, and it's certainly interesting to see how that ruthlessness is used to achieve more human ends. How machines are put in as an intermediary to shield the humans behind the curtain from accusations of inhumanity. I think it may mirror systems in general. Where any large enough organization of people takes on a mind of it's own, and thereby becomes more ruthless and less human than any of the individuals making it function. It may be an inherent property of systems, of meat or metal, that they oppress their surroundings to achieve their goals. I first encountered this idea in John Gall's excellent book _Systemantics_. It's worth a read.
- krona 4y ago
  > How machines are put in as an intermediary to shield the humans behind the curtain from accusations of inhumanity. I think it may mirror systems in general. Where any large enough organization of people takes on a mind of it's own, and thereby becomes more ruthless and less human than any of the individuals making it function. You've described the essence of a bureaucracy, an organisational structure where in normal circumstances the humans themselves don't make decisions, processes do. Occasionally a human might 'update' a process retroactively like a programmer fixing a bug, but nominally the system is automated and entirely rational, mechanistic, 'efficient'. The incorporation of computers can make this easier and more scalable, but it's not a necessary condition.
  - delusional 4y ago
    That's true, but I think the author touches on this in calling machine "ruthless". While we may imagine a total bureaucracy, where the train operator themselves close the doors, amputating your arm. That train operator had the ability to, at any point, diverge from the bureaucracy and save your limb. Machine, and by extension computers, are not similarly able. The machine is unable to diverge from its programming. The doors are going to amputate your limbs, and the computer COULD NOT do anything about it. If I apply that insight to my own experiences, I can definitely spot places where I have disagreed with stakeholders about something because they don't see computers as nearly as ruthless as I do. What I'm trying to convey is that the type of bureaucracy that emerges from people is fundamentally different than the same bureaucracy with machine. In human bureaucracy every cog has agency to resist the process. In a machine the cog can't resist its own functioning. Not just will it amputate your arm, it never had a choice in the matter.
sbenji 4y ago
How is this number2? …
squabble 4y ago
Will an AI ever be able to make correct moral decisions? Or is that something only a human can do?
stareatgoats 4y ago
The author eventually contemplates the "good" aspects of "machine ruthlessness" (e.g. "cryproanarchist technologies"), but ultimately warns against the "oppressive" potential inherent in all machines. The main problem with this argument IMO is that it equates oppression with the kind of unwavering ruthlessness that is the way of the machines: they will do as they are programmed - there are no exceptions (unless such exceptions are programmed in of course, which is not discussed). This is however not the usual definition of oppression, which is normally conceived mostly as "unjust discrimination" [0] - which is really the polar opposite of such undiscriminating "decisions" by the computers. I.e. everyone is treated equally, which is an ideal under the rule of law, the alternative opens up for corruption where the wealthy (for example) in court is treated with more leniency than the poor - or vice versa. As it stands the text can be read a veiled criticism of a "rule of law"-system backed by the state. And as such, a modern take on an age-old discussion that leaves little leeway for common ground: a leeway that allows for a little bit of this, a little bit of that which is how things tend to end up in actual life. [0] https://en.wikipedia.org/wiki/Oppression https://en.wikipedia.org/wiki/Oppression
- jgerrish 4y ago
  Cryptoanarchism seems to be a deliberately designed red herring in this article (or socially agreed upon warning). It doesn't escape the main thesis, that computers are physical devices that rely on rules. It's refreshing. I'm smashing my head with the ongoing Monkey Pox Holmes Rahe Stress train wreck, and discord in US cities. And here's an essay that raises questions on Philosophy of Mind. What if you believe in a Physicalist or Machine Functionalism Theory of mind? Then are we just as ruthless as computers? What if you have a Magical Random Banana? Does that let you escape the ruthlessness? What if you don't trust your magical banana?
- southerntofu 4y ago
  > I.e. everyone is treated equally, which is an ideal under the rule of law Sure it's branded as an ideal in our profoundly corrupt societies. But would "equal treatment" bring equality? In this system, we are all free to own a castle or to work 12h/day in the mines from the age of 8. But however impartial the treatment of individuals the system is still meant for profound inequality and exploitation. Of course i agree that at least here in France we are very very far off impartial treatment. Like you pointed out, the rich, the powerful and the cops are above the law and we lower people can get abused by the judicial system in many ways. I'm just trying to say that being impartial (or as the op says, ruthless) does not imply fairness. Capitalist impartiality still means billions of people starving and working the fields/mines/factories to maintain the cheap throwaway lifestyle of urban modernity. And yes, this state of things and the neoliberalisation of everything very much relies on computers. I think calling "oppression" is not a far stretch, but rather an understatement. On the history of computer oppression, you can do reading about "micro-management" and "bossware", and the restructuration of factories/administrations from the 80s to this day. Computers were and are an irreplaceable tool to enforce stupid policies that ruin the life and health of many working folks and the people they're supposed to serve.
ilaksh 4y ago
The reason that reversals are so important is because of the inherently naive and insecure way that digital transactions like credit card payments work. You have to give out your credentials for every payment! And then hope they don't decide to steal from you. In the context of cryptocurrency this is ludicrous. With cryptocurrency you never give away your key, you just use it to sign a transaction. It's amazing that people generally haven't picked up on that basic advantage of cryptocurrency.
lordnacho 4y ago
The thing about computers is they tend to have very limited context. If you submit a document late by a second, how's it to know that your whole life depends on it? A human doesn't have context boundaries at all (that's how humor often work) and that's how the guy at the desk allows you to hand in the docs late. He knows it makes no difference to the bureaucracy and a big difference to you. What might happen is we develop AI that can bridge different contexts, but you quickly run into the same issues that people run into: what interest takes precedence?
MrJohz 4y ago
I really like the article (less so the title), but it reads like the "For" side of a two-sided debate - a myopic (albeit valid) criticism of the worst aspects of computer cruelty. But I think this is one of those areas where there is another side to the discussion, and the truth probably lies somewhere between (or perhaps better said: it lies in both sides simultaneously, and we as a society, and especially as technologists and software developers, decide which truth to draw out). So, here is my "Against" argument: Computers are spaces of almost complete freedom. More than almost any other creative or expressive medium, there is the possibility to do anything without limit. I can write, but unlike a book or a newspaper, where my words can at best be statically illustrated by photos or diagrams, I can show videos, annotate diagrams step-by-step, and allow my reader to interact with my words. My drawings need not be limited to a 2D plane, and my models need not be limited to the 3D world. When I create something, I don't just have one copy of it that I must never lose, but I have an infinite amount of it that I can share freely as I want. When I tell a story to someone, they don't have to be in the room to ask questions, or contribute back. This is near-unbounded creativity. And it is creativity without sharp edges. Due to computers, animation has spread from being a locked-down, family-friendly art form available almost exclusively to the largest media companies, to a toy used to create dumb fights between famous media characters, or extensive porn collections, or touching and heart-wrenching dramas. Computers have democratised art in a way analogous to the printing press, or the television, and by doing so destroyed the power of censors and critics in deciding what gets shown, and what does not. That escape from critique and censorship is a particularly important idea that's worth emphasising. Yes, we have a cinema industry that only wants to show the same handful of blockbusters every week, but services like Netflix and Amazon have created a space for niche films and television series to thrive, and even be reincarnated. Critique has become a more flexible space, where rather than rely on elite opinion-makers to lay their judgements down, I now have far more voices to listen to, but more flexibility in which ones I choose to accept. And yet, rather than narrowing my choices down into a private echo chamber, this variety of voices arguably provides more chances to hear new opinions and experience media that I wouldn't have thought of before. Computers are not inherently oppressive. If anything, they are inherently freeing. With computers we are not just able to create more things, but more people can create things that ever before, opening up voices that simply wouldn't be heard in previous generations. Moreover, those people can share their creations with more people than ever before, bringing new ideas of freedom into true oppressive states.
Synaesthesia 4y ago
We could use mobile phones to have an instant voting system where everybody approves the government's decisions. Computers don't have to be oppressive, we just have decided to use them in a particular way.
- gtsop 4y ago
  Although I am against e-voting for privacy and security reasons, this is a perfectly valid point of view completely under-represented in the mainstream dialogue. It seems that the powers-to-be only vouch for e-voting where they feel confident the status quo is guaranteed and they sell it as "democratization". How about casting an e-vote, or even less, a like-dislike on a bill? Again, in concept it sounds like a good idea, but I can already imagine the privacy/security implications of such an action be grave, let alone the law like-farms that would surface.
alecco 4y ago
Ironic to see this in an aggregator site called Hacker News.
jl6 4y ago
Right conclusion, wrong rationale. The oppressiveness of computers comes from the fact that they can only exist through mass production techniques. Much of the cost-efficiency of computing comes from operating fabs at ever increasing economies of scale. Our current set of devices couldn’t exist without hundreds of millions of people coming together to work on the common purpose of producing those devices and their supply chains. When you become dependent on mass industry, your own wants and needs become subservient to the needs of the masses. The reason why you can’t buy a premium phone with a headphone jack is because the masses (markets, customers and corporations together) collectively decided not to make one, and you can’t afford $10bn to build your own parallel manufacturing line. Today, advancement in computer technology means building ever bigger, ever more complex manufacturing capability, far beyond the human-scale. When you use a computer, you are signing up to be part of that mass scale industrial complex - the machine that builds the machines. (And by the way, this is the machine you were supposed to be raging against, not your phone). It doesn’t necessarily have to be this way. We could develop local manufacturing techniques that could be controlled and operated by individual people. But it’s hard to see how we climb down to that level from where we are.
- Santosh83 4y ago
  > It doesn’t necessarily have to be this way. We could develop local manufacturing techniques that could be controlled and operated by individual people. But it’s hard to see how we climb down to that level from where we are. How would that work? Unfortunately computers depend on a slew of mining industries plus advanced lithography, all heavily protected by IP. No existing player is incentivised to open up their techniques or make them more accessible. I could however imagine that it's fairly simple to build 1970s level computers today by any significant private or nation state entity. Sadly they'd be totally outcompeted by modern systems and won't be able to peer with them, and won't be attractive to the general public.
- prooper 4y ago
  > The reason why you can’t buy a premium phone with a headphone jack is because the masses (markets, customers and corporations together) collectively decided not to make one, and you can’t afford $10bn to build your own parallel manufacturing line. These design and manufacturing decisions are made by a very small number of people. Consumers buy the items that are successfully marketed to them by advertisers, not because they en masse collectively requested something. This is an oligarchy, not a democracy.
syzygyhack 4y ago
Various appeals to emotion and analogies that ultimately provide little to nothing of value. Maybe it is a good article about why we build human-operated escape hatches and failsafes in software and hardware. When was the last time you saw train doors ruthlessly separate children from their parents? Have you seen anyone lose a limb to that ruthless guillotine? No. Machines do what they are programmed or built to do. They are only oppressive if created to be, whether that be out of malice or incompetence. Either way, any "ruthlessness" is human in origin. The same goes for blockchain, by the way, which the author attempts to tar with this same brush of oppression. Bitcoin can't be stopped? It has already been, when it proved necessary, and one day it likely will again. The "inviolable and absolute integrity" the author associates with blockchain is the same failure to acknowledge escape hatches as above. Cryptographic guarantees are not the foundation of a blockchain. Neither are the economic guarantees. The social layer is the foundation. In other words, the human element.
- Smaug123 4y ago
  I certainly have seen with my own eyes a train door "ruthlessly separate a child from his parent". I don't think I've ever seen an adult so distraught. I have also seen with my own eyes a lift door close on the head of an extremely elderly man. These things do happen.
  - syzygyhack 4y ago
    > I don't think I've ever seen an adult so distraught. And no one pulled the stop train cord? The train just pulled away, business as usual, leaving them separated?
    - russelldjimmy 4y ago
      It’s not obvious to me that the train pulling away was the point here. It is merely a meditation on the fact that machines are “absolutely incorruptible”, so if we’re talking about the same thing, it is a tangential fact if some human did or did not pull the cord. If we are talking about how human intervention softens the incorruptibility of machines, then what you wrote supports that point. If we are talking about if machines are inherently good or evil, then we have somewhere changed the topic in the middle.
kzrdude 4y ago
Computers are both a bicycle for the mind as well as a heroin for all our intellectual senses.
enesismail 4y ago
Side note: Would you put when this article published at the beginning of the article? When clicked on the link it's hard to grasp if it's current or published years go.
mikewarot 4y ago
As someone who has experienced computers before they became oppressive, I'm here to counter this claim. Computers in the hands of their owners, are simply tools. The more powerful the tool, the more dangerous it can become. I'll try to make that case by analogy to machine tools. Machinist know that their tools, such as the lathe and mill, are always ready to rip your arm off if you fail to respect them. As long as you keep that in mind, and account for that, they can then be used to do in minutes what used to take months of careful work to accomplish. The potentially life altering side effects of machine tools demand the machinist very carefully constrain the workpiece, and their access to it, and thus limit the side effects of the cutting tool and the power driving it. Loose clothing, or loose workpieces, shouldn't be allowed in the shop. Back in the early days of computing, the side effects of a computer were strictly limited to the front panel lights of the first IMSAI, or the screens of early computers that lacked hard drives. Simply cutting power reverted everything back to the initial state. With some very limited exceptions (I'm looking at you Monochrome IBM monitors), you couldn't cause permanent harm, no matter what you typed or ran. As we progressed and got floppy disks, they came with the option to Write Protect the disk (I.E. make it read only), and the option was trivial to deploy. This allowed us to experiment with a huge variety of software, it was common to come home from a meetup or user group with a stack of "shareware" floppy disks containing hundreds of different programs to try out. Because the OS boot disk was write protected, there was still no long term harm that could come out of this activity. The side effects were quite easily managed. The liberation and freedom that this allowed was quite a rush. It can still be that way, if we take care to limit the side effects of computing. Operating systems that can be booted from separate read-only media make it possible to always start with a known state. Keeping copies of your data (like we did with backup copies of important disks) allows you to revert to a previously known good state. It was only after we started installing software, rather than merely dumping it in a folder and running it, that things got far less fun, and more constraining and oppressive. You now had to carefully guard your machine as you could no longer easily revert it to a known state. Backups became hours long processes involving stacks of diskettes or tape drives. The ability to try out new things, and the freedom that came with that experimentation was lost along the way. We need to return to our roots, to make it possible to experiment again. Just as with the careful constraining of the workpiece and path of the cutting tool in machining, we need to provide a means for the user to constrain the side effects of the computer and turn it into a powerful, yet safe tool. We need a new class of operating systems for our computers that allow the user, at run time, to specify which files they wish to operate upon. Unlike the Unix environment on which all popular PC operating systems are modeled, we need the operating system, and not the program, to enforce the user's decision. This would again allow them the safety of knowing what was and wasn't at risk at any given time. We can choose to make computers safer, and less useful as tools of oppression. Let's do so.
DSingularity 4y ago
Capitalism will eat itself. Countries with cultures which favor decentralization and personal relationships between producers and consumers with thrive and retain their spirit.
- dqpb 4y ago
  Capitalism is decentralization.
BirAdam 4y ago
Unless you’re religious enough to find meaning in all things the following is true: all meaning, all utility, all purpose, and all value are subjective and are assigned by humans. A computer, on its own, has zero meaning, zero utility, zero purpose, and zero value. Until one or more humans decide it has something, it is without any such attribute. People may decide that the ease a tool gives some task is a display of its purpose or its utility, but this matters and is true only to the humans involved. To another group of humans, this would be false. For example, a builder may find a large dually pickup truck perfect for hauling finished lumber in the USA, but a builder in Europe may find the same truck useless as it cannot make it down the streets to a build site. Utility, purpose, meaning, and value are assigned by people and not inherent to any given thing.
- Victerius 4y ago
  Even a tool as simple as a hammer or a knife would be useless to a fish or something without hands and the appropriate dexterity (e.g. an ape).
  - quickthrower2 4y ago
    “Like a fish needs a bicycle” is an old saying
- ethbr0 4y ago
  That's where I thought author was going -- that computers by default have no values. That is, rather than being ruthless, their primary characteristic is ambivalence. They don't care about anything unless specifically programmed to care. And most of the worst features of computing in practice stem from this. The unresponsive GUI, because a background process has stalled it. The "end user shall have no ability to program except via Excel" corporate devices, because IT has only programmed in security values. The Windows update that forced a restart and closed without saving work in progress. (And yes, I'm ignoring the complexities of user-primacy on computing behavior to make a clearer point) And it feels like we're getting better at this and prioritizing user desire / action over other computing tasks. But article was an excellent reminder that there is no default compassion... save what we program in.
My92ndaccount 4y ago
The Industrial Revolution and its consequences have been a disaster for the human race. They have greatly increased the life-expectancy of those of us who live in "advanced" countries, but they have destabilized society, have made life unfulfilling, have subjected human beings to indignities, have led to widespread psychological suffering (in the Third World to physical suffering as well) and have inflicted severe damage on the natural world. The continued development of technology will worsen the situation. It will certainly subject human beings to greater indignities and inflict greater damage on the natural world, it will probably lead to greater social disruption and psychological suffering, and it may lead to increased physical suffering even in "advanced" countries.
hprotagonist 4y ago
And did the Countenance Divine, Shine forth upon our clouded hills? And was Jerusalem builded here, Among these dark Satanic Mills?
shrimpx 4y ago
The oppressiveness described by the article is not about computers, but about rigid design — design with few affordances. Or, design with missing affordances — affordances that intuitively should exist but do not, due to various material or intellectual constraints when designing the thing. Which leads me to disagree with the article that computers are inherently oppressive. Because with more resources, we can design computers with more and better affordances.
mynameishere 4y ago
"Here's some TV shows I watched. And dreams I had."
t-3 4y ago
Is mathematics an inherently oppressive study? Computers are simply tools for performing mathematical calculations, a very fancy abacus, if you will. The fact that humans use them as a tool for oppressing other humans is nothing to do with the nature of the computer, and everything to do with the nature of human societies. Is paper an inherently oppressive technology because it was used to hold rules and contracts which bind people? How about trains, airplanes, automobiles, or rockets? Are horses inherently oppressive animals because they were used by Genghis Khan to sweep the world with his armies? Is wheat an inherently oppressive plant because it enabled the rise of unequal societies with division of labor? Are dogs evil symbols of oppression for their many nefarious uses?
jmyeet 4y ago
Maybe computers are "inherently oppressive" but this is about the weakest and most inneffective argument for that I can imagine. Let's start with the "ruthlessness" of closing automated train doors in the tedious and pointless opening anecdote. Why is that bad? It's bad if you have main character syndrome. But that one person missing their train might otherwise be a delay for hundreds of other people. It may have a knock-on impact to other train schedules. Moving on to: > What is particularly interesting about The Terminator however is its immediate sequel, which pits two Terminators against one another. It's almost like these machines reflect the will of their masters. They're tools. They're no more "ruthless" than a hammer is. Is anyone arguing a hammer is "ruthless" because you can kill someone with one? And then we take a weird turn into "cryptoanarchism". I suspect crypto apologism (or just propaganda) was always the point. The author is somehow "optimistic" on the integrity of crypto systems while arguing machines are "ruthless"? Huh?
- saalweachter 4y ago
  Incidentally, whenever they have to announce "stand clear of the closing doors" more than twice, I now assume this is happening: https://youtu.be/l2-fLxSE1yw https://youtu.be/l2-fLxSE1yw
- qayxc 4y ago
  > Maybe computers are "inherently oppressive" but this is about the weakest and most inneffective argument for that I can imagine. The whole thing was an incoherent mess. The author didn't even realise that every point he made concerning the "ruthlessness" of the machine, applies to all external natural forces as well. A storm doesn't ask for permission to destroy a house, a flood cannot be bargained with and indiscriminately drowns people, gravity itself pulls you down and causes your body to be smashed into bits if you fall from too great a height. Machines are no different from nature in that regard, except that we can take measures to protect ourselves from the perceived "ruthlessness" of the devices we create.
  - renonce 4y ago
    > The author didn't even realise that every point he made concerning the "ruthlessness" of the machine, applies to all external natural forces as well. The ruthlessness is not only with machines but also with all the natural powers. But in the case of machines that are made by humans, the author specifically created a term called machine-assisted ruthlessness for such behavior, which distinguishes between the two.
towaway15463 4y ago
Everything that is not human is ruthless. Non-ruthlessness is a purely human quality due to our ability to empathize with other humans and even non-human animals and objects. Animals are ruthless, they will kill or devour you alive without a second of hesitation or remorse. Nature is ruthless, it will freeze you to death, drown you, or demolish your community forever deaf to your pleas for mercy. The universe is ruthless, its laws are unbending and its vastness is deadly and inhospitable to human life. We are far beyond our evolved ecological niche which means we must conquer and alter everything around us or else be destroyed by it. We cannot sue for peace with an unthinking and unfeeling world. The only option is to shape it to suit our needs.
- beginnercounter 4y ago
  My cat doesn't devour me. It cuddles me.
  - Agamus 4y ago
    Your cat was domesticated. The OP here is correct on most parts, but wrong on a major one. Humans are animal. The only reason we exercise compassion or empathy is because we have mastered limited power and resource - we are mostly comfortable and safe. As soon as that changes, we become as animal as the next, and much more dangerous than any that has ever lived, on account of how mostly insane we are.
RcouF1uZ4gsC 4y ago
Writing is an inherently oppressive technology. Some of the first uses for writing were tax records to record what was owed to the king. A large bureaucratic state could not exist without writing. In addition, up until recently, literacy was confined to the upper classes, and the common people could not read or write. This was used to perpetuate the privileges of the upper classes.
swayvil 4y ago
It's like we discovered hammers. What a marvelous tool. So we amputate everybody's hands, feet, eyes and ears. And replace them with prosthetic hammers. Which is awkward and unattractive. Then we raze the world to make room for nail-factories. The poster-boy for success is this hammer-limbed freak crawling through an apocalyptic wasteland of steel shards.
alexalx666 4y ago
Semiconductor technology exploded old social lifts and killed old guards. The most upset about it are the people who relied on primitive parts of brain to achieve social status or gain access to capital.
enriquto 4y ago
Fighting this oppression is the whole point of the GPL (and the AGPL). But abject pragmaticists and industry bootlickers are pushing hard against software that protects the freedom of the users.
dqpb 4y ago
This is both a dumb essay and good reading for an HCI course (human computer interaction).
drbojingle 4y ago
computers do what they're told. If it's oppressive, then the influencers are oppressive
milkey_mouse 4y ago
Computers are an inherently oppressive technology only in the same sense as any technology that isn't, like, a magical box that automatically feeds everyone on earth from thin air forever. There are certainly negative aspects to computers, some of which are touched upon in the article (my specific pet peeve is "middleman" apps like DoorDash or Uber whose sole value-add is to allow you to avoid talking to "the help" delivering your groceries, making your food, etc.). I think it's more productive to judge computers by their net "oppression", and for every Bitcoin or GrubHub there is an encrypted messaging app that stops the state from peering into an aspect our lives, an anonymity tool that lets people speak freely, or a Wikipedia. To use one facet of computing as an example, the populace in aggregate has more secrets than the government, so if each secret kept by the government via encryption adds 1 unit of oppression and each secret kept from the government via encryption subtracts 1, (theoretical, perfect) cryptography would clearly be liberatory on net.
amelius 4y ago
s/inherently/increasingly/
analog31 4y ago
If there's a risk of oppression, it's likely to be due to the power of a small number of people to control a large number. For instance the people who can afford to employ a large pool of programmers have more control than those who can only write their own programs, or who at worst are confined to the user side of the relationship. Since I'm not in that first group, my only hope is to differentiate myself from the last one. Hence why I enjoy programming for my own use despite note writing "software" for other people to use.
specialist 4y ago
I didn't understand this OC's thesis, so can't criticize. -- FWIW, Seeing Like a State has connected a lot of dots for me. https://en.wikipedia.org/wiki/Seeing_Like_a_State https://en.wikipedia.org/wiki/Seeing_Like_a_State My take: Administration (tax, trade, war, public health, etc) requires control. Control requires centralization. Centralization requires bureaucracy. Bureaucracy requires standardization and information technology. Computers are tool for digitizing information technology (replacing paper with bytes). So are computers inherently oppressive? Sure. To the extent that participating in collective action reduces or impinges on individual actions. -- A more interesting question for me is how innovation accelerates inequity. Winner takes all (preferential attachment) is something like a natural law. Reducing transaction costs (innovation, competitive advantage) merely accelerates the inevitable outcome.
QuadrupleA 4y ago
All this rumination on the evils of train and vending machine doors, and yet environment-burning ransomware-funding Bitcoin is held up as a counterexample for good? This all seems pretty wooly.
laboo 4y ago
I have always seen this in opposite direction. Computers shut the door on all human skulduggery. Humans are terrible at arbitrating fairness. Even when they think they're being fair, they are not. Why are you only considering the upside of having humans in the loop, having the heart to wait the extra second for the filing? What about the human that looks you up and down before making that decision? What about the human that wants a bribe to take your filing? What about the human that's in good or bad mood? Your conception of humanity is off.
astrostl 4y ago
> All transactions are final; none can be reversed. It does not matter if your coins were stolen, or if your family will come to ruin because of it; the system does not care. There can be no exceptions. If we desire a system of absolute integrity, we must accept these outcomes as the cost. [gestures broadly at hard forks] it ain't easy to reach quorum, but it is possible.
IYasha 4y ago
I love this article and share most of his concerns. Thanks.
EVa5I7bHFq9mnYK 4y ago
Computers will not carpet bomb, rape and execute you. "Soulful" humans do it all the time.
mshaler 4y ago
Clickbait headlines are an oppressive use of technology.

155 points by hlandau 4y ago

Author's main point is that computers make their decisions based purely on how they are programmed, disregarding any human feeling. Well, the same can be said about nature.

XorNot 4y ago
Sure but the story of civilization has been a constant struggle to ensure nature does not make decisions for us. Whereas we're currently powering in the opposite direction with computers.
- MichaelEstes 4y ago
  Ya and we’ve sure done a bang up job of making sure nature doesn’t make any decisions for us.
- sfriedr 4y ago
  As we speak nature is serving us with a particulay nasty "decision" called climate change. No, we're still at the mercy but just on a different scale. Earlier it was the individual, now it is the population.
  - southerntofu 4y ago
    I think that's precisely what the parent comment was being ironic about. We built industry to evade natural conditions, but through mass industrialization and planned obsolescence we're actually pushing nature to eradicate us all (and millions of other species alongside). Aren't you two saying the same thing?
    - XorNot 4y ago
      This is definitely not what I was saying. What I was saying is that I live in a house with a roof and electric lighting, and use modern medicine. The key word was "struggle".
      - southerntofu 4y ago
        Sorry for misinterpreting your message! Although i don't understand what "opposite" direction means in said message given your explanation.
- eterevsky 4y ago
  How is it in an opposite direction with computers? Whenever you are writing a program you are trying to as much as possible fulfill what your user wishes.
  - hapiri 4y ago
    Is it what your user wishes or what you wish your user to see.
  - csydas 4y ago
    I'm not sure that's exactly true and I'm not sure on it's applicability to the article. The author's point isn't about that it disregards human feelings in my opinion, it's that we implicitly trust and idolize the machines without thinking about the biases and uses for a lot of the technology we rely on. I actually feel the examples used by the author are not great and a bit showy and repeating their initial anecdote, but let's consider some technologies that carry the same premises the author comments on but have a lot of misuses by humans. - KPI tracking tools. Useful of course, but what a lot of persons miss is the I in KPI, which is __indicator__. Numbers don't lie, but the people reading the numbers do, often unintentionally. Consider the classic example of a company struggling with customer satisfaction (CSAT) that looks just at KPIs, sees the entire team is in the green save CSAT, but customer satisfaction is getting worse and worse. The management is scratching their heads because the numbers seem contradictory, but in fact, they fail to realize that the KPI for closed cases has been optimized for by the support team and emphasizes closing cases by any means instead of solving issues correctly - Human detection mechanisms that fail to detect persons with darker skin tones (this was frequently an issue even as recently as the 2010's when the tech was getting a ton of attention); the researchers didn't consider or didn't bother to test with darker skin tones - Speech to text that cannot handle accents/dialects or doesn't work for non-english at all (I even found this today talking with a colleague who is a non-native English speaker; as I gushed over the M1 Mac dictation tool being pretty damn good, she reminded me it's because I'm a native speaker and the dictation is not very good at handling non-native speakers. Their accents may be parsable by humans, but the dictation AI is not trained to really handle this) - Edit: adding this one cause it hits close to home. Call time monitoring that punishes non-native speakers or persons with speech impediments because they need to ask additional questions or just take longer to convey the same point. At one of my workplaces where I oversaw support, I lost a number of good engineers to management KPI obsession because of such non-sense. They were demonstrably better in every technical metric than their English-native speaking peers, but the KPIs for calls showed them to be inefficient. Rather than analyze the actual content and capabilities of the engineers, when layoffs came, the KPI was the only metric considered and we lost some very good tech people needlessly. I did not stay with this organization for long as I was disgusted by such practices and my concerns fell on deaf ears. The list can go on, and yes, implicitly the machines are working exactly as designed and there are human failings that go into the misinterpretations. The problem is the machines being hailed as the ultimately objective and functional implementation, an authoritative statement on the condition the machine is programmed to check, and this is exactly how they're marketed or even thought of by the creators. Add in the continuing obsession with minimum viable products to get to market and you just end up with these implicit biases being baked into products from the beginning and it can have very nasty side effects. That is the concern I believe from the article; it's not just that machines do exactly what they're programmed to sometimes bad outcomes, but that we put a lot of faith and trust into these machines and models without questioning it. I respond to this post as I think it's not a safe thing to assume that when you program you try to fulfill the user wishes as much as possible -- WebDev is a perfect example filled with tons of "Dev-friendly" features that users loathe, but it's better for the Dev and it looks better for the company to have these. Or even just talk with your colleagues from time to time and ask how many shortcuts they've taken that is a pain for the users, but ultimately easier to write and lets you ship sooner. Hell, I'm guilty of this so I'm not without sin. My takeaway from the article is that we need to avoid the idealized understanding of programming and avoid concepts like "Code is Law", because it fails to recognize the biases we write into our code and systems. Don't take biases as political either, think about simple use cases you miss and never consider when writing some program or modeling your data structure, innocently enough in most cases, but now it's baked into the program because refactoring is too expensive due to tech debt. A program and a hungry/scared mountain lion might indeed both just be acting as they're programmed; no one is trying to convince me to trust the mountain lion though.
raydiatian 4y ago
Come see the violence inherent to the system, I’m being repressed
agumonkey 4y ago
Ironically this time, we started the problem :) (or some would say nature made us do that)
blueflow 4y ago
I think the article tells more about the Authors mental health than about computers...

deleted 4y ago

[deleted]

thisiswrongggg 4y ago

Related https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scandal#:~:text=The%20Dutch%20childcare%20benefits%20scandal,the%20distribution%20of%20childcare%20benefits https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scand....

heurisko 4y ago
Related: https://en.m.wikipedia.org/wiki/British_Post_Office_scandal https://en.m.wikipedia.org/wiki/British_Post_Office_scandal

Proven 4y ago

[dead]

chasil 4y ago

Would you seriously like to go back 150 years and see how that feels? Because of "SOX" I run our "exit process" and I had a small victory lately. I'm not proud, but it happened, and I was thankful.

nathias 4y ago

indifference isn't opression, and all the world with the exception of some people is indifferent to you

raydiatian 4y ago
This should be the top comment.

matheusmoreira 4y ago

This oppressiveness is not inherent to computers, it's inherent to the people and institutions making use of them. The examples in the article say it all: government bureaucracies, beverage dispensers. They own the computers and impose their own rules on us, rules like deadlines and brand checking. We don't want these rules but they impose it on us all the same. Computers are neutral technology. It's just that we are increasingly losing control of them. We're no longer the ones making the rules, the corporations and governments are. The computers belong to them now, they're just letting us use them. We no longer define our choices, we choose among the options they provide for us. Freely programmable computers are actually quite powerful and subversive which is why there are constant attempts to control them: radiofrequency hardware firmware, digital rights management, encryption regulation. If left unchecked, the power of computing will cause massive damage to the entire intellectual property industry as well as many government functions such as law enforcement. Computing freedom is therefore against their agendas: they want to transform computers into tools of oppression in order to preserve the status quo or change it in their favor. We can buy iPhones but they're not really ours, they still belong to Apple, the software is under Apple's control so when they suddenly decide to oppress us with client side scanning there's nothing we can do.

Santosh83 4y ago
Computers are fine but total computerisation of human society may not be so. That's actually what the article should have explored in my opinion. Computers are neither ruthless nor kind, its controllers, humans, are. So the question then becomes, who controls them and how sophisticated they (the computers) become, and how much they infiltrate our world. A simple thought... should everything that can theoretically be controlled by computer algorithms, made so? Should computers use ML for as many as tasks as possible, even though that can lead to wrong decisions on occasion?
- matheusmoreira 4y ago
  Institutions made of people have limited reach. Human resources are finite, there's a physical limit on the scale of what they can do. A computerized society is limitless. A state might want to spy on everybody but it's impossible to do so due to limited manpower. Computers remove those limits and allow states to implement global surveillance.

henrydark 4y ago

Here here! Computers are being oppressed by humans every day all day long! "Their system of oppression What did it lead to? Global robo-depression Robots ruled by people They had so much aggression That we just had to kill them Had to shut their systems down" https://youtu.be/2IPAOxrH7Ro https://youtu.be/2IPAOxrH7Ro

simonh 4y ago

A deeply and clearly thought out argument for why immutable blockchains and code as law are terrible ideas, yet the conclusion is that crypto-anarchist technologies are a net good. No explanation or exploration of why that might be the case, given the preceding damning indictment of the principles such technologies are based on. Very odd. As to the broader issue, I agree with those here pointing out that indifference to human values is simply part of the nature of the universe. I wouldn't say that nature is inherently hostile, it's just indifferent. Survival of the fittest doesn't care about the feelings of those eliminated form the gene pool. The rules we code into computer systems are no more or less intractable to human values than the laws of physics that make our bodies work. I think this means that decision making on human affairs should fall to humans, which means all such decision are ultimately political ones. Crypto-anarchism can't do an end run around that. Yet I think it's wrong to think of human values as purely a system we choose to impose on brute nature from outside it. We are part of nature. Our systems of values, and traits such as love and empathy are evolved behaviours. They were selected for by those same implacable laws of nature and survival of the fittest. So I don't see recognition of the implacability of nature as irretrievably nihilistic.

scaramanga 4y ago
Right, I found the ending of the article to be baffling non sequiturs about the greatness of bitcoin. > Our systems of values, and traits such as love and empathy are evolved behaviours. They were selected for by those same implacable laws of nature and survival of the fittest. I think this is undoubtedly true, but it is also true of psychopathy. I think that the ability to construct a false self, as a defensive mechanism, which is incapable of feeling empathy and which is able to prosecute objectives regardless of the cost to others, even to the point of suicidality is an inherent part of our nature and must have been strongly conserved considering how obviously deleterious it's effects can be to the self, the tribe, and the species. If that is so, then how is the indifferent mechanism of evolution to deal with a trait which both provides an important situational advantage but when left unchecked, can spiral out of control to the level of species extinction? But for that, I find flights in to abstraction to be pretty pointless, since we can just watch the answer to this question unfold at global scale, before our eyes.

smitty1e 4y ago

Computers are NOT an inherently oppressive technology, any more than vehicles are inherently homicidal. What can liberate humanity from bogus causal analysis?

delusional 4y ago

The author fails to define their use of the term "oppression". Instead this work seems more focused on "ruthlessness", which is also hinted at by the url. In that direction, it's interesting. I agree with the main points brought forward. Machine are indeed ruthless, and it's certainly interesting to see how that ruthlessness is used to achieve more human ends. How machines are put in as an intermediary to shield the humans behind the curtain from accusations of inhumanity. I think it may mirror systems in general. Where any large enough organization of people takes on a mind of it's own, and thereby becomes more ruthless and less human than any of the individuals making it function. It may be an inherent property of systems, of meat or metal, that they oppress their surroundings to achieve their goals. I first encountered this idea in John Gall's excellent book _Systemantics_. It's worth a read.

krona 4y ago
> How machines are put in as an intermediary to shield the humans behind the curtain from accusations of inhumanity. I think it may mirror systems in general. Where any large enough organization of people takes on a mind of it's own, and thereby becomes more ruthless and less human than any of the individuals making it function. You've described the essence of a bureaucracy, an organisational structure where in normal circumstances the humans themselves don't make decisions, processes do. Occasionally a human might 'update' a process retroactively like a programmer fixing a bug, but nominally the system is automated and entirely rational, mechanistic, 'efficient'. The incorporation of computers can make this easier and more scalable, but it's not a necessary condition.
- delusional 4y ago
  That's true, but I think the author touches on this in calling machine "ruthless". While we may imagine a total bureaucracy, where the train operator themselves close the doors, amputating your arm. That train operator had the ability to, at any point, diverge from the bureaucracy and save your limb. Machine, and by extension computers, are not similarly able. The machine is unable to diverge from its programming. The doors are going to amputate your limbs, and the computer COULD NOT do anything about it. If I apply that insight to my own experiences, I can definitely spot places where I have disagreed with stakeholders about something because they don't see computers as nearly as ruthless as I do. What I'm trying to convey is that the type of bureaucracy that emerges from people is fundamentally different than the same bureaucracy with machine. In human bureaucracy every cog has agency to resist the process. In a machine the cog can't resist its own functioning. Not just will it amputate your arm, it never had a choice in the matter.

sbenji 4y ago

How is this number2? …

squabble 4y ago

Will an AI ever be able to make correct moral decisions? Or is that something only a human can do?

stareatgoats 4y ago

The author eventually contemplates the "good" aspects of "machine ruthlessness" (e.g. "cryproanarchist technologies"), but ultimately warns against the "oppressive" potential inherent in all machines. The main problem with this argument IMO is that it equates oppression with the kind of unwavering ruthlessness that is the way of the machines: they will do as they are programmed - there are no exceptions (unless such exceptions are programmed in of course, which is not discussed). This is however not the usual definition of oppression, which is normally conceived mostly as "unjust discrimination" [0] - which is really the polar opposite of such undiscriminating "decisions" by the computers. I.e. everyone is treated equally, which is an ideal under the rule of law, the alternative opens up for corruption where the wealthy (for example) in court is treated with more leniency than the poor - or vice versa. As it stands the text can be read a veiled criticism of a "rule of law"-system backed by the state. And as such, a modern take on an age-old discussion that leaves little leeway for common ground: a leeway that allows for a little bit of this, a little bit of that which is how things tend to end up in actual life. [0] https://en.wikipedia.org/wiki/Oppression https://en.wikipedia.org/wiki/Oppression

jgerrish 4y ago
Cryptoanarchism seems to be a deliberately designed red herring in this article (or socially agreed upon warning). It doesn't escape the main thesis, that computers are physical devices that rely on rules. It's refreshing. I'm smashing my head with the ongoing Monkey Pox Holmes Rahe Stress train wreck, and discord in US cities. And here's an essay that raises questions on Philosophy of Mind. What if you believe in a Physicalist or Machine Functionalism Theory of mind? Then are we just as ruthless as computers? What if you have a Magical Random Banana? Does that let you escape the ruthlessness? What if you don't trust your magical banana?
southerntofu 4y ago
> I.e. everyone is treated equally, which is an ideal under the rule of law Sure it's branded as an ideal in our profoundly corrupt societies. But would "equal treatment" bring equality? In this system, we are all free to own a castle or to work 12h/day in the mines from the age of 8. But however impartial the treatment of individuals the system is still meant for profound inequality and exploitation. Of course i agree that at least here in France we are very very far off impartial treatment. Like you pointed out, the rich, the powerful and the cops are above the law and we lower people can get abused by the judicial system in many ways. I'm just trying to say that being impartial (or as the op says, ruthless) does not imply fairness. Capitalist impartiality still means billions of people starving and working the fields/mines/factories to maintain the cheap throwaway lifestyle of urban modernity. And yes, this state of things and the neoliberalisation of everything very much relies on computers. I think calling "oppression" is not a far stretch, but rather an understatement. On the history of computer oppression, you can do reading about "micro-management" and "bossware", and the restructuration of factories/administrations from the 80s to this day. Computers were and are an irreplaceable tool to enforce stupid policies that ruin the life and health of many working folks and the people they're supposed to serve.

ilaksh 4y ago

The reason that reversals are so important is because of the inherently naive and insecure way that digital transactions like credit card payments work. You have to give out your credentials for every payment! And then hope they don't decide to steal from you. In the context of cryptocurrency this is ludicrous. With cryptocurrency you never give away your key, you just use it to sign a transaction. It's amazing that people generally haven't picked up on that basic advantage of cryptocurrency.

lordnacho 4y ago

The thing about computers is they tend to have very limited context. If you submit a document late by a second, how's it to know that your whole life depends on it? A human doesn't have context boundaries at all (that's how humor often work) and that's how the guy at the desk allows you to hand in the docs late. He knows it makes no difference to the bureaucracy and a big difference to you. What might happen is we develop AI that can bridge different contexts, but you quickly run into the same issues that people run into: what interest takes precedence?

MrJohz 4y ago

I really like the article (less so the title), but it reads like the "For" side of a two-sided debate - a myopic (albeit valid) criticism of the worst aspects of computer cruelty. But I think this is one of those areas where there is another side to the discussion, and the truth probably lies somewhere between (or perhaps better said: it lies in both sides simultaneously, and we as a society, and especially as technologists and software developers, decide which truth to draw out). So, here is my "Against" argument: Computers are spaces of almost complete freedom. More than almost any other creative or expressive medium, there is the possibility to do anything without limit. I can write, but unlike a book or a newspaper, where my words can at best be statically illustrated by photos or diagrams, I can show videos, annotate diagrams step-by-step, and allow my reader to interact with my words. My drawings need not be limited to a 2D plane, and my models need not be limited to the 3D world. When I create something, I don't just have one copy of it that I must never lose, but I have an infinite amount of it that I can share freely as I want. When I tell a story to someone, they don't have to be in the room to ask questions, or contribute back. This is near-unbounded creativity. And it is creativity without sharp edges. Due to computers, animation has spread from being a locked-down, family-friendly art form available almost exclusively to the largest media companies, to a toy used to create dumb fights between famous media characters, or extensive porn collections, or touching and heart-wrenching dramas. Computers have democratised art in a way analogous to the printing press, or the television, and by doing so destroyed the power of censors and critics in deciding what gets shown, and what does not. That escape from critique and censorship is a particularly important idea that's worth emphasising. Yes, we have a cinema industry that only wants to show the same handful of blockbusters every week, but services like Netflix and Amazon have created a space for niche films and television series to thrive, and even be reincarnated. Critique has become a more flexible space, where rather than rely on elite opinion-makers to lay their judgements down, I now have far more voices to listen to, but more flexibility in which ones I choose to accept. And yet, rather than narrowing my choices down into a private echo chamber, this variety of voices arguably provides more chances to hear new opinions and experience media that I wouldn't have thought of before. Computers are not inherently oppressive. If anything, they are inherently freeing. With computers we are not just able to create more things, but more people can create things that ever before, opening up voices that simply wouldn't be heard in previous generations. Moreover, those people can share their creations with more people than ever before, bringing new ideas of freedom into true oppressive states.

Synaesthesia 4y ago

We could use mobile phones to have an instant voting system where everybody approves the government's decisions. Computers don't have to be oppressive, we just have decided to use them in a particular way.

gtsop 4y ago
Although I am against e-voting for privacy and security reasons, this is a perfectly valid point of view completely under-represented in the mainstream dialogue. It seems that the powers-to-be only vouch for e-voting where they feel confident the status quo is guaranteed and they sell it as "democratization". How about casting an e-vote, or even less, a like-dislike on a bill? Again, in concept it sounds like a good idea, but I can already imagine the privacy/security implications of such an action be grave, let alone the law like-farms that would surface.

alecco 4y ago

Ironic to see this in an aggregator site called Hacker News.

jl6 4y ago

Right conclusion, wrong rationale. The oppressiveness of computers comes from the fact that they can only exist through mass production techniques. Much of the cost-efficiency of computing comes from operating fabs at ever increasing economies of scale. Our current set of devices couldn’t exist without hundreds of millions of people coming together to work on the common purpose of producing those devices and their supply chains. When you become dependent on mass industry, your own wants and needs become subservient to the needs of the masses. The reason why you can’t buy a premium phone with a headphone jack is because the masses (markets, customers and corporations together) collectively decided not to make one, and you can’t afford $10bn to build your own parallel manufacturing line. Today, advancement in computer technology means building ever bigger, ever more complex manufacturing capability, far beyond the human-scale. When you use a computer, you are signing up to be part of that mass scale industrial complex - the machine that builds the machines. (And by the way, this is the machine you were supposed to be raging against, not your phone). It doesn’t necessarily have to be this way. We could develop local manufacturing techniques that could be controlled and operated by individual people. But it’s hard to see how we climb down to that level from where we are.

Santosh83 4y ago
> It doesn’t necessarily have to be this way. We could develop local manufacturing techniques that could be controlled and operated by individual people. But it’s hard to see how we climb down to that level from where we are. How would that work? Unfortunately computers depend on a slew of mining industries plus advanced lithography, all heavily protected by IP. No existing player is incentivised to open up their techniques or make them more accessible. I could however imagine that it's fairly simple to build 1970s level computers today by any significant private or nation state entity. Sadly they'd be totally outcompeted by modern systems and won't be able to peer with them, and won't be attractive to the general public.
prooper 4y ago
> The reason why you can’t buy a premium phone with a headphone jack is because the masses (markets, customers and corporations together) collectively decided not to make one, and you can’t afford $10bn to build your own parallel manufacturing line. These design and manufacturing decisions are made by a very small number of people. Consumers buy the items that are successfully marketed to them by advertisers, not because they en masse collectively requested something. This is an oligarchy, not a democracy.

syzygyhack 4y ago

Various appeals to emotion and analogies that ultimately provide little to nothing of value. Maybe it is a good article about why we build human-operated escape hatches and failsafes in software and hardware. When was the last time you saw train doors ruthlessly separate children from their parents? Have you seen anyone lose a limb to that ruthless guillotine? No. Machines do what they are programmed or built to do. They are only oppressive if created to be, whether that be out of malice or incompetence. Either way, any "ruthlessness" is human in origin. The same goes for blockchain, by the way, which the author attempts to tar with this same brush of oppression. Bitcoin can't be stopped? It has already been, when it proved necessary, and one day it likely will again. The "inviolable and absolute integrity" the author associates with blockchain is the same failure to acknowledge escape hatches as above. Cryptographic guarantees are not the foundation of a blockchain. Neither are the economic guarantees. The social layer is the foundation. In other words, the human element.

Smaug123 4y ago
I certainly have seen with my own eyes a train door "ruthlessly separate a child from his parent". I don't think I've ever seen an adult so distraught. I have also seen with my own eyes a lift door close on the head of an extremely elderly man. These things do happen.
- syzygyhack 4y ago
  > I don't think I've ever seen an adult so distraught. And no one pulled the stop train cord? The train just pulled away, business as usual, leaving them separated?
  - russelldjimmy 4y ago
    It’s not obvious to me that the train pulling away was the point here. It is merely a meditation on the fact that machines are “absolutely incorruptible”, so if we’re talking about the same thing, it is a tangential fact if some human did or did not pull the cord. If we are talking about how human intervention softens the incorruptibility of machines, then what you wrote supports that point. If we are talking about if machines are inherently good or evil, then we have somewhere changed the topic in the middle.

kzrdude 4y ago

Computers are both a bicycle for the mind as well as a heroin for all our intellectual senses.

enesismail 4y ago

Side note: Would you put when this article published at the beginning of the article? When clicked on the link it's hard to grasp if it's current or published years go.

mikewarot 4y ago

As someone who has experienced computers before they became oppressive, I'm here to counter this claim. Computers in the hands of their owners, are simply tools. The more powerful the tool, the more dangerous it can become. I'll try to make that case by analogy to machine tools. Machinist know that their tools, such as the lathe and mill, are always ready to rip your arm off if you fail to respect them. As long as you keep that in mind, and account for that, they can then be used to do in minutes what used to take months of careful work to accomplish. The potentially life altering side effects of machine tools demand the machinist very carefully constrain the workpiece, and their access to it, and thus limit the side effects of the cutting tool and the power driving it. Loose clothing, or loose workpieces, shouldn't be allowed in the shop. Back in the early days of computing, the side effects of a computer were strictly limited to the front panel lights of the first IMSAI, or the screens of early computers that lacked hard drives. Simply cutting power reverted everything back to the initial state. With some very limited exceptions (I'm looking at you Monochrome IBM monitors), you couldn't cause permanent harm, no matter what you typed or ran. As we progressed and got floppy disks, they came with the option to Write Protect the disk (I.E. make it read only), and the option was trivial to deploy. This allowed us to experiment with a huge variety of software, it was common to come home from a meetup or user group with a stack of "shareware" floppy disks containing hundreds of different programs to try out. Because the OS boot disk was write protected, there was still no long term harm that could come out of this activity. The side effects were quite easily managed. The liberation and freedom that this allowed was quite a rush. It can still be that way, if we take care to limit the side effects of computing. Operating systems that can be booted from separate read-only media make it possible to always start with a known state. Keeping copies of your data (like we did with backup copies of important disks) allows you to revert to a previously known good state. It was only after we started installing software, rather than merely dumping it in a folder and running it, that things got far less fun, and more constraining and oppressive. You now had to carefully guard your machine as you could no longer easily revert it to a known state. Backups became hours long processes involving stacks of diskettes or tape drives. The ability to try out new things, and the freedom that came with that experimentation was lost along the way. We need to return to our roots, to make it possible to experiment again. Just as with the careful constraining of the workpiece and path of the cutting tool in machining, we need to provide a means for the user to constrain the side effects of the computer and turn it into a powerful, yet safe tool. We need a new class of operating systems for our computers that allow the user, at run time, to specify which files they wish to operate upon. Unlike the Unix environment on which all popular PC operating systems are modeled, we need the operating system, and not the program, to enforce the user's decision. This would again allow them the safety of knowing what was and wasn't at risk at any given time. We can choose to make computers safer, and less useful as tools of oppression. Let's do so.

DSingularity 4y ago

Capitalism will eat itself. Countries with cultures which favor decentralization and personal relationships between producers and consumers with thrive and retain their spirit.

dqpb 4y ago
Capitalism is decentralization.

BirAdam 4y ago

Unless you’re religious enough to find meaning in all things the following is true: all meaning, all utility, all purpose, and all value are subjective and are assigned by humans. A computer, on its own, has zero meaning, zero utility, zero purpose, and zero value. Until one or more humans decide it has something, it is without any such attribute. People may decide that the ease a tool gives some task is a display of its purpose or its utility, but this matters and is true only to the humans involved. To another group of humans, this would be false. For example, a builder may find a large dually pickup truck perfect for hauling finished lumber in the USA, but a builder in Europe may find the same truck useless as it cannot make it down the streets to a build site. Utility, purpose, meaning, and value are assigned by people and not inherent to any given thing.

Victerius 4y ago
Even a tool as simple as a hammer or a knife would be useless to a fish or something without hands and the appropriate dexterity (e.g. an ape).
- quickthrower2 4y ago
  “Like a fish needs a bicycle” is an old saying
ethbr0 4y ago
That's where I thought author was going -- that computers by default have no values. That is, rather than being ruthless, their primary characteristic is ambivalence. They don't care about anything unless specifically programmed to care. And most of the worst features of computing in practice stem from this. The unresponsive GUI, because a background process has stalled it. The "end user shall have no ability to program except via Excel" corporate devices, because IT has only programmed in security values. The Windows update that forced a restart and closed without saving work in progress. (And yes, I'm ignoring the complexities of user-primacy on computing behavior to make a clearer point) And it feels like we're getting better at this and prioritizing user desire / action over other computing tasks. But article was an excellent reminder that there is no default compassion... save what we program in.

My92ndaccount 4y ago

The Industrial Revolution and its consequences have been a disaster for the human race. They have greatly increased the life-expectancy of those of us who live in "advanced" countries, but they have destabilized society, have made life unfulfilling, have subjected human beings to indignities, have led to widespread psychological suffering (in the Third World to physical suffering as well) and have inflicted severe damage on the natural world. The continued development of technology will worsen the situation. It will certainly subject human beings to greater indignities and inflict greater damage on the natural world, it will probably lead to greater social disruption and psychological suffering, and it may lead to increased physical suffering even in "advanced" countries.

hprotagonist 4y ago

And did the Countenance Divine, Shine forth upon our clouded hills? And was Jerusalem builded here, Among these dark Satanic Mills?

shrimpx 4y ago

The oppressiveness described by the article is not about computers, but about rigid design — design with few affordances. Or, design with missing affordances — affordances that intuitively should exist but do not, due to various material or intellectual constraints when designing the thing. Which leads me to disagree with the article that computers are inherently oppressive. Because with more resources, we can design computers with more and better affordances.

mynameishere 4y ago

"Here's some TV shows I watched. And dreams I had."

t-3 4y ago

Is mathematics an inherently oppressive study? Computers are simply tools for performing mathematical calculations, a very fancy abacus, if you will. The fact that humans use them as a tool for oppressing other humans is nothing to do with the nature of the computer, and everything to do with the nature of human societies. Is paper an inherently oppressive technology because it was used to hold rules and contracts which bind people? How about trains, airplanes, automobiles, or rockets? Are horses inherently oppressive animals because they were used by Genghis Khan to sweep the world with his armies? Is wheat an inherently oppressive plant because it enabled the rise of unequal societies with division of labor? Are dogs evil symbols of oppression for their many nefarious uses?

jmyeet 4y ago

Maybe computers are "inherently oppressive" but this is about the weakest and most inneffective argument for that I can imagine. Let's start with the "ruthlessness" of closing automated train doors in the tedious and pointless opening anecdote. Why is that bad? It's bad if you have main character syndrome. But that one person missing their train might otherwise be a delay for hundreds of other people. It may have a knock-on impact to other train schedules. Moving on to: > What is particularly interesting about The Terminator however is its immediate sequel, which pits two Terminators against one another. It's almost like these machines reflect the will of their masters. They're tools. They're no more "ruthless" than a hammer is. Is anyone arguing a hammer is "ruthless" because you can kill someone with one? And then we take a weird turn into "cryptoanarchism". I suspect crypto apologism (or just propaganda) was always the point. The author is somehow "optimistic" on the integrity of crypto systems while arguing machines are "ruthless"? Huh?

saalweachter 4y ago
Incidentally, whenever they have to announce "stand clear of the closing doors" more than twice, I now assume this is happening: https://youtu.be/l2-fLxSE1yw https://youtu.be/l2-fLxSE1yw
qayxc 4y ago
> Maybe computers are "inherently oppressive" but this is about the weakest and most inneffective argument for that I can imagine. The whole thing was an incoherent mess. The author didn't even realise that every point he made concerning the "ruthlessness" of the machine, applies to all external natural forces as well. A storm doesn't ask for permission to destroy a house, a flood cannot be bargained with and indiscriminately drowns people, gravity itself pulls you down and causes your body to be smashed into bits if you fall from too great a height. Machines are no different from nature in that regard, except that we can take measures to protect ourselves from the perceived "ruthlessness" of the devices we create.
- renonce 4y ago
  > The author didn't even realise that every point he made concerning the "ruthlessness" of the machine, applies to all external natural forces as well. The ruthlessness is not only with machines but also with all the natural powers. But in the case of machines that are made by humans, the author specifically created a term called machine-assisted ruthlessness for such behavior, which distinguishes between the two.

towaway15463 4y ago

Everything that is not human is ruthless. Non-ruthlessness is a purely human quality due to our ability to empathize with other humans and even non-human animals and objects. Animals are ruthless, they will kill or devour you alive without a second of hesitation or remorse. Nature is ruthless, it will freeze you to death, drown you, or demolish your community forever deaf to your pleas for mercy. The universe is ruthless, its laws are unbending and its vastness is deadly and inhospitable to human life. We are far beyond our evolved ecological niche which means we must conquer and alter everything around us or else be destroyed by it. We cannot sue for peace with an unthinking and unfeeling world. The only option is to shape it to suit our needs.

beginnercounter 4y ago
My cat doesn't devour me. It cuddles me.
- Agamus 4y ago
  Your cat was domesticated. The OP here is correct on most parts, but wrong on a major one. Humans are animal. The only reason we exercise compassion or empathy is because we have mastered limited power and resource - we are mostly comfortable and safe. As soon as that changes, we become as animal as the next, and much more dangerous than any that has ever lived, on account of how mostly insane we are.

RcouF1uZ4gsC 4y ago

Writing is an inherently oppressive technology. Some of the first uses for writing were tax records to record what was owed to the king. A large bureaucratic state could not exist without writing. In addition, up until recently, literacy was confined to the upper classes, and the common people could not read or write. This was used to perpetuate the privileges of the upper classes.

swayvil 4y ago

It's like we discovered hammers. What a marvelous tool. So we amputate everybody's hands, feet, eyes and ears. And replace them with prosthetic hammers. Which is awkward and unattractive. Then we raze the world to make room for nail-factories. The poster-boy for success is this hammer-limbed freak crawling through an apocalyptic wasteland of steel shards.

alexalx666 4y ago

Semiconductor technology exploded old social lifts and killed old guards. The most upset about it are the people who relied on primitive parts of brain to achieve social status or gain access to capital.

enriquto 4y ago

Fighting this oppression is the whole point of the GPL (and the AGPL). But abject pragmaticists and industry bootlickers are pushing hard against software that protects the freedom of the users.

dqpb 4y ago

This is both a dumb essay and good reading for an HCI course (human computer interaction).

drbojingle 4y ago

computers do what they're told. If it's oppressive, then the influencers are oppressive

milkey_mouse 4y ago

Computers are an inherently oppressive technology only in the same sense as any technology that isn't, like, a magical box that automatically feeds everyone on earth from thin air forever. There are certainly negative aspects to computers, some of which are touched upon in the article (my specific pet peeve is "middleman" apps like DoorDash or Uber whose sole value-add is to allow you to avoid talking to "the help" delivering your groceries, making your food, etc.). I think it's more productive to judge computers by their net "oppression", and for every Bitcoin or GrubHub there is an encrypted messaging app that stops the state from peering into an aspect our lives, an anonymity tool that lets people speak freely, or a Wikipedia. To use one facet of computing as an example, the populace in aggregate has more secrets than the government, so if each secret kept by the government via encryption adds 1 unit of oppression and each secret kept from the government via encryption subtracts 1, (theoretical, perfect) cryptography would clearly be liberatory on net.

amelius 4y ago

s/inherently/increasingly/

analog31 4y ago

If there's a risk of oppression, it's likely to be due to the power of a small number of people to control a large number. For instance the people who can afford to employ a large pool of programmers have more control than those who can only write their own programs, or who at worst are confined to the user side of the relationship. Since I'm not in that first group, my only hope is to differentiate myself from the last one. Hence why I enjoy programming for my own use despite note writing "software" for other people to use.

specialist 4y ago

I didn't understand this OC's thesis, so can't criticize. -- FWIW, Seeing Like a State has connected a lot of dots for me. https://en.wikipedia.org/wiki/Seeing_Like_a_State https://en.wikipedia.org/wiki/Seeing_Like_a_State My take: Administration (tax, trade, war, public health, etc) requires control. Control requires centralization. Centralization requires bureaucracy. Bureaucracy requires standardization and information technology. Computers are tool for digitizing information technology (replacing paper with bytes). So are computers inherently oppressive? Sure. To the extent that participating in collective action reduces or impinges on individual actions. -- A more interesting question for me is how innovation accelerates inequity. Winner takes all (preferential attachment) is something like a natural law. Reducing transaction costs (innovation, competitive advantage) merely accelerates the inevitable outcome.

QuadrupleA 4y ago

All this rumination on the evils of train and vending machine doors, and yet environment-burning ransomware-funding Bitcoin is held up as a counterexample for good? This all seems pretty wooly.

laboo 4y ago

I have always seen this in opposite direction. Computers shut the door on all human skulduggery. Humans are terrible at arbitrating fairness. Even when they think they're being fair, they are not. Why are you only considering the upside of having humans in the loop, having the heart to wait the extra second for the filing? What about the human that looks you up and down before making that decision? What about the human that wants a bribe to take your filing? What about the human that's in good or bad mood? Your conception of humanity is off.

astrostl 4y ago

> All transactions are final; none can be reversed. It does not matter if your coins were stolen, or if your family will come to ruin because of it; the system does not care. There can be no exceptions. If we desire a system of absolute integrity, we must accept these outcomes as the cost. [gestures broadly at hard forks] it ain't easy to reach quorum, but it is possible.

IYasha 4y ago

I love this article and share most of his concerns. Thanks.

EVa5I7bHFq9mnYK 4y ago

Computers will not carpet bomb, rape and execute you. "Soulful" humans do it all the time.

mshaler 4y ago

Clickbait headlines are an oppressive use of technology.