6 ms·

I have been thinking about the difference between human users and computational agents. There is a widespread pattern of enforcing that only human users may int

I have been thinking about the difference between human users and computational agents. There is a widespread pattern of enforcing that only human users may interact with a service, not computational agents. Running a user-installed extension is one example; additional examples are CAPTCHAs, account verification by phone number, or services which refuse to provide API access such as banks. Sometimes the reason is to protect users (e.g. banks), sometimes to protect the service from abuse. But often the reason is to ensure value can be extracted from users (e.g. serving ad impressions) or the power that can be exerted over users. Regardless, I believe this comes down to power: computer scientists understand that acting through a computational agent is fundamentally more powerful than acting directly as a human user. Computational agents can operate at scale, they provide virtualization (e.g. throwaway identities), and thereby conserve the ultimate scarce resource: our attention. Many systems seem to have computational power asymmetry as their fundamental principle: as little human contact on the system's side as possible; as much human contact on the user's side as possible. If there's any path ahead that avoids a totalitarian nightmare resulting from asymptotic concentration of power, it depends on cultivating a more widespread understanding of computational power and developing new social norms and policies. As a CS educator, this is one of my top priorities.

ddingus 5y ago
Thank you. That is good work.
wussboy 5y ago
I haven’t thought about it like this before. Thank you.
- MarcoZavala 5y ago
userbinator 5y ago
Knowledge is power... and they don't want you to have too much. Imagine if almost everyone knew that they could use things like userscripts and stylesheets to enhance their browsing experience. Or that DRM can often be cracked by changing a single byte. This is why the war on general-purpose computing started. There are some things they don't want you to know because they'd lose control.
rektide 5y ago
I have what feels like a religious belief, that denying humanity the ability to use tools to explore, to poke & prod & pry at the virtual materials in front of us is Anti-Enlightenment, flies in the face of the godhood & what nature & destiny & whatever provenance might be has built in creating mankind. No material in the universe has ever been resistant to understanding, to probing, to exploration. It's from this exploration that humankind has emerged & whatever of greatness of civilization that we have has been built. These mechanized systems that we legally are not allowed to explore or work is one of the most de-humanizing hells I can imagine. Humanity as we know it can not endure in a world where we are never allowed to understand or go further. Shame on any company or law that acts otherwise; they are monsters. Thank you for your very succinct well captured write up. Humankind seems desperately in need of a defense force, needs to be able to grasp what an existential threat forced-consumierzation is, how radical of a historical upset this juncture we're at is. I 100% agree that giving people insight & visibility & faculties to work with, understand, see, & monkey with computers is essential to keeping our core liberaties alive, in an increasingly mechanizing & automating world.
quantified 5y ago
This is why large roomfuls of people spend their days curating a set of social media profiles. It’s a jobs-creation policy for the lower rungs of cyber-scamming and mischief, while maybe discouraging it from getting out of control. Eg [https://www.theguardian.com/technology/2013/aug/02/click-farms-appearance-online-popularity https://www.theguardian.com/technology/2013/aug/02/click-far...], was looking for other references too.
matheusmoreira 5y ago
> But often the reason is to ensure value can be extracted from users (e.g. serving ad impressions) or the power that can be exerted over users. Yeah, I assume this is the most common reason. This needs to be fought on principle. It doesn't matter what their terms and conditions say. It doesn't matter how much money they lose. We should always have full control. We should be able to block their advertising. We should be able to automate our browsers. We should be able to turn our user agents into cyberweapons against them. This is legitimate self defense against their abuse. We're talking about a company that gets people addicted to bullshit feeds so they can make money by showing ads. Of course an "unfollow all" button is necessary. They won't provide one because they want us addicted to their feed. So we'll provide one for them. People spend their days on social media because they're addicted to it. If a browser extension can somehow help people break free, it's objectively a good thing for humanity. If this world was just, courts would rip Facebook a new one if they dared to waste their time with this stuff.
TeMPOraL 5y ago
> Computational agents (...) conserve the ultimate scarce resource: our attention. Many systems seem to have computational power asymmetry as their fundamental principle: as little human contact on the system's side as possible; as much human contact on the user's side as possible. That's the key thing nobody is telling people about the so-called "attention economy": it's whole point is to make things as inefficient as possible - because money is made on the friction. In order to make money off attention, companies need to capture it (thereby making you spend it on something else than what you wanted to). To make more money, they need that attention to last longer (thereby making you spend more time, and/or spend it more often, on things that make them money). Seen this way, it's clear why services hate end-user automation: their whole business is causing the very friction the users would like to automate away.
- dredmorbius 5y ago
  Another element is that time is the ultimate zero-sum, nonfungible, rivalrous resource. Time spent on FB is time not spent on another site. And the higher-friction FB can make activities (to the pain point of driving away users), the better for FB. There's a similar notion in other areas, and one of the most ridiculously malevolent examples I can think of is a brick-and-mortar sales promotion gimmick: a furniture store would run a midsummer sales event on a weekend where every family visiting the store would receive a free gallon of ice cream. Because if you've just put a gallon of ice cream in your trunk, you can't visit another furniture store, you've got to head home to put it in the freezer. And time not spent in a competitor's furniture store is time not spent assessing and purchasing their products. (That leverages the sunk cost fallacy as well, among others.)
  - bongoman37 5y ago
    Another store could presumably offer an ice box giveaway.
    - dredmorbius 5y ago
      One of several possible responses, yes. Better, offer to hold (and refrigerate) the ice cream until the customer is ready to pick it up, as that would entice them to the store twice.
fnord77 5y ago
Humans are the product that facebook sells to advertisers, data brokers etc.
Waterluvian 5y ago
And the whole gosh darn thing is one sided and hypocritical. They badly want to bot the #%^* out of us.
ByteWelder 5y ago
> or services which refuse to provide API access such as banks. Just as a sidenote/fyi: In Europe, the Payment Services Directive (PSD2) requires banks to provide API access to payment accounts. https://www.ecb.europa.eu/paym/intro/mip-online/2018/html/1803_revisedpsd.en.html https://www.ecb.europa.eu/paym/intro/mip-online/2018/html/18...

ddingus 5y ago

Thank you. That is good work.

wussboy 5y ago

I haven’t thought about it like this before. Thank you.

MarcoZavala 5y ago

userbinator 5y ago

Knowledge is power... and they don't want you to have too much. Imagine if almost everyone knew that they could use things like userscripts and stylesheets to enhance their browsing experience. Or that DRM can often be cracked by changing a single byte. This is why the war on general-purpose computing started. There are some things they don't want you to know because they'd lose control.

rektide 5y ago

I have what feels like a religious belief, that denying humanity the ability to use tools to explore, to poke & prod & pry at the virtual materials in front of us is Anti-Enlightenment, flies in the face of the godhood & what nature & destiny & whatever provenance might be has built in creating mankind. No material in the universe has ever been resistant to understanding, to probing, to exploration. It's from this exploration that humankind has emerged & whatever of greatness of civilization that we have has been built. These mechanized systems that we legally are not allowed to explore or work is one of the most de-humanizing hells I can imagine. Humanity as we know it can not endure in a world where we are never allowed to understand or go further. Shame on any company or law that acts otherwise; they are monsters. Thank you for your very succinct well captured write up. Humankind seems desperately in need of a defense force, needs to be able to grasp what an existential threat forced-consumierzation is, how radical of a historical upset this juncture we're at is. I 100% agree that giving people insight & visibility & faculties to work with, understand, see, & monkey with computers is essential to keeping our core liberaties alive, in an increasingly mechanizing & automating world.

quantified 5y ago

This is why large roomfuls of people spend their days curating a set of social media profiles. It’s a jobs-creation policy for the lower rungs of cyber-scamming and mischief, while maybe discouraging it from getting out of control. Eg [https://www.theguardian.com/technology/2013/aug/02/click-farms-appearance-online-popularity https://www.theguardian.com/technology/2013/aug/02/click-far...], was looking for other references too.

matheusmoreira 5y ago

> But often the reason is to ensure value can be extracted from users (e.g. serving ad impressions) or the power that can be exerted over users. Yeah, I assume this is the most common reason. This needs to be fought on principle. It doesn't matter what their terms and conditions say. It doesn't matter how much money they lose. We should always have full control. We should be able to block their advertising. We should be able to automate our browsers. We should be able to turn our user agents into cyberweapons against them. This is legitimate self defense against their abuse. We're talking about a company that gets people addicted to bullshit feeds so they can make money by showing ads. Of course an "unfollow all" button is necessary. They won't provide one because they want us addicted to their feed. So we'll provide one for them. People spend their days on social media because they're addicted to it. If a browser extension can somehow help people break free, it's objectively a good thing for humanity. If this world was just, courts would rip Facebook a new one if they dared to waste their time with this stuff.

TeMPOraL 5y ago

> Computational agents (...) conserve the ultimate scarce resource: our attention. Many systems seem to have computational power asymmetry as their fundamental principle: as little human contact on the system's side as possible; as much human contact on the user's side as possible. That's the key thing nobody is telling people about the so-called "attention economy": it's whole point is to make things as inefficient as possible - because money is made on the friction. In order to make money off attention, companies need to capture it (thereby making you spend it on something else than what you wanted to). To make more money, they need that attention to last longer (thereby making you spend more time, and/or spend it more often, on things that make them money). Seen this way, it's clear why services hate end-user automation: their whole business is causing the very friction the users would like to automate away.

dredmorbius 5y ago
Another element is that time is the ultimate zero-sum, nonfungible, rivalrous resource. Time spent on FB is time not spent on another site. And the higher-friction FB can make activities (to the pain point of driving away users), the better for FB. There's a similar notion in other areas, and one of the most ridiculously malevolent examples I can think of is a brick-and-mortar sales promotion gimmick: a furniture store would run a midsummer sales event on a weekend where every family visiting the store would receive a free gallon of ice cream. Because if you've just put a gallon of ice cream in your trunk, you can't visit another furniture store, you've got to head home to put it in the freezer. And time not spent in a competitor's furniture store is time not spent assessing and purchasing their products. (That leverages the sunk cost fallacy as well, among others.)
- bongoman37 5y ago
  Another store could presumably offer an ice box giveaway.
  - dredmorbius 5y ago
    One of several possible responses, yes. Better, offer to hold (and refrigerate) the ice cream until the customer is ready to pick it up, as that would entice them to the store twice.

fnord77 5y ago

Humans are the product that facebook sells to advertisers, data brokers etc.

Waterluvian 5y ago

And the whole gosh darn thing is one sided and hypocritical. They badly want to bot the #%^* out of us.

ByteWelder 5y ago

> or services which refuse to provide API access such as banks. Just as a sidenote/fyi: In Europe, the Payment Services Directive (PSD2) requires banks to provide API access to payment accounts. https://www.ecb.europa.eu/paym/intro/mip-online/2018/html/1803_revisedpsd.en.html https://www.ecb.europa.eu/paym/intro/mip-online/2018/html/18...