8 ms·

Amazon Has to Disclose How Its Algorithms Judge Workers Per a New California Law

156 points by mardiyah 5y ago

raxxorrax 5y ago
This is an important issue. If we talk about regulating tech, this is the most pressing one. I distance myself from all colleagues that help to implement such productivity surveillance. Do something sensible with your life, the money isn't worth it.
- thinkharderdev 5y ago
  Wouldn't this law make productivity surveillance more pervasive? Based on the description in the article it seems to require that "Going forward, warehouse workers who think their quotas lead to unsafe behavior will be able to ask for 90 days' worth of documentation of how their work meets or fails the quota"?
  - raxxorrax 5y ago
    That can be the result of bad regulation, yes, but I think companies were too ambitious in implementing said tools that regulation became a necessity.
  - pjc50 5y ago
    No, that can only have been existing documentation. After all, it's Amazon's choice to have quotas, and if they're enforcing them, they must already have been documented, right?
- deleted 5y ago
  [deleted]
popotamonga 5y ago
It will be fun, workers will start do maximize for the algorithm
- emptyparadise 5y ago
  Well employers have been doing that for god knows how long, why can't the employees?
  - raxxorrax 5y ago
    Such a system will never be to the benefit of the worker. Those that say otherwise want to sell shitty software or are a bit deluded about the consequences of manufacturing.
    - emptyparadise 5y ago
      But when workers come together, interesting things happen.
- barrenko 5y ago
  The algorithm is displacing the Firm :). Wondering how long before something like Amazon is basically just a DAO?
  - idiotsecant 5y ago
    The means of production provides way too much power for humans to ever let it outside their control.
  - Arrath 5y ago
    Reminds me of a (not great) book I read about a secretive trading firm that build an AI to maximize profits except it gained enough awareness to take things into its own hands, by hiring mercenaries and assassins to drive the market and ever increase its portfolio. After, of course, killing everyone at the firm.
heymax054 5y ago
I wonder if this will impact productivity in an unexpected way. When you disclose the numbers by which you "track" people, people will find ways to maximize those numbers without necessarily maximizing output. But then, is there a good alternative? Big tech asked for this by being extremely non-transparent and mistreating people. While I agree with the intent, I don't think simple solutions like these are necessarily the best ones to solve the problem.
- ganzuul 5y ago
  They had the opportunity to solve their own problem. They didn't. Perhaps in the future they will perform better.
  - pbaka 5y ago
    --- Beginning of an imagination exercise --- As I age, nearing 50, my bladder is getting weaker. I cannot outperform the younger folks who can hold it for four hours. Should I, as an obviously unperforming resource, save up for a electro-mechanical bladder implant to replace my biological one ? Does the company offer me a savings plan for the same ? Or is it paid from the fused unemployment/health improvement fund of the state where my fulfillment center is located ? How will I manage maintenance payments in 30 years, after the rest of my body is too old for this warehouse job ? Then, of course, I have that auto-immune asthma since the lockdowns of 2022. I remember it like yesterday. I wasn't allowed in school (that was before virtuaducation). I didn't build up an immunity to basic flu, and was wrecked by a pneumonia when I started school. Never really recovered... Then it turned out the 6 vaccine shots had strange interactions with my immune system five years after the troubles ended. My body never quite worked as it should. I always had strange "after-diseases"... For the last three years, there was some stuff my body didn't like in the yearly combine-shots, ended up in the Hospitrailer, again, in coma for five days. The Autodoc said next time might be fatal. My work.pad shows me alerts from the warehouse AI that I might not be allowed on the floor next year if I don't solve the problem - both because of regulation and work time lost. The problem is, antihistamites (yeah, funny branding for those histamine dispensing nanobots) cant solve work with my immune problem. My "Pearly Melinda Gates" insurance's doctorobot said I don't have premiums accrued to pay for the cloned organ assays needed to see what part of my immune system doesn't like it. I even have less money for the immune reprogramming therapy. It's three years wages... It suggested me sell or collateralize the home I inherited. Wheesh, 12% interest. That's steep... Maybe I should just give up, and go on that famed UBI ? Those container-box habunits don't look too bad, and the virtual immersion gear is included in the monthly stipend. Just don't get caught inside when the chlore/fluoride autowash kicks in, hahaha. Then again, I'm not sure. After it proved financially impossible to sustain in the 2040's, it's now basically just food and water ration QR stamps. It's like in the time of my grandparents in the 1980's, except the monthly syntheclothing you receive are soooo butt ugly, not even classy with those huge shoulderpads. And you can't even modify the unitransracesex cut of the garment... it's against the law, as it's state property, because it could differentiate people... To be honest, it's not a future I'm looking forwards to... but at the same time, I don't want to become like the crazy homeless old man outside the warehouse with his "Rage Against the Machines" sign... --- End of imagination exercise ---
MrBuddyCasino 5y ago
> “We cannot allow corporations to put profit over people” This implies corporations should "put people over profit", which is welfare.
- vincnetas 5y ago
  By people i think we mean "workers". People who do impact on physical reality. And who are, or should be part of corporation. I'm starting to sound like communist :)
- NoahTheDuke 5y ago
  No, it doesn't. It means not privileging profits at the expense of people. It's possible to have profit as a goal without hurting people (in this fashion).
  - MrBuddyCasino 5y ago
    > without hurting people You need to be specific. Injuries? Stress? Responsibility? Overtime? General drudgery? People are paid a salary to do something they wouldn't do otherwise. There is always a downside. Laws exist to limit certain downsides, eg undue physical stress, sexual exploitation or too high a risk of injuries. If that was the actual policy goal, you would not say “We cannot allow corporations to put profit over people”, you would be mentioning a specific downside. Doing otherwise would be dishonest sloganeering.
    - Mordisquitos 5y ago
      Why should they mention a specific downside when they are merely expressing support for the overall idea? Ultimately it is all down to value judgements, in the same way you do not specify what constitutes undue physical stress as opposed to appropriate physical stress, nor when a risk of injury becomes too high.
      - MrBuddyCasino 5y ago
        Please vote for my new party, The Good Party. Our "overall idea" of policy is that we prefer to do good things. You should vote for us if you, in general, like good things. None of the things that we are in favour for have drawbacks, and given enough time and money, our regime will inevitably result in Fully Automated Luxury Communism.
sdedovic 5y ago
I wonder if this will be a net-loss. I'm not discussing whether or not Amazon's system has merit, nor if algorithmic performance evaluation is inherently bad. Look at standardized testing in the USA (SAT, ACT) - there is tons of coaching and preparation specific to the test that isn't training anyone on the material. Instead, we know how much time to spend per question, how to quickly eliminate possible answers on a multiple choice to improve odds when guessing, what patterns in the language a question exist, etc. I think this is so so similar to customer support and that issue big corporations (and especially us, as their customers, face) which is that quality evaluation is not something that can be boiled down to quantitative measurement. Sure, it can weed out many common problems, but it's just not worth it. Judging workers solely based on some rubric will create a class of people who min-max the system and will lower the overall quality of the service provided, while disenfranchising the best employees. On the other hand, maybe this will help move these opaque and (almost certainly) biased processes towards some better baseline.
- advael 5y ago
  Sure, you're going to get some people gaming the metrics, (as happens with all metrics). Currently, by keeping the metrics opaque, Amazon (et al) prevents people from explicitly doing this, although it is using said metrics as a strong selection pressure on their employees, which has kind of a similar effect. But this misses the point entirely. Because these metrics and criteria are not transparent, the people affected by them have no recourse. There is no specific thing they can point to that is unreasonable about the standards by which they are judged, because there is nothing specific accessible to them at all. With no transparency, the "algorithm" could be some stupid nonsense, or something highly illegal, or something boring and human like managers playing favorites by up/downvoting employees on a secret app. The intention of this regulation is to shine a light on these practices, so that both the people affected and regulators can see how stupid and unethical they probably are - instead of speculating - and do specific things about it
  - Applejinx 5y ago
    Very much this. One of the dark patterns of Silicon Valley is the working out of how to take behavior that would be a big problem if you had humans directly doing it, and delegate it to an algorithm. There are a lot of things in society where you can gain an edge and/or profit by doing crime: either straight up stealing, or extorting/assault/bullying, or subtler things like playing into long-standing biases for your own benefit through hiring (say, for a sales force?) only the demographic that people most expect to see in positions of power, and refusing anyone who wouldn't fit that image on the behalf of your most prejudiced customers. They're all variations on crimes to the extent that they're coded into current law. If you plausibly build these crimes into the algorithms in any way, you benefit from committing them while being able to pretend you're not doing it. Stands to reason that the law-makers are interested. In the absence of regulation it's a race to see who can commit the most profitable crimes the most enthusiastically, with much hiring competition for whoever is the best at doing it. With regulation, it's more about competition for whoever is the best at HIDING it :) If it wasn't somewhat (or very) adversarial by nature, we wouldn't need to have regulators in the first place… they're a natural part of a system that inherently tries to grow by any means necessary and can't regulate itself in any meaningful sense. Sounds like things are proceeding as they have to proceed.
antattack 5y ago
We need more of this. Companies should not be able excuse their actions because of 'software mistake' while the tool works exactly as designed until someone points out a shortcoming or bias.
- mabub24 5y ago
  There is very annoying double-speak that goes on when a company's algo comes under fire. They often present the algos as if they are real AI, something set loose and allowed to run free, like a wild animal, rather than something designed to match business purposes. Algos aren't mysterious black boxes, but marketing them as such sure makes the people that make and sell them look like "wizards" to oblivious execs/c-suite people. It's about removing and hiding accountability while they fiddle under the hood behind a curtain like the Wizard of Oz.
  - jjk166 5y ago
    At the end of the day, algorithms are industrial equipment, like a bulldozer. If a bulldozer runs over a bunch of people, saying "the operator doesn't know how to correctly operate a bulldozer" doesn't get you off the hook, if anything it makes it worse.
    - mabub24 5y ago
      I agree. When people act with disbelief at the results of an algo they've made, they're either sincere, in which case they don't actually understand the thing they have built (which would suggest a lack of competence, which many are terrified to admit); or, they're lying, and attempting to absolve themselves of any responsibility for those outcomes. As soon it comes in contact with a human, you don't get to wipe your hands of the outcome if you dislike it. You have to own the successes and the failures.
    - Spivak 5y ago
      It is really bonkers to me that we don't design systems to be introspectable and replay-able. If you have a system relies on sampling from a random distribution to make decisions then you really ought to make the whole system deterministic and provide it with a single logged entropy source so when something goes south you can replay exactly what happened.
spywaregorilla 5y ago
It seems likely to me that the use of algorithmic quotas and whatever is not helpful. Fear of the unknown quota might be, which is why this law is good. But basically... let's assume tasks come to the warehouse on some sort of random poisson distribution, each with a value, and a time they must be completed by. Everything must be completed. If these parameters are highly differentiated, a really smart system might be able to prioritize tasks such that high value things get done quicker than lower value things. But this isn't something that the worker should be figuring out. If the parameters aren't highly differentiated, all you really want to do is move stuff through the queue faster. There's just one master metric in the "perfect" system which is, "how long did it take you to move this item through the queue vs. the expected time to move it through the queue". But to be a FAIR system, you need to be able to parse out what portion of the time to move distribution is based on random chance vs. individual agency. Delays are likely to be long skewed right. That is, most items will pass through with minimal delays but some items will have big logistical problems that completely slow down the process to no fault of the worker. Measuring this is likely to be incredibly difficult and mostly picking up on noise. It could likely catch people who are truly slacking off. But... so could a human manager with basic observational skills.
- raxxorrax 5y ago
  Before defining technicalities, which are of course even more complicated, we should ask ourselves if we want to work this way. It would get a clear 'no' from me. Also the worker doesn't profit in any way if he works faster. Sure, there is very like some form of bonus, but in the end throughput doesn't matter to him. So why the haste?
  - jasode 5y ago
    >Also the worker doesn't profit in any way if he works faster. [...] So why the haste? Yes, it's true that they're not paid piecemeal per box so working "faster" seems pointless but some workers still hustle because they want to keep their jobs. A worker having a defiant attitude of "I don't get paid to work any faster" is fine -- up until the point they are replaced by another worker who is faster -- which means they're unemployed and make $0.
PerkinWarwick 5y ago
Two questions. Are the algorithms necessarily explainable? It seems to me that over time, the all-seeing eye will have a logic that no human really groks. How is this different from surveillance and grading in any other company or profession? As the cost of spying on your employees drops to zero, I would expect this everywhere. Physical injuries are only one aspect.
- hallway_monitor 5y ago
  This is an important point. It would be prudent to very soon outlaw making non-explainable algorithms responsible for decisions that materially affect people's lives.
- deleted 5y ago
  [deleted]
harlanji 5y ago
I had an overall good time working an entry level job with a quota in the Bay Area, that is to say with very smart mostly young folks. The model was clear when I paid attention to how the numbers changed. Our app showed us a velocity metric in real time. Little doubt it was gamed by supervisors and smart employees alike. Some activities pushed the metric up fast, while others chipped it down quick. Pressing buttons in the app faster gave me the biggest performance boost. There were about 3 phases to the job and one gave basically no credit and another was at the whim of customers not keeping us waiting (or their predicted arrival not being too optimistic). I never worried about the metric because I had a detailed enough account that I could defend myself. I left the job for others reasons, overall was happy with it... being a manual job done mostly solo it just gave me time to think a lot. Glad for the workers who are subject to the metrics still because it was indeed a pretty bad feeling to see metrics drop and rise so fast for certain activities and know we could have problems because of it.
akomtu 5y ago
I seriously believe that this "governed by software" is the grim future for 90% of the overcrowded planet in the future: all activity will be uploaded to a "central governing unit" that will send down punishments for misbehaving peons. And algorithms will be trivial: management will be identifying undesired activities on an adhoc basis, invent a metric that somewhat accurately detects this activity (or train an ML model that separates bad apples from good apples) and associate this metric with a negative score. Inputs to the metric function will be all the invidual activity data. The entire system will be millions of opaque metrics that override each other and no one will dare to overhaul the system for it would cause unpredictable effects in society. There will be human judges who will occasionally override the algorithm. All this will be the reaction to the increasing difficulty to keep peons in check. And thanks to the complete and ubiquitous surveillance and reliable internet even in Himalayas, there won't be many places to hide.
literallyaduck 5y ago
We feed numbers of good and bad employees into a machine learning algorithm and it picks likely good employees. Oh, that doesn't work for you? I call a 3rd SaS api and give it numbers and it gives me people to fire, or promote.

156 points by mardiyah 5y ago

This is an important issue. If we talk about regulating tech, this is the most pressing one. I distance myself from all colleagues that help to implement such productivity surveillance. Do something sensible with your life, the money isn't worth it.

thinkharderdev 5y ago
Wouldn't this law make productivity surveillance more pervasive? Based on the description in the article it seems to require that "Going forward, warehouse workers who think their quotas lead to unsafe behavior will be able to ask for 90 days' worth of documentation of how their work meets or fails the quota"?
- raxxorrax 5y ago
  That can be the result of bad regulation, yes, but I think companies were too ambitious in implementing said tools that regulation became a necessity.
- pjc50 5y ago
  No, that can only have been existing documentation. After all, it's Amazon's choice to have quotas, and if they're enforcing them, they must already have been documented, right?
deleted 5y ago
[deleted]

popotamonga 5y ago

It will be fun, workers will start do maximize for the algorithm

emptyparadise 5y ago
Well employers have been doing that for god knows how long, why can't the employees?
- raxxorrax 5y ago
  Such a system will never be to the benefit of the worker. Those that say otherwise want to sell shitty software or are a bit deluded about the consequences of manufacturing.
  - emptyparadise 5y ago
    But when workers come together, interesting things happen.
barrenko 5y ago
The algorithm is displacing the Firm :). Wondering how long before something like Amazon is basically just a DAO?
- idiotsecant 5y ago
  The means of production provides way too much power for humans to ever let it outside their control.
- Arrath 5y ago
  Reminds me of a (not great) book I read about a secretive trading firm that build an AI to maximize profits except it gained enough awareness to take things into its own hands, by hiring mercenaries and assassins to drive the market and ever increase its portfolio. After, of course, killing everyone at the firm.

heymax054 5y ago

I wonder if this will impact productivity in an unexpected way. When you disclose the numbers by which you "track" people, people will find ways to maximize those numbers without necessarily maximizing output. But then, is there a good alternative? Big tech asked for this by being extremely non-transparent and mistreating people. While I agree with the intent, I don't think simple solutions like these are necessarily the best ones to solve the problem.

ganzuul 5y ago
They had the opportunity to solve their own problem. They didn't. Perhaps in the future they will perform better.
- pbaka 5y ago
  --- Beginning of an imagination exercise --- As I age, nearing 50, my bladder is getting weaker. I cannot outperform the younger folks who can hold it for four hours. Should I, as an obviously unperforming resource, save up for a electro-mechanical bladder implant to replace my biological one ? Does the company offer me a savings plan for the same ? Or is it paid from the fused unemployment/health improvement fund of the state where my fulfillment center is located ? How will I manage maintenance payments in 30 years, after the rest of my body is too old for this warehouse job ? Then, of course, I have that auto-immune asthma since the lockdowns of 2022. I remember it like yesterday. I wasn't allowed in school (that was before virtuaducation). I didn't build up an immunity to basic flu, and was wrecked by a pneumonia when I started school. Never really recovered... Then it turned out the 6 vaccine shots had strange interactions with my immune system five years after the troubles ended. My body never quite worked as it should. I always had strange "after-diseases"... For the last three years, there was some stuff my body didn't like in the yearly combine-shots, ended up in the Hospitrailer, again, in coma for five days. The Autodoc said next time might be fatal. My work.pad shows me alerts from the warehouse AI that I might not be allowed on the floor next year if I don't solve the problem - both because of regulation and work time lost. The problem is, antihistamites (yeah, funny branding for those histamine dispensing nanobots) cant solve work with my immune problem. My "Pearly Melinda Gates" insurance's doctorobot said I don't have premiums accrued to pay for the cloned organ assays needed to see what part of my immune system doesn't like it. I even have less money for the immune reprogramming therapy. It's three years wages... It suggested me sell or collateralize the home I inherited. Wheesh, 12% interest. That's steep... Maybe I should just give up, and go on that famed UBI ? Those container-box habunits don't look too bad, and the virtual immersion gear is included in the monthly stipend. Just don't get caught inside when the chlore/fluoride autowash kicks in, hahaha. Then again, I'm not sure. After it proved financially impossible to sustain in the 2040's, it's now basically just food and water ration QR stamps. It's like in the time of my grandparents in the 1980's, except the monthly syntheclothing you receive are soooo butt ugly, not even classy with those huge shoulderpads. And you can't even modify the unitransracesex cut of the garment... it's against the law, as it's state property, because it could differentiate people... To be honest, it's not a future I'm looking forwards to... but at the same time, I don't want to become like the crazy homeless old man outside the warehouse with his "Rage Against the Machines" sign... --- End of imagination exercise ---

MrBuddyCasino 5y ago

> “We cannot allow corporations to put profit over people” This implies corporations should "put people over profit", which is welfare.

vincnetas 5y ago
By people i think we mean "workers". People who do impact on physical reality. And who are, or should be part of corporation. I'm starting to sound like communist :)
NoahTheDuke 5y ago
No, it doesn't. It means not privileging profits at the expense of people. It's possible to have profit as a goal without hurting people (in this fashion).
- MrBuddyCasino 5y ago
  > without hurting people You need to be specific. Injuries? Stress? Responsibility? Overtime? General drudgery? People are paid a salary to do something they wouldn't do otherwise. There is always a downside. Laws exist to limit certain downsides, eg undue physical stress, sexual exploitation or too high a risk of injuries. If that was the actual policy goal, you would not say “We cannot allow corporations to put profit over people”, you would be mentioning a specific downside. Doing otherwise would be dishonest sloganeering.
  - Mordisquitos 5y ago
    Why should they mention a specific downside when they are merely expressing support for the overall idea? Ultimately it is all down to value judgements, in the same way you do not specify what constitutes undue physical stress as opposed to appropriate physical stress, nor when a risk of injury becomes too high.
    - MrBuddyCasino 5y ago
      Please vote for my new party, The Good Party. Our "overall idea" of policy is that we prefer to do good things. You should vote for us if you, in general, like good things. None of the things that we are in favour for have drawbacks, and given enough time and money, our regime will inevitably result in Fully Automated Luxury Communism.

sdedovic 5y ago

I wonder if this will be a net-loss. I'm not discussing whether or not Amazon's system has merit, nor if algorithmic performance evaluation is inherently bad. Look at standardized testing in the USA (SAT, ACT) - there is tons of coaching and preparation specific to the test that isn't training anyone on the material. Instead, we know how much time to spend per question, how to quickly eliminate possible answers on a multiple choice to improve odds when guessing, what patterns in the language a question exist, etc. I think this is so so similar to customer support and that issue big corporations (and especially us, as their customers, face) which is that quality evaluation is not something that can be boiled down to quantitative measurement. Sure, it can weed out many common problems, but it's just not worth it. Judging workers solely based on some rubric will create a class of people who min-max the system and will lower the overall quality of the service provided, while disenfranchising the best employees. On the other hand, maybe this will help move these opaque and (almost certainly) biased processes towards some better baseline.

advael 5y ago
Sure, you're going to get some people gaming the metrics, (as happens with all metrics). Currently, by keeping the metrics opaque, Amazon (et al) prevents people from explicitly doing this, although it is using said metrics as a strong selection pressure on their employees, which has kind of a similar effect. But this misses the point entirely. Because these metrics and criteria are not transparent, the people affected by them have no recourse. There is no specific thing they can point to that is unreasonable about the standards by which they are judged, because there is nothing specific accessible to them at all. With no transparency, the "algorithm" could be some stupid nonsense, or something highly illegal, or something boring and human like managers playing favorites by up/downvoting employees on a secret app. The intention of this regulation is to shine a light on these practices, so that both the people affected and regulators can see how stupid and unethical they probably are - instead of speculating - and do specific things about it
- Applejinx 5y ago
  Very much this. One of the dark patterns of Silicon Valley is the working out of how to take behavior that would be a big problem if you had humans directly doing it, and delegate it to an algorithm. There are a lot of things in society where you can gain an edge and/or profit by doing crime: either straight up stealing, or extorting/assault/bullying, or subtler things like playing into long-standing biases for your own benefit through hiring (say, for a sales force?) only the demographic that people most expect to see in positions of power, and refusing anyone who wouldn't fit that image on the behalf of your most prejudiced customers. They're all variations on crimes to the extent that they're coded into current law. If you plausibly build these crimes into the algorithms in any way, you benefit from committing them while being able to pretend you're not doing it. Stands to reason that the law-makers are interested. In the absence of regulation it's a race to see who can commit the most profitable crimes the most enthusiastically, with much hiring competition for whoever is the best at doing it. With regulation, it's more about competition for whoever is the best at HIDING it :) If it wasn't somewhat (or very) adversarial by nature, we wouldn't need to have regulators in the first place… they're a natural part of a system that inherently tries to grow by any means necessary and can't regulate itself in any meaningful sense. Sounds like things are proceeding as they have to proceed.

antattack 5y ago

We need more of this. Companies should not be able excuse their actions because of 'software mistake' while the tool works exactly as designed until someone points out a shortcoming or bias.

mabub24 5y ago
There is very annoying double-speak that goes on when a company's algo comes under fire. They often present the algos as if they are real AI, something set loose and allowed to run free, like a wild animal, rather than something designed to match business purposes. Algos aren't mysterious black boxes, but marketing them as such sure makes the people that make and sell them look like "wizards" to oblivious execs/c-suite people. It's about removing and hiding accountability while they fiddle under the hood behind a curtain like the Wizard of Oz.
- jjk166 5y ago
  At the end of the day, algorithms are industrial equipment, like a bulldozer. If a bulldozer runs over a bunch of people, saying "the operator doesn't know how to correctly operate a bulldozer" doesn't get you off the hook, if anything it makes it worse.
  - mabub24 5y ago
    I agree. When people act with disbelief at the results of an algo they've made, they're either sincere, in which case they don't actually understand the thing they have built (which would suggest a lack of competence, which many are terrified to admit); or, they're lying, and attempting to absolve themselves of any responsibility for those outcomes. As soon it comes in contact with a human, you don't get to wipe your hands of the outcome if you dislike it. You have to own the successes and the failures.
  - Spivak 5y ago
    It is really bonkers to me that we don't design systems to be introspectable and replay-able. If you have a system relies on sampling from a random distribution to make decisions then you really ought to make the whole system deterministic and provide it with a single logged entropy source so when something goes south you can replay exactly what happened.

spywaregorilla 5y ago

It seems likely to me that the use of algorithmic quotas and whatever is not helpful. Fear of the unknown quota might be, which is why this law is good. But basically... let's assume tasks come to the warehouse on some sort of random poisson distribution, each with a value, and a time they must be completed by. Everything must be completed. If these parameters are highly differentiated, a really smart system might be able to prioritize tasks such that high value things get done quicker than lower value things. But this isn't something that the worker should be figuring out. If the parameters aren't highly differentiated, all you really want to do is move stuff through the queue faster. There's just one master metric in the "perfect" system which is, "how long did it take you to move this item through the queue vs. the expected time to move it through the queue". But to be a FAIR system, you need to be able to parse out what portion of the time to move distribution is based on random chance vs. individual agency. Delays are likely to be long skewed right. That is, most items will pass through with minimal delays but some items will have big logistical problems that completely slow down the process to no fault of the worker. Measuring this is likely to be incredibly difficult and mostly picking up on noise. It could likely catch people who are truly slacking off. But... so could a human manager with basic observational skills.

raxxorrax 5y ago
Before defining technicalities, which are of course even more complicated, we should ask ourselves if we want to work this way. It would get a clear 'no' from me. Also the worker doesn't profit in any way if he works faster. Sure, there is very like some form of bonus, but in the end throughput doesn't matter to him. So why the haste?
- jasode 5y ago
  >Also the worker doesn't profit in any way if he works faster. [...] So why the haste? Yes, it's true that they're not paid piecemeal per box so working "faster" seems pointless but some workers still hustle because they want to keep their jobs. A worker having a defiant attitude of "I don't get paid to work any faster" is fine -- up until the point they are replaced by another worker who is faster -- which means they're unemployed and make $0.

PerkinWarwick 5y ago

Two questions. Are the algorithms necessarily explainable? It seems to me that over time, the all-seeing eye will have a logic that no human really groks. How is this different from surveillance and grading in any other company or profession? As the cost of spying on your employees drops to zero, I would expect this everywhere. Physical injuries are only one aspect.

hallway_monitor 5y ago
This is an important point. It would be prudent to very soon outlaw making non-explainable algorithms responsible for decisions that materially affect people's lives.
deleted 5y ago
[deleted]

harlanji 5y ago

I had an overall good time working an entry level job with a quota in the Bay Area, that is to say with very smart mostly young folks. The model was clear when I paid attention to how the numbers changed. Our app showed us a velocity metric in real time. Little doubt it was gamed by supervisors and smart employees alike. Some activities pushed the metric up fast, while others chipped it down quick. Pressing buttons in the app faster gave me the biggest performance boost. There were about 3 phases to the job and one gave basically no credit and another was at the whim of customers not keeping us waiting (or their predicted arrival not being too optimistic). I never worried about the metric because I had a detailed enough account that I could defend myself. I left the job for others reasons, overall was happy with it... being a manual job done mostly solo it just gave me time to think a lot. Glad for the workers who are subject to the metrics still because it was indeed a pretty bad feeling to see metrics drop and rise so fast for certain activities and know we could have problems because of it.

akomtu 5y ago

I seriously believe that this "governed by software" is the grim future for 90% of the overcrowded planet in the future: all activity will be uploaded to a "central governing unit" that will send down punishments for misbehaving peons. And algorithms will be trivial: management will be identifying undesired activities on an adhoc basis, invent a metric that somewhat accurately detects this activity (or train an ML model that separates bad apples from good apples) and associate this metric with a negative score. Inputs to the metric function will be all the invidual activity data. The entire system will be millions of opaque metrics that override each other and no one will dare to overhaul the system for it would cause unpredictable effects in society. There will be human judges who will occasionally override the algorithm. All this will be the reaction to the increasing difficulty to keep peons in check. And thanks to the complete and ubiquitous surveillance and reliable internet even in Himalayas, there won't be many places to hide.

literallyaduck 5y ago

We feed numbers of good and bad employees into a machine learning algorithm and it picks likely good employees. Oh, that doesn't work for you? I call a 3rd SaS api and give it numbers and it gives me people to fire, or promote.