14 ms·

Bad automated policy is bad because the policy is bad, not because automation is bad. Automation applies policy evenly. An employee can be sure that the firin

Bad automated policy is bad because the policy is bad, not because automation is bad. Automation applies policy evenly. An employee can be sure that the firing wasn't due to a boss having a bad day, not liking the employee personally, not noticing worse performance of colleagues, etc. The real problem comes when automated assessment is poorly designed. And it's a shame not only because of the heartless actions that impact real people's lives and the needless public relations damage that is bound to emerge. It's also a shame because of the lost opportunity to help workers' performance improve via feedback at a level of detail that the automated systems could provide. Damn the secrecy. Develop an open rating system that provides continuous feedback to the worker. Build in dampers that provide ample time to learn, correct mistakes, and improve. Publish every factor, every consideration, every weight that goes into the assessment. Openly. Refine over and over until both management and the workforce see it as fair and helpful. I would take that system over the subjective, unevenly applied, unclear criteria that's historically been used to assess performance. No surprises to the workers, and management has to own the criteria rather than hide behind unpublished automation. Objectivity should be a virtue. Without transparency, it's going to prove to be quite the vice.

nkingsy 5y ago
I was thinking similarly in the "FU IG" thread. The only way I see capitalism surviving long-term is if we can embrace data collection in all aspects of life and make it work for us to internalize negative externalitites on an ongoing basis.
crazygringo 5y ago
This is a great point. However, I'll just add that, having tried to design such systems in the past (for large-scale Mechanical Turk work), it is fiendishly complex. Not only is it incredibly difficult to turn our intuitions of "what's fair" into formulas, but then you've got to deal with people trying to game those rules for their own advantage once they figure them out, which ultimately means you can only ever be partially transparent at best. In fact, I'd go so far as to say it's impossible for all pretty much practical purposes that need to be cost-effective, sadly. It's a really tough problem.
- planet-and-halo 5y ago
  This is one of those things I wish more people understood. Second-order effects can be hard to predict, but if it's a rule and it involves human beings, then you can be pretty sure the second-order effect will be "people try to game the rule."
- derefr 5y ago
  > people trying to game those rules for their own advantage once they figure them out I mean, good for them. They deserve my money for obeying the letter of my requirements. That's exactly what I want them to do. If the letter of the requirements don't accurately and precisely represent the spirit of the requirements, that's on me. IMHO that sort of adversarial relationship is extremely helpful at developing a good formal spec for the requirements in the first place. It's TDD, but with a monetary incentive on the part of the developer to "code to make the test pass" and nothing more. You don't get to rely on the developer's charitability—the only knob you get to turn, is what test you write! Of course, in the end, the "tests" might need to be Sufficiently Advanced for this to work, e.g. a spec that specifies something as "what a layman would understand X to be" — where actually evaluating that in an automated way would involve automating a "jury of your peers" type setup to act as the "layman."
  - crazygringo 5y ago
    I think you're misunderstanding, so here's an example: The rules might be set up to allow 4 excuses of "building couldn't be accessed" per week before a warning, and 3 warnings in any 60-day period before firing. If you don't know this, then you simply do your job, and report when a building can't be accessed. If you do know this, then sometimes you're feeling lazy, want to go home a bit earlier and not do your job, and you lie and report the building can't be accessed, because you know you're under the threshold and so won't be fired. This isn't about requirements. It's about setting thresholds to catch people who are intentionally not performing their job.
    - gjs278 5y ago
      the rules might also say that excuse only works when a system that checks the doors confirms that the building wasn’t accessible. saying I couldn’t access the building with zero proof (a picture) could lead to a review later on where the termination happens for lying. it doesn’t have to be 100% automated and even if it was, they could automate out liars eventually.
6gvONxR4sf7o 5y ago
Automated policy like this will be bad until we have general purpose strong AI, because there's no room for people to be edge cases in human ways. You need to automate room for life to happen.
- deleted 5y ago
  [deleted]
burkaman 5y ago
Why do you think that evaluation period, where there are still humans involved tweaking the system, would ever end? The world is going to keep changing. An attendance policy that everyone agreed on suddenly isn't going to work in a pandemic, and humans will need to intervene. A rating system will need to be changed when someone finds a vulnerability in the website and spams every worker with a thousand bad reviews. It's going to constantly need changes, need constant supervision and review, there will never be a point when the humans can step away and let the algorithm run the ship. It just seems like, at the absolute best, normal human management with extra steps.
- kjhughes 5y ago
  You're right: continuous refinement will indeed have to continue indefinitely. This is true with and without policy automation because policy itself has to evolve to address, as you say, a changing world, as well as the organization's changing goals and its outlook on the effectiveness of the existing policy in meeting those goals.
brundolf 5y ago
Totally disagree. No system of rules could ever handle all the obscure edge-cases that humans can. Just look to all the Google account automated-flagging issues that pop up on HN, for reference. What could maybe work is an automated system for the "easy 90%", paired with an appeals system well-staffed by human beings. But that isn't Amazon's plan (or Google's, etc), because that 10% will cost much more than the first 90%, so it's easier for them to just write it off as a loss. And at their scale, that margin of error translates to thousands of potentially-ruined lives.
- kjhughes 5y ago
  If no system of rules can ever cover the policy space, then the policy itself is necessarily ill-defined. That alone is a problem before automation considerations. I'm arguing that automation provides the opportunity to make good, open assessment part of a process of improvement rather than opaquely driving firings.
  - brundolf 5y ago
    > then the policy itself is necessarily ill-defined I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. The conceit that it could is one of the biggest driving factors of our nascent tech-dystopia. An open policy would of course be better than a closed policy, if only because it would lay bare all of its flaws, but even then it could never be completely fixed.
    - neuronexmachina 5y ago
      > I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. "All models are wrong, but some are useful."
      - version_five 5y ago
        Not when people are punished as a result of them. It's fine to make a generalization that works on average for some low stakes, benign decision, but when it materially affects real people, it is not right to generalize. This is the core argument imo against algorithmic decision making of this kind.
Retric 5y ago
Often in Amazon’s case it’s not about the policy as written it’s about trying to break the law. Laying people off is very expensive. However, constantly fire people at random and you can quickly grow and shrink your workforce for free!
WalterBright 5y ago
Any algorithm can and will be gamed. Making it public means people will adhere to every detail, and yet still be unproductive. There's even a name for it: "work to rule". https://en.wikipedia.org/wiki/Work-to-rule https://en.wikipedia.org/wiki/Work-to-rule
- kjhughes 5y ago
  Work-to-rule arises due to misaligned goals. Poor objective functions can be gamed; good ones align goals such that gaming and aligned goal attainment become the same. I'd rather face the (admittedly formidable) challenge of designing such objective functions in the open than rely on undefined or secretive policy that's hidden. Assessment requires definition, by definition.
  - WalterBright 5y ago
    Every place I've ever worked, everyone knew who the slackers were. No assessments were necessary. It's pretty much impossible to hide incompetence from your coworkers.
grayfaced 5y ago
I came to similar conclusion of transparency but with completely different reasoning. I see this as amazon trying to completely eliminate responsibility from supervisors and lower management. Does amazon really think they can discover new unknown faults in the business with automation alone? Suppose I reverse engineer how the system works and find a way to steal, but have that loss attributed to different workers. How will that get discovered? how many honest workers will get fired with no recourse? You can't replace trust with automation. You still need subjective critical thinking. If the algorithm can find the bad apples, it should be capable of producing metrics showing why they're underperforming to be reviewed by supervisor.
- asteroidbelt 5y ago
  Good point about supervisors and middle managers (or rather low managers). I think it is possible to have 10 effective supervisors, but not 1000. 50 of these 1000 start making subjective decisions (based on sex, race, having hungry children and other prejudices), and the whole system will start working much less efficiently than ML, and it will be worse for both Amazon (worse quality) and hard working delivery men (being fired because of supervision misjudgment).
deleted 5y ago
[deleted]
barrkel 5y ago
Quantity is a quality all of its own. A single photo is a picture. 150 thousand photos is a movie.
pretendgeneer 5y ago
Automation doesn't stop you from getting fired because the boss is having a bad day. Strong labour laws and unions stop you from being fired because the boss is having a bad day.
- asteroidbelt 5y ago
  There was an All In episode about unions recently. Basically they said, unions are necessary in places with monopolies (ex mining towns) and workers need protection from company abuse of power, but unions are harmful where high mobility is possible: high performers would leave because they are not paid enough, and low performers would not leave, union protects them. Eventually a union will slow down the company.
powerapple 5y ago
Automation is probably better than HR. And if the cost saved by not having HR can be transferred to workers, it would be a very good solution. I trust computers more than human to be honest.
raxxorrax 5y ago
> The real problem comes when automated assessment is poorly designed Using automation for worker surveillance is the least innovative employment of tech. That workplace will never be good. The thing you are correct about is that it isn't due to the technology itself. > It's also a shame because of the lost opportunity to help workers' performance improve via feedback at a level of detail that the automated systems could provide. Unrealistic with the one exception that you provide the info only to the specific worker. Will never work that way if that info isn't kept private. Source: Experience of anyone that ever worked at a company where middle management must defend its existence to higher ups. It is just a level of controlling that isn't needed. The time sink alone is a larger waste than the performance increases you net from such measures. And it really gamifies you work, so don't use it on anyone smarter than a jellyfish.
omgwtfbyobbq 5y ago
> Automation applies policy evenly. Automation applies policy based on it's inputs/systems. If someone went out of their way to insure/test that it applies policy evenly, there's a decent chance it will. If not, it'll apply policy based on whatever it's inputs/systems are.

nkingsy 5y ago

I was thinking similarly in the "FU IG" thread. The only way I see capitalism surviving long-term is if we can embrace data collection in all aspects of life and make it work for us to internalize negative externalitites on an ongoing basis.

crazygringo 5y ago

This is a great point. However, I'll just add that, having tried to design such systems in the past (for large-scale Mechanical Turk work), it is fiendishly complex. Not only is it incredibly difficult to turn our intuitions of "what's fair" into formulas, but then you've got to deal with people trying to game those rules for their own advantage once they figure them out, which ultimately means you can only ever be partially transparent at best. In fact, I'd go so far as to say it's impossible for all pretty much practical purposes that need to be cost-effective, sadly. It's a really tough problem.

planet-and-halo 5y ago
This is one of those things I wish more people understood. Second-order effects can be hard to predict, but if it's a rule and it involves human beings, then you can be pretty sure the second-order effect will be "people try to game the rule."
derefr 5y ago
> people trying to game those rules for their own advantage once they figure them out I mean, good for them. They deserve my money for obeying the letter of my requirements. That's exactly what I want them to do. If the letter of the requirements don't accurately and precisely represent the spirit of the requirements, that's on me. IMHO that sort of adversarial relationship is extremely helpful at developing a good formal spec for the requirements in the first place. It's TDD, but with a monetary incentive on the part of the developer to "code to make the test pass" and nothing more. You don't get to rely on the developer's charitability—the only knob you get to turn, is what test you write! Of course, in the end, the "tests" might need to be Sufficiently Advanced for this to work, e.g. a spec that specifies something as "what a layman would understand X to be" — where actually evaluating that in an automated way would involve automating a "jury of your peers" type setup to act as the "layman."
- crazygringo 5y ago
  I think you're misunderstanding, so here's an example: The rules might be set up to allow 4 excuses of "building couldn't be accessed" per week before a warning, and 3 warnings in any 60-day period before firing. If you don't know this, then you simply do your job, and report when a building can't be accessed. If you do know this, then sometimes you're feeling lazy, want to go home a bit earlier and not do your job, and you lie and report the building can't be accessed, because you know you're under the threshold and so won't be fired. This isn't about requirements. It's about setting thresholds to catch people who are intentionally not performing their job.
  - gjs278 5y ago
    the rules might also say that excuse only works when a system that checks the doors confirms that the building wasn’t accessible. saying I couldn’t access the building with zero proof (a picture) could lead to a review later on where the termination happens for lying. it doesn’t have to be 100% automated and even if it was, they could automate out liars eventually.

6gvONxR4sf7o 5y ago

Automated policy like this will be bad until we have general purpose strong AI, because there's no room for people to be edge cases in human ways. You need to automate room for life to happen.

deleted 5y ago
[deleted]

burkaman 5y ago

Why do you think that evaluation period, where there are still humans involved tweaking the system, would ever end? The world is going to keep changing. An attendance policy that everyone agreed on suddenly isn't going to work in a pandemic, and humans will need to intervene. A rating system will need to be changed when someone finds a vulnerability in the website and spams every worker with a thousand bad reviews. It's going to constantly need changes, need constant supervision and review, there will never be a point when the humans can step away and let the algorithm run the ship. It just seems like, at the absolute best, normal human management with extra steps.

kjhughes 5y ago
You're right: continuous refinement will indeed have to continue indefinitely. This is true with and without policy automation because policy itself has to evolve to address, as you say, a changing world, as well as the organization's changing goals and its outlook on the effectiveness of the existing policy in meeting those goals.

brundolf 5y ago

Totally disagree. No system of rules could ever handle all the obscure edge-cases that humans can. Just look to all the Google account automated-flagging issues that pop up on HN, for reference. What could maybe work is an automated system for the "easy 90%", paired with an appeals system well-staffed by human beings. But that isn't Amazon's plan (or Google's, etc), because that 10% will cost much more than the first 90%, so it's easier for them to just write it off as a loss. And at their scale, that margin of error translates to thousands of potentially-ruined lives.

kjhughes 5y ago
If no system of rules can ever cover the policy space, then the policy itself is necessarily ill-defined. That alone is a problem before automation considerations. I'm arguing that automation provides the opportunity to make good, open assessment part of a process of improvement rather than opaquely driving firings.
- brundolf 5y ago
  > then the policy itself is necessarily ill-defined I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. The conceit that it could is one of the biggest driving factors of our nascent tech-dystopia. An open policy would of course be better than a closed policy, if only because it would lay bare all of its flaws, but even then it could never be completely fixed.
  - neuronexmachina 5y ago
    > I'm saying no policy could ever be defined well enough to cover all possible contingencies of a messy, human world. "All models are wrong, but some are useful."
    - version_five 5y ago
      Not when people are punished as a result of them. It's fine to make a generalization that works on average for some low stakes, benign decision, but when it materially affects real people, it is not right to generalize. This is the core argument imo against algorithmic decision making of this kind.

Retric 5y ago

Often in Amazon’s case it’s not about the policy as written it’s about trying to break the law. Laying people off is very expensive. However, constantly fire people at random and you can quickly grow and shrink your workforce for free!

WalterBright 5y ago

Any algorithm can and will be gamed. Making it public means people will adhere to every detail, and yet still be unproductive. There's even a name for it: "work to rule". https://en.wikipedia.org/wiki/Work-to-rule https://en.wikipedia.org/wiki/Work-to-rule

kjhughes 5y ago
Work-to-rule arises due to misaligned goals. Poor objective functions can be gamed; good ones align goals such that gaming and aligned goal attainment become the same. I'd rather face the (admittedly formidable) challenge of designing such objective functions in the open than rely on undefined or secretive policy that's hidden. Assessment requires definition, by definition.
- WalterBright 5y ago
  Every place I've ever worked, everyone knew who the slackers were. No assessments were necessary. It's pretty much impossible to hide incompetence from your coworkers.

grayfaced 5y ago

I came to similar conclusion of transparency but with completely different reasoning. I see this as amazon trying to completely eliminate responsibility from supervisors and lower management. Does amazon really think they can discover new unknown faults in the business with automation alone? Suppose I reverse engineer how the system works and find a way to steal, but have that loss attributed to different workers. How will that get discovered? how many honest workers will get fired with no recourse? You can't replace trust with automation. You still need subjective critical thinking. If the algorithm can find the bad apples, it should be capable of producing metrics showing why they're underperforming to be reviewed by supervisor.

asteroidbelt 5y ago
Good point about supervisors and middle managers (or rather low managers). I think it is possible to have 10 effective supervisors, but not 1000. 50 of these 1000 start making subjective decisions (based on sex, race, having hungry children and other prejudices), and the whole system will start working much less efficiently than ML, and it will be worse for both Amazon (worse quality) and hard working delivery men (being fired because of supervision misjudgment).

deleted 5y ago

[deleted]

barrkel 5y ago

Quantity is a quality all of its own. A single photo is a picture. 150 thousand photos is a movie.

pretendgeneer 5y ago

Automation doesn't stop you from getting fired because the boss is having a bad day. Strong labour laws and unions stop you from being fired because the boss is having a bad day.

asteroidbelt 5y ago
There was an All In episode about unions recently. Basically they said, unions are necessary in places with monopolies (ex mining towns) and workers need protection from company abuse of power, but unions are harmful where high mobility is possible: high performers would leave because they are not paid enough, and low performers would not leave, union protects them. Eventually a union will slow down the company.

powerapple 5y ago

Automation is probably better than HR. And if the cost saved by not having HR can be transferred to workers, it would be a very good solution. I trust computers more than human to be honest.

raxxorrax 5y ago

> The real problem comes when automated assessment is poorly designed Using automation for worker surveillance is the least innovative employment of tech. That workplace will never be good. The thing you are correct about is that it isn't due to the technology itself. > It's also a shame because of the lost opportunity to help workers' performance improve via feedback at a level of detail that the automated systems could provide. Unrealistic with the one exception that you provide the info only to the specific worker. Will never work that way if that info isn't kept private. Source: Experience of anyone that ever worked at a company where middle management must defend its existence to higher ups. It is just a level of controlling that isn't needed. The time sink alone is a larger waste than the performance increases you net from such measures. And it really gamifies you work, so don't use it on anyone smarter than a jellyfish.

omgwtfbyobbq 5y ago

> Automation applies policy evenly. Automation applies policy based on it's inputs/systems. If someone went out of their way to insure/test that it applies policy evenly, there's a decent chance it will. If not, it'll apply policy based on whatever it's inputs/systems are.