4 ms·

> Maybe it's greed in this case, but in my time at Amazon I saw many teams in "Keep Lights On" mode where one or two people with vague memories of how services

> Maybe it's greed in this case, but in my time at Amazon I saw many teams in "Keep Lights On" mode where one or two people with vague memories of how services are supposed to work try to keep them running I worked for Amazon as well, and I'd say it is greed, not in that the behavior was purposely designed to be malicious, but in the way they don't properly fund developers to work through their tech depts, and spend proper time and effort to make sure all the systems work, have been thoroughly tested, have lots of test coverage, that the teams manages to keep subject matter expert around, etc. Amazon doesn't because it's too expansive to have dev resources doing all that, so most projects get released using the bare minimum, and get maintained using the bare minimum.

a2800276 5y ago
I'm always amazed that people can't see this as greed. If the CEO of a medical equipment company said: "screw all these expensive software engineers, physicists and testers, I have a 14 year old niece who can code up our radiation therapy machine for nickels" no one would be blaming incompetence of devs. I don't see how all the lack of solid requirements, software development process, testing and means of recourse are any different, apart from the potential consequences being non fatal, obviously.
- ALittleLight 5y ago
  I just meant to differentiate it from people at Amazon going "Let's cheat the publishers out of their royalty!" Of course that might be what happened, but it would surprise me. If you want to think of not hiring enough people to do the work that's required as greedy - then I think it's fair to call greed as the source of the problem. Another element of this that cuts against "greed" as the motivator in my mind is that it isn't like anyone who would implement the details here would give a hoot if Amazon went plus or minus ten million dollars (or whatever the amount is). If you are the leaf node in the org chart making the exchange of money happen then you will get paid the same regardless of what happens. There may be some big boss guy somewhere who cares about that number, but he doesn't have the ability to actually do the cheating, he'd have to order someone else to do it. Again, maybe that's what happened, but it just seems unlikely to me as it would be so easy for the people receiving those orders to go "Hmmm, no, I don't think so."
  - a2800276 5y ago
    "Let's cheat the publishers out of their royalty!" is greed and: "Let's save money creating the services we provide, almost certainly at the cost of our customers in order to be more profitable" is not? "in my mind is that it isn't like anyone who would implement the details here would give a hoot if Amazon went plus or minus ten million dollars" in that case I'd argue it's a massive management failure on Amazon to have a bunch of people working for them that do not care about the consequences of their actions. To labor my previous metaphor: somehow the radiation machine manufacturer finds a way to get its employees to care that their products work as intended. And while I agree it would be morally worse if they were intentionally bilking customers (be it content producers or listeners), sins-of-omission are very real as well. You're right that $10mio are nothing to Amazon, but $10k could easily be the difference between wild success of an independent audio book creator and their financial ruin.
- Aeolun 5y ago
  > apart from the potential consequences being non fatal Kind of a major factor though.
- jboy55 5y ago
  Its not just company greed, you can't get any engineers to agree to maintain a service in Keep the Lights on mode. Its not in their interest, let me explain. First, there's 0 chance at promotion doing maintenance work, so L4 (jr) engineers will bounce after a year or so as they have a clock running and have to move up or out. L5 engineers will stagnate and they will at risk for URA as they are supposed to keep improving. Frankly the work isn't L6 worthy, an L6 maintaining this as the only thing they are doing won't last a quarter. So these services that just 'do the job' get bounced around from team to team and perhaps when they get old and creaky enough, an L4 can promo themselves doing an upgrade/refactor. Until then, they just exist and are a bane to some team where they only produce off hour pages, even if a whole publisher segment depends on them to get paid.
  - dogleash 5y ago
    >you can't get any engineers to agree to maintain a service in Keep the Lights on mode ...and then you go on to explain in business terms that only apply to the handful of companies that have cargo-culted similar dev team designs and incentives. I started writing a comment here about how my current employer structures and incentivizes maintenance but I've deleted it and decided not to give it away that info for free.
    - jboy55 5y ago
      I don't think the culture I designed is a 'cargo-cult', but more of the result of a first order solution to designing engineer incentives. "We want to encourage engineers to get better over time, be entrepreneurial and feel in charge of their promotion" leads directly to "Maintenance work does not allow me to succeed".

a2800276 5y ago

I'm always amazed that people can't see this as greed. If the CEO of a medical equipment company said: "screw all these expensive software engineers, physicists and testers, I have a 14 year old niece who can code up our radiation therapy machine for nickels" no one would be blaming incompetence of devs. I don't see how all the lack of solid requirements, software development process, testing and means of recourse are any different, apart from the potential consequences being non fatal, obviously.

ALittleLight 5y ago
I just meant to differentiate it from people at Amazon going "Let's cheat the publishers out of their royalty!" Of course that might be what happened, but it would surprise me. If you want to think of not hiring enough people to do the work that's required as greedy - then I think it's fair to call greed as the source of the problem. Another element of this that cuts against "greed" as the motivator in my mind is that it isn't like anyone who would implement the details here would give a hoot if Amazon went plus or minus ten million dollars (or whatever the amount is). If you are the leaf node in the org chart making the exchange of money happen then you will get paid the same regardless of what happens. There may be some big boss guy somewhere who cares about that number, but he doesn't have the ability to actually do the cheating, he'd have to order someone else to do it. Again, maybe that's what happened, but it just seems unlikely to me as it would be so easy for the people receiving those orders to go "Hmmm, no, I don't think so."
- a2800276 5y ago
  "Let's cheat the publishers out of their royalty!" is greed and: "Let's save money creating the services we provide, almost certainly at the cost of our customers in order to be more profitable" is not? "in my mind is that it isn't like anyone who would implement the details here would give a hoot if Amazon went plus or minus ten million dollars" in that case I'd argue it's a massive management failure on Amazon to have a bunch of people working for them that do not care about the consequences of their actions. To labor my previous metaphor: somehow the radiation machine manufacturer finds a way to get its employees to care that their products work as intended. And while I agree it would be morally worse if they were intentionally bilking customers (be it content producers or listeners), sins-of-omission are very real as well. You're right that $10mio are nothing to Amazon, but $10k could easily be the difference between wild success of an independent audio book creator and their financial ruin.
Aeolun 5y ago
> apart from the potential consequences being non fatal Kind of a major factor though.
jboy55 5y ago
Its not just company greed, you can't get any engineers to agree to maintain a service in Keep the Lights on mode. Its not in their interest, let me explain. First, there's 0 chance at promotion doing maintenance work, so L4 (jr) engineers will bounce after a year or so as they have a clock running and have to move up or out. L5 engineers will stagnate and they will at risk for URA as they are supposed to keep improving. Frankly the work isn't L6 worthy, an L6 maintaining this as the only thing they are doing won't last a quarter. So these services that just 'do the job' get bounced around from team to team and perhaps when they get old and creaky enough, an L4 can promo themselves doing an upgrade/refactor. Until then, they just exist and are a bane to some team where they only produce off hour pages, even if a whole publisher segment depends on them to get paid.
- dogleash 5y ago
  >you can't get any engineers to agree to maintain a service in Keep the Lights on mode ...and then you go on to explain in business terms that only apply to the handful of companies that have cargo-culted similar dev team designs and incentives. I started writing a comment here about how my current employer structures and incentivizes maintenance but I've deleted it and decided not to give it away that info for free.
  - jboy55 5y ago
    I don't think the culture I designed is a 'cargo-cult', but more of the result of a first order solution to designing engineer incentives. "We want to encourage engineers to get better over time, be entrepreneurial and feel in charge of their promotion" leads directly to "Maintenance work does not allow me to succeed".