19 ms·

Former AWS engineer here. I worked on a pretty critical product in AWS (big AWS service with lots of traffic) and I can safely say that it's totally up to your

• middleclick 5y ago
  > Code quality was atrocious. We had one enormous Java method (>1000 lines) which would take care of nearly every single request coming into our service... With only about 7-8 unit tests AWS is pretty reliable for the most part so I am pretty surprise that the code quality is that bad.
  • colde 5y ago
    I think that highly depends on the service. The new App Runner service for instance is a wild ride of buggyness, lack of testing and incorrect documentation.
  • salil999 5y ago
    I think one thing I learned from AWS is that there's so much hidden away from the customer. There definitely were (and probably still are) many issues which the customers won't actively experience. Reliability doesn't necessarily equate to good standards and good practice. But yes, from a customer point of view, AWS is pretty nice.
    • cpach 5y ago
      Reminds me of that old quote by John Godfrey Saxe: ”Laws, like sausages, cease to inspire respect in proportion as we know how they are made.”
      • markus_zhang 5y ago
        "I just had one for breakfast." -- Jim Hacker
  • wpietri 5y ago
    > AWS is pretty reliable for the most part so I am pretty surprise that the code quality is that bad. I'm not totally surprised because of two factors: very stable product definitions and lots and lots of users. A number of years back, I was talking with people at a famous and popular site with a broad audience. I asked them how much unit testing they did. They said that particular isolated pieces sometimes had tests. But most of the user-facing stuff didn't because they had one-button rollout and one-button rollback. Instead of bothering with unit tests, they'd just frequently release changes, watch the metrics and the customer support queue, and quickly roll back if they'd introduced a bug.
• lucasyvas 5y ago
  Yeah there's no way I'd put up with this. Trillion dollar company and still operating like it's amateur hour on many teams it seems.
  • cduzz 5y ago
    The org is getting what it wants out of the team; why would they fix it? As long as you've got meat for the grinder, you can keep making sausage.
    • rodgerd 5y ago
      Also: the cruelty is the point.
      • cduzz 5y ago
        I'd disagree with that. One doesn't need to be cruel to be callous or oblivious. One can intentionally remain oblivious, but that doesn't imply that the cruelty is an actual goal. See also people who happily eat meat (myself included) who are still troubled (as a 5th order concern) by the process.
  • papito 5y ago
    American capitalism is built on employee heroics, but at least you can get the AWS stamp on your resie, which, let's be honest, will make you the "it" girl of job hunting.
    • ghaff 5y ago
      I think you overestimate the appeal of a handful of large companies, taken by itself, on a resume if only because they come with downsides as well.
      • papito 5y ago
        The level of abuse and mismanagement I put up with before I got older is pretty shocking. Young people have a much higher tolerance for BS, in many cases because they don't have enough experience to know that their workplace is broken.
• markus_zhang 5y ago
  I'm always surprised why industry leads (like Amazon) sometimes treat their products as an amateur treats his/her weekend projects. This is definitely not the worst as I know one of the leading options marketmaker has been using a giant shit mountain of MS Access/Excel VBA code to run their system since the 90s. Last time I heard about it (a few years ago) they are planning to replace that shit mountain with something new but I don't know if it's done now.
  • nobleach 5y ago
    It's built into the culture to ship, ship, ship. Shipped code is better than good code, or clean code, or fast code. At my last job, the C-level was fascinated by Amazon success stories. They wanted to achieve the same success, so they urged us to ship, ship, ship. Unfortunately, we were all very seasoned engineers, and we knew the nightmare that would ensue if we purposely piled on the tech debt. The part of this article that refers to "every ticket taking 4 to 5 days regardless of complexity" should be a warning sign to ANYONE attempting to model their startup after Amazon. The message should be: You are not Amazon, and you will NOT get to Amazon scale by modeling their worst practices.
    • markus_zhang 5y ago
      I think this culture is OK or even good to a company in start-up because you have to be quick. Once it grow into maturity those rules should be abandonned.
      • nobleach 5y ago
        There are a few problems with that approach. One is that there is rarely a good time to say, "ok, we've proven that this idea works, let's now go back and do it correctly". It's a constant stream of fixes on a system that "already works". Secondly, telling management that a team was able to go fast previously, but is now going to start being slow so that things can be done correctly, is a quick way to be shown the door. This is evidenced by Amazon and that thousand line Java method that's probably existed for 10 years. Finding that time where you're mature enough to switch gears never seems to happen in practice. I now advocate a "cut corners, cut scope, but do NOT cut quality" approach. Unit tests do not take much longer to write - but they pay dividends when it's time to refactor. I'm now back in a startup where the code was written with that "just ship it" attitude. The code is so terrible that it can take days to fix a bug. I can rewrite entire features (correctly) in that amount of time.
• awscurrentdev 5y ago
  Current AWS engineer here, can confirm. I'm absolutely broken. I'd second the point about the manager and pre-existing conditions making up the job. It's not clear to me if it's endemic, these are big orgs with teams run very differently. That being said, the on-call sucks. It's really awful, and something I've never seen before. It's also typically the primary cause of team churn for people in my org. This varies, as I've seen other teams stacked with L6 engineers with very little churn (4-8 years of tenure each). This is very much a pit of despair of our own making, but I still haven't figured out how teams like mine get out of it. My own view is that normalization of deviance means that engineers who've only worked at AWS just accept that getting paged many times in a week at awful hours for false positives is OK. There's certainly a view that the only way to get promoted (which is incredibly difficult) is to create new features or products. You read this of many orgs though, not just AWS. I'm not convinced it's fair to single out AWS. It can be endemic in some teams, and I've certainly worked with engineers who are clearly only focused on shining for promotion. The worst I've seen was a team go from 8 engineers with > 2+ yrs tenure down to 4 people with < 6 months, over the course of a couple of months. This was for an enormous product. That team had a very tough 4th quarter. AWS does handle operations failure incredibly well. If you've hopped on a LSE call before, the execution to identify, mitigate and review correction of error (COE) is world class. Doc / design review is also very thorough. There's ample opportunity to learn a lot during your time at AWS, and many engineers have carved out incredible careers in this place. Just go in with your eyes wide open.
  • laegooose 5y ago
    What is LSE?
    • exikyut 5y ago
      Quote-Googling produced "Large Scale Event"
    • kablow 5y ago
      "large scale event" or something to that affect - these issues are visible to everyone company-wide
• kottapar 5y ago
  > We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. wow, honestly this is surprising. For me as an end customer I was always impressed with the way the services are engineered. Kudos to people like you for making this happen. But then I was also under the impression that AWS has very good best practices to take care of repeating issues. Wouldn't interim patch-ups cause stability issues in the long term?
  • hardwaresofton 5y ago
    > Wouldn't interim patch-ups cause stability issues in the long term? No matter what people tell you or put on their marketing blog, it feels like this is the state of play for 99% of software teams. The only time it doesn't end up like this is when you leave time up front to pay down technical debt (like Intel's tick-tock model), and almost no one does that.
    • kazen44 5y ago
      my senior once said to me that the world is held together by two things. Ducttape and shellscripts. This was a while ago, but the older i get the more i tend to agree.
• muststopmyths 5y ago
  >but I know of other teams who would have no issues in taking in a fresh college grad, making them do work for 6-12 months and then just randomly putting them on PIP. stack ranking came up in another thread a few days ago and this practice of forced attrition seems like just another way to do the same thing. I thought it was understood that this kind of structure just incentivizes internecine fighting and politics over and above any imagined positive effects. Mind-boggling that there still exists upper management that thinks otherwise. I mean, early Amazon must have hired a lot of ex-Microsofties with stack-ranking PTSD, being based in the same area. You would think they would know better. It seems like the more employees you have under you, the less you see them as human beings instead of inputs and outputs into a Rube Goldbergian machine you are trying to keep running.
  • epolanski 5y ago
    > It seems like the more employees you have under you, the less you see them as human beings instead of inputs and outputs I mean, there's a department in most companies called "human *resources*" to give away that employees are still resources, just human ones.
    • jjtheblunt 5y ago
      well said
  • ignoramous 5y ago
    > Mind-boggling that there still exists upper management that thinks otherwise. Far too many Harvard MBAs calling the shots, I guess?
    • deleted 5y ago
      [deleted]
    • lupire 5y ago
      I know one Harvard MBA (also an engineer) and she was a brilliant leader of tech and teams.
• awsthro00945 5y ago
  >Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. AWS engineer here and I confirm everything you say, but this quote really struck home with me. The thing I've noticed at Amazon is that not only are the pre-existing conditions awful, but nobody has any interest or willpower to fix it. Everyone will happily vent to you and tell you how awful things are, but any suggestions to fix it or make things more efficient (even if the fix is very simple and requires low effort) will be met with hostility. And I'm not just talking tech issues, but also process/workload issues. I've worked across multiple teams and there is an "institutional ego" at Amazon where everyone, especially L7s+, think that Amazon is the best/smartest company in the world and have an attitude of "if Amazon, the best company ever, hasn't already figured out a way to solve this problem, then it must be an unsolvable problem and we won't even try". The thing is, a lot of these problems are in no way unique to Amazon and many other companies across the world have already found fantastic solutions to reduce things like on-call load. But adopting those solutions would require admitting that other companies were able to solve something that Amazon hasn't, which would hurt the ego. This all applies to the very issue being talked about in the OP, too. Even managers will vent to you about how their team goes through 50% attrition every year, and how everyone is overloaded and finding new engineers is hard. They just accept 50% attrition as "something that just happens every year" as if having such a shitty team is normal, and there is no movement at all to fix it.
  • kablow 5y ago
    > The thing I've noticed at Amazon is that not only are the pre-existing conditions awful, but nobody has any interest or willpower to fix it. Everyone will happily vent to you and tell you how awful things are, but any suggestions to fix it or make things more efficient (even if the fix is very simple and requires low effort) will be met with hostility. And I'm not just talking tech issues, but also process/workload issues. In my case, direct management seems interested in these issues and understand there are problems we need to fix, but ultimately the feature/product launches always make it into the sprint and the larger bug fixes never do. It's very much "actions speak louder than words".
• rantwasp 5y ago
  worked in AWS for a while, but it was 5 years ago. you’re right that your manager can make or break your experience. For the first couple of years there it kinda sucked, mostly due to oncall (our ticket queue was at 3000 tickets at some point) and constantly being yelled at when things broke. We would basically only work on sev2s. Having the pager was super-stressful. We “owned” so much cruft/dead projects/experiments that a couple of time we were paged for something that we didn’t know existed. After I build a little bit of leverage / gathered some political capital I somehow ended up in the position of “team lead” with I guess management’s intent to move me to be a full time manager (after the team’s manager was PIPed). I made a good case that half the team is going to leave, me included, if we don’t do anything (6 people team). So what did we do? I introduced a “secondary” oncall. After you were oncall for a week, you were secondary for a week. While secondary you got the chance to try fixing some shit without worrying about being paged every hour. You were also motivated because you just got offcall. People went for annoyances that would generate a lot of busywork or for… fixing the alerting and monitoring (a lot of autocuts due to wrongly setup alerting thresholds or even alerts that should not have been alerts in the first place). After we exhausted the low hanging fruit, we put some effort into automating some tedious task that would take a lot of time but understood and never meant to be done manually at the scale we did them. Towards the end of the journey we aggressively deprecated/migrated the shit that was not used. By the end of this (took more than one year) we had an empty-ish oncall queue and for the first time in ages people coupd breathe (we now got a sev2 every other week - which in Amazon terms is freaking awesome). I wish this story had a happy ending. There was close to zero recognition for what happened there and most of the team migrated together, internally, to another opportunity after. I left Amazon 6 months after this migration. From what I hear from the people that stayed there, entropy took over and in another 2 years they were roughly in the same shitty place as far as oncall goes.
  • bpicolo 5y ago
    I use reducing the rate of incoming tickets as the primary OKR for on-call engineers. Has never failed to reduce that burden over time. Just like you say - I'm convinced it's the only correct strategy to make the problem go away.
• ako 5y ago
  What is really problematic is that from a management perspective they're doing really great: company is growing and extremely valueable, stock price is doing great, revenue and profits are doing great, and bezos is the richest person in the world. Management has zero incentive to change any of problems you signal, and probably don't see them as an issue. Probably the opposite, they see this as a winning system, and who can blaim them. AWS practices are often used as examples of best practices: you build it, you run it, 2 pizza teams, api first, etc. Survivorship bias and all, the probably regard the other characterestics of the current system as best practices as well.
  • tablespoon 5y ago
    > Management has zero incentive to change any of problems you signal, and probably don't see them as an issue. It sounds like management intentionally created these practices. This is from a recent NY Times article about Amazon warehouse workers, but it wouldn't surprise me if this attitude was applied to all workers in the company to some degree: https://www.nytimes.com/2021/06/15/us/politics/amazon-warehouse-workers.html https://www.nytimes.com/2021/06/15/us/politics/amazon-wareho...: > 5. Many of Amazon’s most contentious policies go back to Jeff Bezos’ original vision. Some of the practices that most frustrate employees — the short-term-employment model, with little opportunity for advancement, and the use of technology to hire, monitor and manage workers — come from Jeff Bezos, Amazon’s founder and chief executive. > He believed that an entrenched work force created a “march to mediocrity,” said David Niekerk, a former long-serving vice president who built the company’s original human resources operations in the warehouses. > Company data showed that most employees became less eager over time, he said, and Mr. Bezos believed that people were inherently lazy. “What he would say is that our nature as humans is to expend as little energy as possible to get what we want or need,” Mr. Niekerk said. That conviction was embedded throughout the business, from the ease of instant ordering to the pervasive use of data to get the most out of employees. In many cases software engineers overestimate their own value to the company, often to the point were they'll ape the attitudes of owners and management (e.g. rejecting the the idea of a union out of hand). But the fact of the matter is they're cogs just like warehouse workers, and in many situations competent management will exploit the hell out of them to extract maximum value for the shareholders.
• kablow 5y ago
  I'm in non-AWS and things aren't any better here. It is very much dependent on the manager and team, and I've been optimistic (~3 years now), but it gets harder as time goes on. > I'm just not 100% sure about the whole PIP scene. Our service was extremely critical and we were extremely understaffed. So I don't think it applied to anyone in our org but I know of other teams who would have no issues in taking in a fresh college grad, making them do work for 6-12 months and then just randomly putting them on PIP. Our team is over-worked and has a large ticket queue, constant sev-2 pages, understaffed, etc. - and yet they still PIP'd (and then fired) someone last year who didn't deserve it IMO.
  • ipaddr 5y ago
    Amazon has a lot of money to waste. It is cheaper to hire someone so they can fire them in a few months to keep the rest of the staff scared enough to overwork themselves so they can understaff. They play a good game.
    • adwn 5y ago
      > Amazon has a lot of money to waste. It is cheaper to hire someone so they can fire them [...] So... Amazon has a lot of money therefore they have to resort to money-saving practices? Or is the causality the other way around? How do these two sentences fit together?
      • rejectedandsad 5y ago
        I don’t think they do, but it’s a way to smear and make all Amazon engineers unemployable by proxy. I already have difficulties finding interviews at good companies because the Amazon name implies IBM level talent instead of Google level talent.
      • ipaddr 5y ago
        Amazon has a lot money. These practices make them additional money. If they didn't have a lot of money they couldn't afford to hire to fire. How can they do both? - They waste money when they hire to fire and constantly onboard. - They make money by not staffing enough resources but by using fear of being fired to force overtime. Maybe the hire to fire costs are justified because the savings they get by understaffing outweights the cost.
• JediPig 5y ago
  Amazon begged to interview, so i did. My friend worked there, and thought I go call her and find out how things worked there. That is when I discovered this forced termination of 30%. I thought it was 15% , but who's counting. So I gave amazon a choice. 2 year contract forced pay contract, if they fired me for any reason, other than fraud, I would collect the full amount. I done this twice. It was a defunct mobile company. I had the contracting company put it in the paperwork, to my surprise they signed. cause they have multiple times let go of contractors to make C level "bonus". It was known for it. I got some push back, but they signed it, they needed an engineer who understood mobile routing records. 3 months into the contract, they apparently forgot, and let me and everyone go. Reason 'services no longer needed' aka CEO wanted to make bonus. I said ok, then called the contractor's legal dept. After 3 days, i got a call back. Told him look at page 4 line 30.. he read it. Long silence. 5 minutes of silence. "I never seen a clause like this. I will call you back with an answer." Next day, Both lawyers call, one asked would i be interested in returning. I said "fufill the contract terms". "Your check will be delivered in two weeks." Contracting firm, was getting paid too. they were happy and mad. Happy for $$$, mad that they probably lost a semi big client. I got my check, had a new contract before the 2 weeks. Fast forword to Amazon. They read it and said no. I declined. They offered little more, but I said I will not work for Amazon until they fix their corporate envirnoment. My friend worked there, she lasted 4 months more, after moving to Seattle, and finding out the hard way that amazon is pure corporate monster.
  • hn_throwaway_99 5y ago
    > So I gave amazon a choice. 2 year contract forced pay contract, if they fired me for any reason, other than fraud, I would collect the full amount. I done this twice. As someone who has hired contractors and full time employees many times, this is just insane. I don't care if you are Linus Torvalds himself, as an employer I would never sign a contract that is that lopsided, and to be honest I can't believe that any competent company would either. And given the description you have given of the company that was dumb enough to sign this, they definitely don't sound very competent. So, I mean I guess congratulations that you got 2 companies to sign this, but I can't think of any decently run company that would ever sign something like that.
• M5x7wI3CmbEem10 5y ago
  how many months of work does it take to get the "stamp"? I'm assuming employers would be wary if it's only 6-12 months
  • salil999 5y ago
    If you're brand new to the industry I'd say stay for at least 2 years. I worked on a service which operates an enormous scale and while I'm thankful for the opportunity it takes several months to ramp up (I'd say about 6-8 months for my team). But obviously YMMV.
• hintymad 5y ago
  What I don’t get about Amazon is that AWS customers have way less oncall load, particularly Netflix, whose engineers could afford oncall 24x7 for weeks or months with perfectly normal work-life balance. Why can’t Amazon, the pioneer of cloud computing, achieve the same level of effectiveness?
  • tylersmith 5y ago
    Maybe I'm misunderstanding you but AWS customer's eningeers have less oncall load because the AWS engineers are doing so much of the difficult and failure-prone work. That's largely what you're buying when paying for cloud services.
    • hintymad 5y ago
      What I meant was that Netflix built their platforms and services on top of AWS services, just like AWS teams. Yet AWS teams have brutal oncalls, while Netflix teams enjoy great work-life balance.
      • aaronbrethorst 5y ago
        Company culture and values flow from the top
      • kaesar14 5y ago
        But that’s because the AWS teams are presumably shouldering the burden, then.
        • hintymad 5y ago
          I’m sure you’re right for some services, especially infra ones, such as EC2. Some other services, though, should be built on top of EC2, EBS, Lambda, S3, and etc, in which case Netflix and AWS teams use the same infra, yet Netflix internal services require much less oncall
          • 5y ago
• SheinhardtWigCo 5y ago
  I don’t get this. In this market, if you’re good, why take a job with any on-call work? It’s thankless, shitty work. If you can deliver features, that’s good enough to get paid extremely well at other Amazon-scale companies with dedicated SRE teams. What’s making you stay?
  • awsthro00945 5y ago
    Because changing jobs is really fucking difficult. I'm probably one of the biggest proponents of "quit your job, you deserve better" that you'll ever find, but even I have to admit that finding a new job is ridiculously hard. Even in "this market", even if you're a top engineer, it's still ridiculously hard to even get an interview, let alone get hired. There is only a limited amount of companies that will pay at the same level as Amazon, and those companies often have months-long interview processes with ridiculous requirements that, even if you are a top engineer, still require a lot of time and effort be set aside to prepare for the specific interview processes that the new company is looking for. And that's to say nothing of the nebulous "culture fit" that is just as likely to prevent you from getting an offer and is completely unaffected by how "good" of an engineer you are. Almost everyone I know at AWS is interested in switching jobs/companies, but it really is not just something you wake up one morning and decide to do. It's a long, perpetual process that can take up a huge amount of time and effort (stuff you don't have a lot of when you're working on-call at AWS anyway), not to mention has huge implications if you are relying on your job for something like a work visa.
    • tolbish 5y ago
      It sounds like the real reason is that the money is worth it.
      • awsthro00945 5y ago
        The money is probably the thing that keeps most people in the job, but it's not like willingness to take a pay cut magically makes jobs fall into your lap, either. It expands the pool of companies you could possibly work for, sure, but everything else above that I said still applies. There's also rarely a guarantee that the new company you're trying to join is significantly better, and companies intentionally make it difficult to get the inside scoop on work-life balance/on-call responsibilities until after you're hired. Personally, I would love to find a new company to join... but my experience interviewing is that it takes months of effort for a potential pay cut and no guarantee that I won't just end up in another shitty situation and paid less, so I struggle deciding if it's actually worth it.
• jjtheblunt 5y ago
  Current Amazon engineer: it's far and away the most incompetently run bureaucracy with self-defeating dysfunctions forced on the huge number of layers. The recent "leaks" to Business Insider, pointed out to me by coworkers, are exactly what we see. Everything the parent comment mentioned is exactly what we see. Laughably, our middle manager berates us as incapable noobs, entirely unaware that some of us at least actually have been very valued and trusted and dependable hard core results driven engineers for decades. It's like a failed brainwashing attempt, and it's embarrassing to be associated with such idiocy. I contrast it with excellent experiences at Sun, Motorola, Apple and others over the past 20+ years, where in some cases I had very high engineering ranks with fabulous results, in very well run and just healthy orgs. I do believe there's intentional treatment of engineers as fungible assets, because the engineering quality is so shoddy and the business plan prioritizes maintaining system uptime, with no true priority given to tech debt removal, which actually would in short order surface measurable benefits.
  • tazjin 5y ago
    > I contrast it with excellent experiences at Sun, Motorola, Apple and others over the past 20+ years, where in some cases I had very high engineering ranks with fabulous results, in very well run and just healthy orgs. I can understand how junior devs end up in these positions for a while and stay on the ride, but what is making someone with your level of experience you stick to Amazon? Is the compensation that good?
    • jjtheblunt 5y ago
      Compensation seems good, but I'm there because I got fooled and plan to quit very soon, when graciously (not screw team-mates) feasible, because it's just a waste of time that could be spent helping positive efforts in healthy orgs.
      • Retric 5y ago
        I have felt that way before, but it’s better just to quit ASAP. It’s the company that’s making your teammates suffer and their also better off just quitting.
• vishnugupta 5y ago
  (Disclaimer: Was with Amazon for ~7 years a long time ago). I've been a customer of AWS across multiple startups and I've seen the overall quality of their products continuously degrade which complements your experience. While they continue to launch new products at a rapid clip you can see small cracks beginning to appear as the products age. A permission issue that's not documented, a cryptic error message etc., They aren't show stoppers on their own but if you use AWS long enough you will be worn down by the cumulative pain.
• MuffinFlavored 5y ago
  > I can safely say that it's totally up to your manager and pre-existing conditions which make up the job Isn't it like that at any job?
• gautamnarula 5y ago
  This jives with what I observed as an intern some on an AWS team some years ago. The oncall rotations seemed absolutely brutal and the engineers were so busy and stressed out fighting fires that they barely noticed my existence (which I was okay with) and the tech debt kept accumulating between because between new feature launches and firefighting there wasn't much scope for anything else. My intern project was a fairly no brainer tech debt item that automated a lot of the deployment process and saved our lead engineer several hours a week in babysitting deploys. I resolved to never work on a cloud infra team after that -- while the internship was fine, being a full time engineer seemed absolutely miserable.
• frozenport 5y ago
  Was it Amazon US or Amazon India?
• smlss_sftwr 5y ago
  Had a relatively similar experience (finally understood what drinking out of a firehose meant), the silver lining of the experience though is it finally dispelled any notion of imposter syndrome when I realized everyone was running around as much of a headless chicken as myself ahah
• walshemj 5y ago
  Very weird if you do get called out do you not take TOIL? and get paid on call allowances? Having this cost exposed will lead to things getting fixed. This solution works based on knowing developers on CSS the huge mainframe billing system and my own experience as the lead for a smaller BT billing system I worked on. And TBH a 1-2 kloc method isn't that big I have seen individual SPROCS over 2.5k
• amzn-throw 5y ago
  Piggy-backing on the top-most-voted comment for those that will only read this one and not go to page 2 or onward. Drama gets heavily upvoted. If you keep scrolling you will find many individuals whose experience did NOT match this person (or OP's), who have a good work life balance, solve challenging tasks, and think this is the best job they've ever had. I'm one of those people. I've been at Amazon for almost 10 years, worked on a half dozen teams. Sure, there were outliers and a couple bad managers I've worked with, and some folks who left the company on bad terms. But by and large, the experiences I had in 2 different countries on 2 different continents were mostly positive. Yes, my job is tough. Yes, the oncall can be challenging sometimes too. But I've also never stopped growing in the last decade, and have achieved bigger things than I ever thought possible. Every single product and feature I've shipped have been immensely successful because of Amazon's deeply ingrained "Working backwards from the customer culture". And since that's what drives me - shipping cool stuff and having users on Twitter go crazy for it, I have never been more fulfilled. Amazon is huge. There are now >60,000 Software Development Engineers worldwide. I'm not in Seattle but I'm in one of the largest offices outside of it. So I'm sure every single horror story is true. But even so, they are a minute exception, and not most people's rule.
  • andreilys 5y ago
    Why is it that amongst all the tech Co's, Amazon seems to have the highest proportion of horror stories? It's almost a meme at this point. I mean the forced attrition and PIP culture is a known thing (x% of employees must be let go each year). Google/Apple/etc. do not have forced attrition or a known PIP culture. Bezos is on the record for having said "humans are lazy and we need to give them incentives to quit". No other Tech CEO has said that. So perhaps your story is the anomaly?
    • dan-robertson 5y ago
      I find it easy to believe that Amazon is a shit place to work (as a SWE) and that Amazon is a massive company with different bits being different. But I also find it easy to believe that a meme could just be wrong. It’s also a meme that Google has the best engineering and that their products get shut down after a year and that it sucks to be their customer. If everyone at Google is convinced at orientation that Google is some great company with great engineering then that can create a brand outside the company by the force of the size of their workforce. Maybe Amazon just has a different employer brand. Don’t Netflix also hire relatively many employees also? Perhaps I’m just misremembering. And wasn’t stack ranking and the like quite popular say 20 years ago? It isn’t obvious that the Google or Facebook of Apple or Amazon way is the best way to manage a business. Lots of people online have complaints about each. And I’m not even particularly put off by your Jeff Bezos quote: it doesn’t need to be a bad thing for someone to have an incentive to quit something they are not suited to as they could then go on to something they are better suited to, and it’s true that a lot of people (even well-paid with alternatives) will spend a long time going to a job they dislike because it is easier to stay with the status quo than to leap. I still feel like the hire-to-fire stories are directionally right (and so are eg stories about shipping new products being the only way to get promotions at Google leading to warped incentives)
• fukmbas 5y ago
  You can blame agile and product development on that. It's always build build build and not maintain or improve. Agile manifesto literally says to ignore bugs and keep developing.
• swman 5y ago
  So many of my FANG friends talk about the horrid code quality and sometimes terrible processes that are in place that I’m always surprised. Considering the amount of gatekeeping