8 ms·

Worked for AWS for just under a year and a half; this mirrors my experience exactly. Poorly designed, failure-prone, brittle internal tooling was a time and en

• dimmke 4y ago
  >I'd actually force myself to write even trivial code on weekends to ensure my ability to develop didn't atrophy as a result of lack of use. I did not mention this in my blog post, but I have actually done similar. That's incredible. Thanks for sharing this comment. Our tenures were almost the exact same length too.
  • scruple 4y ago
    When I worked for a BigTechCo, I was doing exercism every single morning for ~30 minutes for the same reason. It really is incredible. I know a few of my former colleagues were doing the same. Today I'm on a team with 4 other engineers, so I keep busy coding.
    • arcturus17 4y ago
      I cannot code in prod anymore and have definitely been looking to develop a habit like this. Any tips from you and others on: a) How to make it stick b) How to maximize its efficacy/efficiency Would be awesome.
      • nso95 4y ago
        There are no tricks, that's just a distraction. Just sit down and do it. Ass in chair gets you there.
        • avip 4y ago
          Thanks for this. This is the right answer to 95% of "how do I" questions.
    • pbronez 4y ago
      Thanks for the tip - Exercism looks great. https://exercism.org/ https://exercism.org/
      • silisili 4y ago
        Thanks for the link. I initially thought it was a misspelling of exorcism, and OP was making a joke.
        • menaerus 4y ago
          Me too and I was like wtf ... Made my day
• buremba 4y ago
  I thought that the deployment pipelines are part of their core business considering AWS has such products that are meant to be used by customers. I assume that the internal tools are different than what AWS customers use for deployments?
  • dimmke 4y ago
    A lot of internal teams don't use the AWS software for various reasons. So like, CodePipeline might be great, but there's an internal analogue (I'm not sure the detail I can go into here) that is awful that a lot of teams use. There's been an internal movement to try to get all teams onto AWS services, but it's incredibly slow moving and there's no timeline that I know of.
    • fdgsdfogijq 4y ago
    • CharlieDigital 4y ago
      > So like, CodePipeline might be great The entire CodeBuild suite kind of sucks when you stack it up to options available on the market including GitHub Actions, GitLab Runners, Azure DevOps, literally almost anything. If the internal analogue is worse than the CodeStar suite, then yikes.
      • banana-19 4y ago
        Codestar is simpler than and has too many feature gaps compared to the internal offerings, which makes building and deploying Amazon scale multi-tier, multi-environment services difficult.
    • bb88 4y ago
      Clearly a mistake. They should be dogfooding their offerings. A lot of tooling will probably just be better if they can host it on AWS.
      • 4y ago
• rad_gruchalski 4y ago
  > Poorly designed, failure-prone, brittle internal tooling was a time and energy sink to the point where even the most trivial deployment change was a nail-biter. Considering rather good quality APIs they expose to the external world, this is shocking to read.
  • sbarre 4y ago
    Internal tools never get the same attention as public-facing products. Public facing products have competition. Internal tools have a captive audience that must use them, no matter what.
  • krageon 4y ago
    Good is a big word, but they exist and have docs. A lot of the time the docs are even correct, if you can find them.
    • rad_gruchalski 4y ago
      I have most often interacted with their APIs via the golang library. Compared to Azure, the AWS stuff is pretty solid.
      • krageon 4y ago
        Everyone says this, but in terms of discoverability of the actual apis I have found the opposite is true. Azure tends to have docs with small mistakes, but they are extremely thorough. With AWS it seems to be a tossup: You can be lucky and have complete and accurate information, or you can be fucked.
• fdgsdfogijq 4y ago
  You should have just switched teams. Amazon has amazing teams working on deep technology, they also have huge systems with technical debt and stress. People dont realize how different it can be inside the company.
  • mr_tristan 4y ago
    How easy is it to transfer between teams? Some companies make this easy, others make it almost harder to apply internally then externally.
    • goostavos 4y ago
      I've been on 4 different teams at Amazon (now migrating to a 5th). It depends entirely on the team and as well as your current level. Some do loops just like with external candidates. Others barely require more than a conversation with someone on the team -- though this one is a huge red flag. Teams looking for warm bodies are seldom good teams. Most of the time its somewhere in between. You'll speak with the manager, speak with the team, you each review each other's artifacts, and if everyone's happy, you swap over. The balance of power is pretty nice. I've dropped out of the process many, many times after starting talking with management. Or even just learning more about what the team itself does (I couldn't live with myself if I worked on pre-roll ads, for example. So I noped out of that one).
      • gruffle 4y ago
        I believe you also need approvals from your current manager(s) to switch teams?
        • ldjkfkdsjnv 4y ago
          Nope no approval required
          • gruffle 4y ago
            Interesting, I was told the opposite by my managers...
• dfxm12 4y ago
  Poorly designed, failure-prone, brittle internal tooling was a time and energy sink Sometimes I get frustrated at my own job for some reason or another, but I appreciate learning that the grass isn't really greener anywhere else & even places where tech is the core business are having a lot of the same problems.
  • nso95 4y ago
    Well, it's definitely not greener at Amazon
    • Firmwarrior 4y ago
      It's green from all the cash lying around, but it is a blasted hellscape
• chrsw 4y ago
  >I'd actually force myself to write even trivial code on weekends to ensure my ability to develop didn't atrophy as a result of lack of use. I figured this was pretty common. I've only had one job where I didn't feel the need to do this.
• gruffle 4y ago
  Completely agree. Interesting how the internal dev satisfaction polling always shows that the vast majority of devs just absolutely love the internal tooling and processes. Might have something to do with the polls not actually being anonymous. When hr/polling teams are asked about anonymity they generally skirt or ignore the question, but I've learned that enough metadata is collected to identify any responder in pretty much any internal poll.
  • deleted 4y ago
    [deleted]
  • Aeolun 4y ago
    Why would you respond positively even on non-anonymous polls? The goal is to determine the state of the internal tools, not to purge any dissenters…
    • gruffle 4y ago
      There is very little transparency as to how the polling data is used. I'm sure many employees don't feel comfortable giving honest answers on many of the questions. Also the surveys go far beyond tooling question - they ask about future career plans, happiness with the company, happiness with management, whether employees are interviewing for other jobs, etc. It's not hard to see how certain answers to those questions could color the responder in a very bad light from managements perspective.
    • glogla 4y ago
      > The goal is to determine the state of the internal tools, not to purge any dissenters… Well ... maybe?
      • Aeolun 4y ago
        I suppose? I’ve always answered honestly and never been purged so far. N1 doesn’t really help, but it’s a start.
        • 4y ago
• gw98 4y ago
  This is the general feeling of what I see from AWS support as well as a consumer of the platform. I'm battling something for weeks now which is a completely broken ass piece of shit inside AWS. I spent the first 3 weeks trying to get attention for this via support tickets and someone to take it seriously, resulting in escalating to account management. I've been on calls with the programme managers, been apologised to constantly and absolutely nothing has been done that is productive. Not only that they sheepishly suggested other customers were in the same shit. My conclusion at the end is that critical bits of AWS are held together with only a couple of people who actually know how it works and they don't even have the ability to do fix anything... Not that the consuming company I work for isn't a complete mess either but I expected better from AWS.
  • horns4lyfe 4y ago
    I know the types of companies that are Microsoft shops come with their own problems, but my quality of life via tooling at two Microsoft shops has been great.
    • throwaway2037 4y ago
      Do you mean Azure? I don't quite follow your comment.
      • wikfwikf 4y ago
        I think they mean places which use Visual Studio, (usually) C#, Windows Server, Azure, etc. Lots of components of this stack tend to work a bit better if you use all the others. Common in finance, because of Excel.
  • lloydatkinson 4y ago
    This is my experience of using AWS too at work. It’s why I strongly prefer Azure. You can tell AWS is totally drowning in poor management.
• bitL 4y ago
  You are pretty much validating that Amazon is a 2-year company, as reflected in their vesting schedule.
  • randomopining 4y ago
    So basically you get most of your comp at 2 years? So crank hard for 2 years and then bounce?
    • spike021 4y ago
      You get a bunch of cash up front; majority of RSU's come from your 3rd year vest date on.
    • marssaxman 4y ago
      You get most of your RSU comp after two years; vesting is back-loaded. Many (most?) people wash out before they collect, which obviously works out well for Amazon.
  • grangerg 4y ago
    Someone on another Amazon related thread mentioned that 50% leave after 2 years and 80% leave after 4. If you do the math on their TC to see how it actually works out from month to month, it makes sense. Unless you get more RSU's, you take a huge hit in pay, unless you weren't cashing in on them to make ends meet.
    • oblio 4y ago
      That's accounting for stock growth. With the flat share price, there's no 4 year cliff anymore...
• giantg2 4y ago
  "Generally a very unpleasant experience all around, sans the compensation and name on the resume." At least you have that. My skills have atrophied and my pay is meh.
• hinkley 4y ago
  I worked a contract there and the team barely understood how some of those configuration panes functioned. It was always a case of staring at someone else's config and then at yours until you found enlightenment. And the target kept changing, so there was a very bad week where my computer worked for 80 minutes total. I coded like hell during that time, pair programmed with the other contractors or did math on a whiteboard for the rest. Another guy on the team intimated once in a while that he didn't like the team dynamics, but he played his cards pretty close to the chest. After he gave notice, on what we all thought was supposed to be his last day, we showed up in the morning just to find his computer and all of his equipment in a pile on a desk. We never did figure out if we or he were off by one or it was his middle finger to the team. All the more mysterious because later that day we did discover his middle finger: The lead dev, who seemed to be one of the more reasonable people on the team, and the resident know-it-all both discovered that someone had fiddled with their configurations in ways that broke the system but produced obscure errors and hard to notice differences (like punctuation). I took a couple lessons away from that project, and reinforced some that I'd already had. Perhaps the most important of which is: Bragging rights for being first mover are very, very short-lived. Once everyone else has copied it, the most accurate adjective for your version is 'old', or 'primitive', not 'first', and absolutely under no circumstances 'best'. Developers don't like bullshit. Don't pretend like your liabilities are assets.
  • Firmwarrior 4y ago
    Man, that's crazy. You can go to big boy jail for that, haha. When I quit Amazon, they ran to my desk and unplugged my dev machine, since they thought I was going to leave a software "time bomb". I thought that was the stupidest concern in the world, but maybe it does actually happen.
    • hinkley 4y ago
      It was maybe his second gig. A lot of us are still little monsters at that point. Not to say that’s okay, but just to say I wasn’t that surprised. What I was surprised by was that I didn’t think he had that kind of animosity. Pretty good at hiding his feelings. I never understood that “shut them out” reaction for people giving notice. Getting laid off, maybe. But I pick the time and place where I hand over my 2 weeks. I have all day or even a few days to do it, but I have known for a month or three that this was coming, I’ve known for days when it was likely to come, and I’ve known for hours or even days that it was happening today. You don’t think if I was going to do something shitty that I would have done it first? You guys are really underestimating my intelligence, or at least my ability to plan. But then that’s probably part of why I’m leaving. Also they say that liars think everyone else is lying, thieves think everyone else is stealing. So I wonder what this says about the paranoid person? Are they a danger to the company?
• deleted 4y ago
  [deleted]
• sumanthvepa 4y ago
  I worked at Amazon from 1998 to 2003. Those were exactly my experiences. Clearly they don't care about the internal development experience and probably never will.
• PartiallyTyped 4y ago
  > I'd actually force myself to write even trivial code on weekends to ensure my ability to develop didn't atrophy as a result of lack of use. I am glad I am doing a master's in parallel because outside that I haven't written code in a while ...
• chocolatemario 4y ago
  My Amazon experience was similar. Especially the lack of coding due to the emphasis on devops/meeting culture. I have since felt that once a product reaches maturity, additional headcount (butts in seats) are operators first and secondly a swarm for something like a broadly scoped breaking change, runtime upgrade or platform migration. There is plenty of fat to trim even in aws despite it being among the more secure jobs to have at the company. I’ve seen work slow down across a few teams and it just leaves me in a place of boredom and complacency with a major technical itch. Despite this itch looking to be scratched, I cannot seem to find the time/energy to pursue coding regularly outside of work in the limited free time I have. Maybe if I were single, no kids, dogs…