5 ms·

I doubt that. I'm somewhat familiar with AWS offerings and have used S3 for years. Actually creating a scalable SAAS offering correctly (emphasis on "correctl

by watty 9y ago

I doubt that. I'm somewhat familiar with AWS offerings and have used S3 for years. Actually creating a scalable SAAS offering correctly (emphasis on "correctly") is still a daunting task. Sure, I just created a CodeStar project which I guess hooked me into 5 or more of AWS services but now what? What if I want to create a DynamoDB or move off of CodeCommit? How do I know if it scales or is redundant? How much does this beast I just created in a few clicks cost / month - can it be cheaper? etc. etc. It may make DevOps jobs easier or help people get into DevOps but it certainly doesn't replace a trained AWS architect.

RussianCow 9y ago
Genuine question: For how many small and medium-sized businesses does any of this stuff matter? Not that I necessarily disagree with you, but it seems like having one engineer who knows AWS decently well is Good Enough™ for most companies, whereas in the pre-AWS world, you needed ops people who could build most of that stuff from scratch. It just doesn't feel like you need nearly the same amount of people to scale something up with AWS as opposed to doing it in-house. But I could be wrong as I don't follow the devops world closely.
- holydude 9y ago
  It depends on what do you call small or medium-sized business. Besides the same question can be asked about "developers". How many mediocre / junior / just developers you need "in-sourced" ? More and more work can either be outsourced to services like AWS is building or cheaper IT professionals anywhere in the world. The only question here is when taking the lead outweighs the costs.
- watty 9y ago
  In my niche commercial industry almost all software is server-side software. Only now is everyone moving to the cloud - mostly as a way to collect data and generate some recurring revenue. So small businesses with a few devs now need to either spin someone up on AWS or hire someone. I could definitely see less ops engineers needed in some companies that move to AWS from some in-house platform but overall "devops" is rapidly increasing in demand.
- nulagrithom 9y ago
  I'd look at it almost like Wix and other WYSIWYG editors: it's perfectly fine for smaller stuff. Small to medium doesn't really need to flesh this stuff out. I could use the hell out of CodeStar currently. I'm doing multiple small projects and this shop is doing the "ops people" thing, so there's a ton of friction between code, test; deploy. But on my last project (much bigger) Elastic Beanstalk started to really show its rough edges, and I was pretty close to tearing it down and setting up by hand, which gets in to becoming the guy who knows AWS Good Enough™. But I could've easily seen that project scaling way up and being tempted to go the Kubernetes route, which would've required a true DevOps person to maintain. Just based on my anecdotal experience, I don't think either of the previous projects were paying for the DevOps person in the first place, and I don't think CodeStar is going to replace them now either. This is just low-hanging fruit.
  - autotune 9y ago
    Just out of curiosity, what are some of those "rough edges" you experienced?
    - nulagrithom 9y ago
      Not sure you'll see this and I'm kind of stretching my memory here but... It was a node.js project and the db was separate, and keep in mind my knowledge of AWS is/was limited. But then, this seems to be the target audience for EB, yes? I'm sure a lot of this could be fixed but... 1) Logging It somehow took me significant time to figure out how I could send meaningful logs anywhere but poorly organized flat text files in S3 without adding some kind of logging sink to the code itself. Sure, I could've just plugged in Winston and spat out everything to MongoDB but the team couldn't understand why we couldn't just get logs out of EB. And yeah, we could could grep through the S3 bucket but that meant syncing the whole bucket and trawling through an absolute ton of log files with no more precision than your knowledge of grep and regex. Not the best thing. I eventually dumped all the logs in to CloudWatch, but the documentation to do so was severely lacking (from my perspective) that dealing with CloudFormation template files made me consider ditching EB entirely -- If I'm dealing with this garbage anyway why don't I just go whole hog and get more control anyway? Even after dumping the logs to CloudWatch, I was still "the guy" for doing any good searching of these logs. Maybe this was just a team quirk, but what they really wanted was logs in MongoDB or something they know, rather than something AWS specific. Again, a logging sink would've fixed this, but there were concerns about uncaught exceptions and attempting to send to MongoDB in an unknown state. 2) Deployment Time & "Health Checks" The team was used to pulling the latest changes manually and restarting the node.js process. Introducing CI ironically increased friction as our deploys were taking an upwards of 10 minutes (20 if immutable). This doesn't sound like much of a problem in a traditional environment, but going from a "cowboy coder" environment to "why the hell does it take 20 minutes to deploy!?" was a stretch. My initial answer was "health checks!" but then we discovered that the health checks weren't actually doing much for us... I had erroneously assumed that part of the health check meant monitoring the responses of the web service and aborting deployment upon massive failure. We had a deploy that crashed and returned 500 on every request, yet somehow didn't abort. Part of this may have been because the developer involved in the bug did everything possible to prevent a "crash", instead of letting the process die upon unrecoverable error (part of a frontend JavaScrtip mindset I assume), but trying to sort out how to make the health check look for 100% 500's brought me even further in to "why not set it up myself?" 3) Deployment Failure We hit a hard limit of deployments and had to clean out a list of artifacts we didn't even know existed before being allowed to deploy again. WTF? This is when the team started to really question the value EB was providing. 4) I ended up "the guy" anyway, so what does it matter? The documentation was too dense for my team, and the quirks were plentiful enough that I ended up "the guy" anyway. One day I came in late and a deployment had gone wrong. EB had been rolling out bad code for 20 minutes and they didn't know how to fix it. I had to clean up the mess. That was really the "fuckit" moment for me. Hand managed EC2 instances wouldn't have caused this problem. Regardless of knowledge problems within the team itself, the whole point of adopting EB was to avoid needing infrastructure specific knowledge to scale & deploy. Now EB was doing more harm than good. --- These are just the things I remember off-hand. I'm sure there's solutions to these problems, but it just wasn't worth it for me to investigate when really my purpose was just to write and ship code, not futz with the infrastructure. We would've run in to less problems with a naive, manual setup in the end. You can ping me at rtannerf dot com if you want to know more. I'm sure I can dredge up a few more annoyances and clarification from an old coworker. Hope that helps.
- unclebucknasty 9y ago
  >It just doesn't feel like you need nearly the same amount of people to scale something up with AWS as opposed to doing it in-house. If this statement is not true, then Amazon would say that AWS is failing to fulfill its mission.
- _delirium 9y ago
  For the small businesses that I know of, one engineer who knows the operational side decently well has been the norm for a long time. In the '90s it was one of the programmers who happened to know enough Solaris to double as the Unix Sysadmin. Maybe one genuine full-time sysadmin if the company grew a little larger. Now you have someone who knows enough AWS to fill that role, which doesn't seem all that different in staffing levels.
BurningFrog 9y ago
Note how OP says DevOps will become obsolete in the future, while you say it is not obsolete right now.
chuparkoff 9y ago
Agree with watty here that this, as with most new Legos from AWS, makes some of the repetitive and frequent devops things easier and consistent. But the devops-person's true value is in stringing all the Legos into complex creations. And sure, while each new AWS offering, simplifies or automates some segment of work, it also feeds the number of cross-tool permutations. And that in turn creates new opportunities for the people like us that can string it all together when needed.
MR4D 9y ago
Give it a few years. Amazon has shown they can scale as big as anyone, and this is just version 1. Bezos is maniacal about winning, and I'm sure this will win where they want it to win.
deleted 9y ago
[deleted]

by watty 9y ago

RussianCow 9y ago

Genuine question: For how many small and medium-sized businesses does any of this stuff matter? Not that I necessarily disagree with you, but it seems like having one engineer who knows AWS decently well is Good Enough™ for most companies, whereas in the pre-AWS world, you needed ops people who could build most of that stuff from scratch. It just doesn't feel like you need nearly the same amount of people to scale something up with AWS as opposed to doing it in-house. But I could be wrong as I don't follow the devops world closely.

holydude 9y ago
It depends on what do you call small or medium-sized business. Besides the same question can be asked about "developers". How many mediocre / junior / just developers you need "in-sourced" ? More and more work can either be outsourced to services like AWS is building or cheaper IT professionals anywhere in the world. The only question here is when taking the lead outweighs the costs.
watty 9y ago
In my niche commercial industry almost all software is server-side software. Only now is everyone moving to the cloud - mostly as a way to collect data and generate some recurring revenue. So small businesses with a few devs now need to either spin someone up on AWS or hire someone. I could definitely see less ops engineers needed in some companies that move to AWS from some in-house platform but overall "devops" is rapidly increasing in demand.
nulagrithom 9y ago
I'd look at it almost like Wix and other WYSIWYG editors: it's perfectly fine for smaller stuff. Small to medium doesn't really need to flesh this stuff out. I could use the hell out of CodeStar currently. I'm doing multiple small projects and this shop is doing the "ops people" thing, so there's a ton of friction between code, test; deploy. But on my last project (much bigger) Elastic Beanstalk started to really show its rough edges, and I was pretty close to tearing it down and setting up by hand, which gets in to becoming the guy who knows AWS Good Enough™. But I could've easily seen that project scaling way up and being tempted to go the Kubernetes route, which would've required a true DevOps person to maintain. Just based on my anecdotal experience, I don't think either of the previous projects were paying for the DevOps person in the first place, and I don't think CodeStar is going to replace them now either. This is just low-hanging fruit.
- autotune 9y ago
  Just out of curiosity, what are some of those "rough edges" you experienced?
  - nulagrithom 9y ago
    Not sure you'll see this and I'm kind of stretching my memory here but... It was a node.js project and the db was separate, and keep in mind my knowledge of AWS is/was limited. But then, this seems to be the target audience for EB, yes? I'm sure a lot of this could be fixed but... 1) Logging It somehow took me significant time to figure out how I could send meaningful logs anywhere but poorly organized flat text files in S3 without adding some kind of logging sink to the code itself. Sure, I could've just plugged in Winston and spat out everything to MongoDB but the team couldn't understand why we couldn't just get logs out of EB. And yeah, we could could grep through the S3 bucket but that meant syncing the whole bucket and trawling through an absolute ton of log files with no more precision than your knowledge of grep and regex. Not the best thing. I eventually dumped all the logs in to CloudWatch, but the documentation to do so was severely lacking (from my perspective) that dealing with CloudFormation template files made me consider ditching EB entirely -- If I'm dealing with this garbage anyway why don't I just go whole hog and get more control anyway? Even after dumping the logs to CloudWatch, I was still "the guy" for doing any good searching of these logs. Maybe this was just a team quirk, but what they really wanted was logs in MongoDB or something they know, rather than something AWS specific. Again, a logging sink would've fixed this, but there were concerns about uncaught exceptions and attempting to send to MongoDB in an unknown state. 2) Deployment Time & "Health Checks" The team was used to pulling the latest changes manually and restarting the node.js process. Introducing CI ironically increased friction as our deploys were taking an upwards of 10 minutes (20 if immutable). This doesn't sound like much of a problem in a traditional environment, but going from a "cowboy coder" environment to "why the hell does it take 20 minutes to deploy!?" was a stretch. My initial answer was "health checks!" but then we discovered that the health checks weren't actually doing much for us... I had erroneously assumed that part of the health check meant monitoring the responses of the web service and aborting deployment upon massive failure. We had a deploy that crashed and returned 500 on every request, yet somehow didn't abort. Part of this may have been because the developer involved in the bug did everything possible to prevent a "crash", instead of letting the process die upon unrecoverable error (part of a frontend JavaScrtip mindset I assume), but trying to sort out how to make the health check look for 100% 500's brought me even further in to "why not set it up myself?" 3) Deployment Failure We hit a hard limit of deployments and had to clean out a list of artifacts we didn't even know existed before being allowed to deploy again. WTF? This is when the team started to really question the value EB was providing. 4) I ended up "the guy" anyway, so what does it matter? The documentation was too dense for my team, and the quirks were plentiful enough that I ended up "the guy" anyway. One day I came in late and a deployment had gone wrong. EB had been rolling out bad code for 20 minutes and they didn't know how to fix it. I had to clean up the mess. That was really the "fuckit" moment for me. Hand managed EC2 instances wouldn't have caused this problem. Regardless of knowledge problems within the team itself, the whole point of adopting EB was to avoid needing infrastructure specific knowledge to scale & deploy. Now EB was doing more harm than good. --- These are just the things I remember off-hand. I'm sure there's solutions to these problems, but it just wasn't worth it for me to investigate when really my purpose was just to write and ship code, not futz with the infrastructure. We would've run in to less problems with a naive, manual setup in the end. You can ping me at rtannerf dot com if you want to know more. I'm sure I can dredge up a few more annoyances and clarification from an old coworker. Hope that helps.
unclebucknasty 9y ago
>It just doesn't feel like you need nearly the same amount of people to scale something up with AWS as opposed to doing it in-house. If this statement is not true, then Amazon would say that AWS is failing to fulfill its mission.
_delirium 9y ago
For the small businesses that I know of, one engineer who knows the operational side decently well has been the norm for a long time. In the '90s it was one of the programmers who happened to know enough Solaris to double as the Unix Sysadmin. Maybe one genuine full-time sysadmin if the company grew a little larger. Now you have someone who knows enough AWS to fill that role, which doesn't seem all that different in staffing levels.

BurningFrog 9y ago

Note how OP says DevOps will become obsolete in the future, while you say it is not obsolete right now.

chuparkoff 9y ago

Agree with watty here that this, as with most new Legos from AWS, makes some of the repetitive and frequent devops things easier and consistent. But the devops-person's true value is in stringing all the Legos into complex creations. And sure, while each new AWS offering, simplifies or automates some segment of work, it also feeds the number of cross-tool permutations. And that in turn creates new opportunities for the people like us that can string it all together when needed.

MR4D 9y ago

Give it a few years. Amazon has shown they can scale as big as anyone, and this is just version 1. Bezos is maniacal about winning, and I'm sure this will win where they want it to win.

deleted 9y ago

[deleted]