4 ms·
You're using S3 for the first time. Twilio has the capability to hire Ops staff that have used S3 for 10+ years. Throw a penny down the street in any major city
by mjayhn 6y ago
You're using S3 for the first time. Twilio has the capability to hire Ops staff that have used S3 for 10+ years. Throw a penny down the street in any major city and you'll hit a DevOps person with AWS experience. It's amazing to me how often I hear a company not set up s3 properly, or not enable budget alerts on AWS, etc. This is all day 1 AWS stuff. If that means taking away the ability for your developers to even create buckets then do so. A lot of you should just focus on development work and stay far away from operations/sec tasks, and that's fine. I'm going to assume most of these buckets aren't even made via code and are manual. That's bad enough.
I understand someone brand new to S3 being in the dark but someone in your PRs approval chain should have enough Ops skill to know to check these things if you're not going to hire an actual Ops person. And no, none of this is hard for anyone who has ever touched S3. Ops teams have known about these issues for over a decade. It's practically a meme at this point.
- somehnguy 6y agoI understand what you're saying but you do realize that a huge number of software shops don't have massive teams or even dedicated OPs people right? The number of heads on my project is ...4. We're it. There is no place or person to just push the work off to. There is no PR approval chain. We regularly meet to sync and discuss plans and thats it. Until very recently all of our infrastructure was in house, right down the hall from me. I (or whoever was in that day) took care of making sure it was running smoothly. Cloud is new to us, and there is no budget to hire a guy who manages AWS full time. That is the case at an insanely high percentage of software shops. Heck I would wager most people outside of 'new tech (sorry, I can't think of a better term)' shops are using point & click in the web UI to manage AWS. There is no code in source control that manages our infrastructure. I'm jealous of the people that get to do it all the 'right way' but for many there isn't enough manpower or time in the day to do it. Now why we're in this position in the first place is another debate about why management thinks we need to go cloud at all. But its reality for many.
- SteveNuts 6y agoTo me this is like buying a high end sports car, and neglecting the maintenance because you never factored in the expensive maintenance, and therefore the true cost of what you're buying.
- somehnguy 6y agoI agree. People do it all the time though.
- mjayhn 6y agoOh trust me I know totally what you mean, I'm usually the first ops person hired at a startup once they've gotten the budget for one. My goal when I work with devs is to get as much of the infrastructure frustration out of their way as possible (unless they want to help with it) and automate the processes that slow down their code from getting deployed. This usually comes with months of fixing cruft/security issues, etc that could've lead to a "hack" like this when just the devs were running the ship. I wish MORE small dev teams brought in ops people and didn't wait until they had a staff of 20-30 or whatever it typically winds up being. But I know that's a pipe dream and you're basically losing a developer salary to hire a devops/ops person which isn't the direction they want to go. edit: typos
- aldarisbm 6y agoI totally understand where this is coming from, but even if we factor AWS out. IAM _IS_ difficult to implement, there really isn't a way to make it "simple" while still make it extremely granular. You can't have without the other.