People are really good at avoiding the pit of success! We run most of our infra on k8s, and if you want a cert, with ACME & auto-renew all managed automatically

People are really good at avoiding the pit of success! We run most of our infra on k8s, and if you want a cert, with ACME & auto-renew all managed automatically for you, you just create a Certificate object. But then we get some vendored product that manages to be completely unable to run in k8s, devs avoid the automation for $reasons, etc. > Does your company have a Wiki page saying how to do it wrong? Sometimes we do! I've found a few of these after really pressing the point of "why are you doing it this way?" hard enough. But you have to a.) get an answer to that and b.) the answer has to reveal they followed some shadow docs. > If you have firewall rules or a proxy, block random unauthorised stuff. Your rouge service implementer just creates their own VPC; they are in control of the firewall. Should my security team either set appropriate privileges or delegate that to my team? Perhaps. I have to get them to adopt like RBAC and ABAC first; they fervently believe our industry's regulations forbid "job function begets access" (i.e., RBAC) type policies. (They desire that, even if a job function begets access, that if you're not needing to exercise that access, it should be revoked until such a time that it is required to be exercised. But this means that you end up with "all security reqs. must flow through security team" style thing, and there are then a lot of them (because they are so ephemeral) and so any process must inherently be ignorant of whether the request is right. So your rouge implementor's request for "I need to implement $high level service" is basically carte blanche. The thing about shadow-docs and shadow-services is that they're hard to find out about in a timely manner. A lot of these comments are fighting the very core of human nature. (We used to be better about this as a company, back when we were very engineer heavy, and filled with good engineers — most better than me. The quality bar definitely fell at some point, and we've hired a lot of not engineers doing things that really would be better served by an engineer. Y'all are working at rainbow companies, and I don't know how to keep a company in that state or move it to that state as a bottom-of-rungs eng.)