2 ms·
> Can they learn what to monitor, at what frequency? Yes. That is part of the design of any system; observability is in everyone's mind when they are writing
by jrockway 7y ago
> Can they learn what to monitor, at what frequency?
Yes. That is part of the design of any system; observability is in everyone's mind when they are writing software. Software that fails mysteriously means less weekends and less time for drinking beer. Software that tells you exactly what it's doing means when something goes wrong, you fix it, and it never happens again. Or, it's so transparent you figure out the bug before you even check it in.
> Security... 'Principle of least privilege' is at odds with "make it work".
There is certainly value in having strong security principles, both in terms of policies and infrastructure. Infrastructure is key here; an ACL-checking proxy in front of all services, mTLS out of the box, the ability to write, deploy, and manage small services that are easy to audit... all help make security the rule instead of the exception. This is what all those "service meshes" and "policy frameworks" and "orchestration frameworks" aim to provide.
> I've seen devs assume the database server has commodity connection to the internet. They never asked if it did or didn't.
That's why the dev team should be operating that service.
> And I've seen devs go 'just spin up some more machines'. [...] You'd be surprised just how angsty the C levels are with this cloud computing and developers with near-free reign.
In my experience this has always been a utilization problem. One shared cluster for everyone is too complicated / expensive / dangerous / conflicts with empire-building, so every team has a bunch of t3.4xlarge instances that average 0% CPU load just because. Meanwhile, companies that care about their computers have seemingly-complicated orchestration frameworks so that they can extract value from every CPU core.
The good news is, a lot of what used to be manual (wrangling machines, consolidating resources, mechanical aspects of security) has some serious open-source investment. People don't need to schedule programs to physica computers anymore. People don't need to write authentication and mTLS code into their application anymore. You don't need to fill out a form and create a purchase order to get a TLS certificate anymore. Times are changing.
Your experience seems to have been shaped by supporting bad developers. We aren't all like that, though, and you'll find that there is less and less room for the incompetent these days. It's true of any field; someone builds you a house and the floor isn't level. "Don't build houses anymore," you say! No! Just don't hire the guy that did a crap job. Everything gets a lot easier when you set out to do a good job.