4 ms·

>They have to install a litany of extra software to diagnose and troubleshoot bugs, rather than expose system metrics and tightly control the app environment an

>They have to install a litany of extra software to diagnose and troubleshoot bugs, rather than expose system metrics and tightly control the app environment and its operation. There's a lot of things that require more than easily exported system metrics and logs to troubleshoot. While I've played around with using PCP's perf plugin to try and remotely do things with perf, generate flamegraphs, etc., it doesn't work nearly as well as just SSH'ing into the thing and running perf directly, especially if the perf data file is going to be large. I don't see how you could do serious performance engineering work without SSH access. But, I think I'm nitpicking here, because I generally agree that there should be very little to no reason to login to servers via SSH day to day.

peterwwillis 7y ago
As an alternative, I try to implement solutions that can install and run software out of bounds and still get what you need without a persistent connection and opening up security groups (example: AWS SSM Run Command).
- cthalupa 7y ago
  I make extensive use of Run Command (And Automations!) but they're not a replacement for every use case, and very much not a replacement for the specific use case you're replying to.
jameshart 7y ago
in the ‘pets vs cattle’ analogy, it totally makes sense that even with cattle, occasionally you bring one in for a checkup by a vet to see if you can detect any problems that might affect the herd. Ssh into a production box to check everything is working as expected and take some readings. Sure. On the other hand, I tend to lean more towards a ‘wild animals’ model, where, sure, you can tranquilize one and bring it in to look over, but once it’s got the smell of humans on it, it’s doomed if you let it back out in the wild again. Once you ssh into a production box, it is forever tainted. Sure, poke around in it, install some perf tools to run some diagnostics, learn what you can about its behavior. But then, rather than putting it back into the wild to serve traffic, out of mercy, you should destroy it and replace it with a clean instance.
- cthalupa 7y ago
  I don't disagree with this, but, at the same time, if I haven't made any actual changes to the application, I'll generally not worry about manually taking it out of service, because autoscaling will be getting rid of it soon enough anyway.

peterwwillis 7y ago

As an alternative, I try to implement solutions that can install and run software out of bounds and still get what you need without a persistent connection and opening up security groups (example: AWS SSM Run Command).

cthalupa 7y ago
I make extensive use of Run Command (And Automations!) but they're not a replacement for every use case, and very much not a replacement for the specific use case you're replying to.

jameshart 7y ago

in the ‘pets vs cattle’ analogy, it totally makes sense that even with cattle, occasionally you bring one in for a checkup by a vet to see if you can detect any problems that might affect the herd. Ssh into a production box to check everything is working as expected and take some readings. Sure. On the other hand, I tend to lean more towards a ‘wild animals’ model, where, sure, you can tranquilize one and bring it in to look over, but once it’s got the smell of humans on it, it’s doomed if you let it back out in the wild again. Once you ssh into a production box, it is forever tainted. Sure, poke around in it, install some perf tools to run some diagnostics, learn what you can about its behavior. But then, rather than putting it back into the wild to serve traffic, out of mercy, you should destroy it and replace it with a clean instance.

cthalupa 7y ago
I don't disagree with this, but, at the same time, if I haven't made any actual changes to the application, I'll generally not worry about manually taking it out of service, because autoscaling will be getting rid of it soon enough anyway.