3 ms·
Engineers shouldn’t deploy to prod directly, but sometimes it’s necessary to SSH into an instance for logs, stack dumps, etc. Source: worked for 2 big to very b
by shawkinaw 6y ago
Engineers shouldn’t deploy to prod directly, but sometimes it’s necessary to SSH into an instance for logs, stack dumps, etc. Source: worked for 2 big to very big tech cos.
- SteveNuts 6y agoNot having those things centralized is also a huge operational failure regardless of company size.
- alias_neo 6y agoFor a large or v large tech co you should probably be aggregating logs to a centralised location that doesn't require access to production systems in this way. Stack dumps should also be collected safely off-system if necessary. Perhaps my industry is a little more security conscious (I don't know which industry you're talking about), but this doesn't seem like good practice.
- adwww 6y agoWho fixes the centralised log system when that needs debugging? Unless prohibited in something like banking, following best practice to the letter is sometimes unacceptably slow for most industries.
- shawkinaw 6y agoLet me be clear, I agree it should not be normal to SSH into a prod box. Our logs are centrally aggregated. But it’s one thing to say it’s not normal, but quite another to say engineers shouldn't have access, because I totally disagree with that.
- throwaway201103 6y agoWhat normally (should) happens in that unusual case is that the engineer is issued a special short-lifetime credential to do what needs to be done. An audit trail is kept of when and to whom the credential was issued, for what purpose, when it was revoked, etc.
- mlnj 6y agoThere should be tools that allow the team to gather such logs. Direct prod access is a recipe for disaster.