3 ms·

> repl'd into prod to do work Like for debugging production problems and fixing customer data? Or for normal development? If its the former that's a great us

• psd1 2y ago
  It's an order of magnitude harder to debug when you don't have access to prod, but there's a reason to block that access. I think you need to put controls on that fairly early in your project's evolution. Any good strategies to reduce the pain? My previous employer never solved this. I always wanted to explore contextual logging - by which I mean, logging is terse by default, but in an error state, the stack is walked for contextual info to make the log entry richer; and also, ideally, previous debug log entries that are suppressed by default are instead written. I guess that implies buffering log entries and writing only a subset at the end of the happy path. To illustrate what I mean: happy path log: 10:21:04 Authenticated 10:21:05 Scumbulated Error condition log: 10:21:04 Authenticating id 49234 request DEADBEEF IDP response OK for 49234 request DEADBEEF https://idp.dundermifflin.com https://idp.dundermifflin.com Session cookie OK request DEADBEEF Authenticated 10:21:05 Scumbulating flange 7671529 user 49234 request DEADBEEF NullFlangeError flange 7671529 at scum.py:265 Frame vars a=42, password=redacted, flags=0x05 I'm reacting to hard-to-repro bugs at $employer where we chucked logging statements at a dartboard, deployed, waited, didn't capture the issue, repeat several times. At a cadence of 5-10 deploys a week, this is below what I consider acceptable velocity. We often took days to fix major bugs, we'd run degraded for weeks at a time.