3 ms·
We occasionally ship a build with the equivalent of 'DEBUG=true' to (parts of) production. Deliberately. For an extremely short time period. What that gains us
by groby_b 9y ago
We occasionally ship a build with the equivalent of 'DEBUG=true' to (parts of) production. Deliberately. For an extremely short time period.
What that gains us is a run with real data and a build that's instrumented with assertions - which flushes out a lot of previously unknown issues.
So yes, you learn useful things. If you ever did it accidentally, the useful thing you learn is that shipping ideally is not an on/off switch, but a carefully monitored process :)
- fredley 9y agoThat's fascinating. Is there any reason to do this in production vs. a staging environment (assuming you've got one prod-like enough with replayed traffic)? It seems a little risky to me!
- groby_b 9y agoIt's sort-of a staging, sort-of production environment :) I'm working on desktop software, and this happens to builds that are full production builds, but that don't go to stable. So a very small subset of our users - who has agreed to being guinea pigs - occasionally gets these. It also helps that we have a clear policy that assertions can only fire in cases that truly would be a bug in the wild - you can't assert and handle the issue. Either you think it might happen, then you handle it - or you state an invariant, in which case there's no reason to handle it being false. In other words, the only price you pay is that your build is slower - any assertion that fires would've been a silent bug before anyways. And, to be as close to non-debug as possible, any assertion firing merely creates a crash dump, it doesn't actually crash the executable. And finally, we simply need to do it in the wild because many of our bugs are simply due to the staggering number of environments desktop software encounters.