4 ms·
Thank you for this work! I've been involved with minidumps in one way or another since around 2010. Was at a startup at the time that had a browser based on Ch
by js2 4y ago
Thank you for this work!
I've been involved with minidumps in one way or another since around 2010. Was at a startup at the time that had a browser based on Chromium and we needed crash reporting for our own app. So I wrote a pretty simply backend that received minidumps, ran them through the breakpad processor and shoved the output into Splunk. That was our crash-reporting system.
Circa 2013 the company gets acquired by Yahoo which at the time was using Crittercism for its mobile apps but Yahoo wasn't happy with it. Somehow I was now the mobile app crash reporting expert at the company though so I built a whole new in-house crash reporting solution.
For iOS I wrote an SDK around PLCrashReporter because unwinding stacks on the client works out way better on iOS than dealing with a minidump.
For Android I had to deal with both JVM (er, Dalvik, er ART) stack traces, easy enough, but also native code crashes. For the latter I used breakpad's crash handler and minidumps. But it turns out that minidumps from Android devices are almost useless for two reasons:
1) If the crashes originate in managed code or calls into managed code you can't trace back through the managed code frames from a minidump. Especially if you don't have frame pointers.
2) You basically cannot get the symbols for all the different flavors of Android. Without symbols any stack trace that breakpad reconstructs is pretty useless.
Eventually I abandoned minidumps on Android and instead unwinding on the phone using corkscrew, wait no, libbacktrace, wait no, libunwind. But that still doesn't give useful stack traces very often. In the end, I ended up capturing logcat output when restarting after a crash which actually tends to have the most useful stack traces.
Which is all to say, both Apple and Google make it really hard for a mobile app to find out why it crashed. Both Android and iOS create a crash report for any app which crashes, but the app can't access those. So we're all shipping apps with third-party crash handlers built-in that try to capture a stack or minidump in-process and make sense of it later.
- palata 4y agoI was recently looking into CoffeeCatch for Android. Are you saying that I'm losing my time? :-)
- js2 4y agoThat may work for your own code which you can pepper with macros, but I needed to be able to capture crashes in native code I didn't write. Android itself has used a variety of unwinders over the years. Extracted versions of the libraries that have been modified to build with the NDK are here: https://github.com/ivanarh/libcorkscrew-ndk https://github.com/ivanarh/libcorkscrew-ndk https://github.com/ivanarh/libbacktrace-ndk https://github.com/ivanarh/libbacktrace-ndk https://github.com/ivanarh/libunwind-ndk https://github.com/ivanarh/libunwind-ndk https://github.com/ivanarh/libunwindstack-ndk https://github.com/ivanarh/libunwindstack-ndk Along with a wrapper that can use all of them: https://github.com/ivanarh/ndcrash https://github.com/ivanarh/ndcrash It's been a while since I dealt with this and I don't recall why Android keeps changing unwinders. Looks like Sentry is using libunwindstack-ndk (well a fork of it anyway) on Android: https://github.com/getsentry/sentry-native/tree/master/external https://github.com/getsentry/sentry-native/tree/master/exter...
- saagarjha 4y agoApple actually recently added support for high-quality crash reports (the kind you'd get from their tooling, more or less) in MetricKit: https://developer.apple.com/documentation/metrickit/mxcrashdiagnostic https://developer.apple.com/documentation/metrickit/mxcrashd....
- js2 4y agoOoh, that's useful. PLCrashReporter actually does a darn good job with capturing crashes. But what it can't capture is when iOS force kills the app (watchdog, memory, etc). MetricsKit looks like it has all sorts of useful data for capturing data about that. I was vaguely aware of MetricsKit but didn't realize how far it had come. Thanks!