4 ms·

There are a few concepts that do not map neatly: 1. For firmware, each user is on their own hardware. Rather than a session you need to track a device and the

by fra 7y ago

There are a few concepts that do not map neatly: 1. For firmware, each user is on their own hardware. Rather than a session you need to track a device and the state thereof. Devices exist for a longer period of time than sessions do, and you need to have a concept of "device history". 2. Sentry assumes backtraces can be generated on the client side, which is impractical for firwmare 3. Our focus on embedded allows us to run some more specific analysis. For example we automatically detect if your MPU (memory protection unit) is misconfigured on an ARM chip.

ggregoire 7y ago
Makes sense. Thanks!
btashton 7y ago
I built a translator service between my devices and Sentry to take the memory dump and a known symbol file to rebuild the context serverside and send it back to Sentry. I only used this during development and early testing, but it was super useful. At one point I thought about building this out more, so I'm glad to see someone is taking this on more seriously.
- tyhoff 7y ago
  We are taking it very seriously ;) Happy to hear that an you thought to build this translation service even for early development! It's usually an after thought at the companies we've talked to, and an expensive one too.
zeeg 7y ago
We have customers using us for firmware errors :)
- fra 7y ago
  Yes, I was excited to meet some of them! It is certainly possible to use Sentry with firmware, but I'd venture to say that Memfault is an easier integration and a better experience.
the_mitsuhiko 7y ago
> 2. Sentry assumes backtraces can be generated on the client side, which is impractical for firwmare If you can produce a minidump you can send it to Sentry and have it stackwalked on the server. Breakpad/Crashpad can produce such dumps for you. We do not have coredumps yet but that could be added if there is demand.
- chrisc11 7y ago
  Yeah, minidump and breakpad/crashpad can be great when running native code on an OS! One challenge with embedded devices (i.e ARM Cortex-Ms) is that it is not always safe/viable to try and grab the thread contexts at the time of crash. These MCUs have none or very limited support for memory management (a MPU at best), the OS/application state usually lives in the same RAM area, and the stack dealing with exception handling is usually quite small. For these reasons, it's usually desirable to grab a very simple memory dump and offload as much of the processing as possible to the server. The "core dump" format that is used for linux is saved as an ELF with a few special PT_NOTE sections to convey thread information. This would be hard to generate on most embedded devices. The coredump we collect for embedded devices is closer to a mini-dump. It is basically just a raw memory dump and the current register state on the device. On the server side, we recover the thread contexts and backtraces by using the debug symbols, the RAM capture, and analyzers for the RTOS that was used (i.e FreeRTOS, Zephyr, ThreadX, Mbed OS)

by fra 7y ago

ggregoire 7y ago

Makes sense. Thanks!

btashton 7y ago

I built a translator service between my devices and Sentry to take the memory dump and a known symbol file to rebuild the context serverside and send it back to Sentry. I only used this during development and early testing, but it was super useful. At one point I thought about building this out more, so I'm glad to see someone is taking this on more seriously.

tyhoff 7y ago
We are taking it very seriously ;) Happy to hear that an you thought to build this translation service even for early development! It's usually an after thought at the companies we've talked to, and an expensive one too.

zeeg 7y ago

We have customers using us for firmware errors :)

fra 7y ago
Yes, I was excited to meet some of them! It is certainly possible to use Sentry with firmware, but I'd venture to say that Memfault is an easier integration and a better experience.

the_mitsuhiko 7y ago

> 2. Sentry assumes backtraces can be generated on the client side, which is impractical for firwmare If you can produce a minidump you can send it to Sentry and have it stackwalked on the server. Breakpad/Crashpad can produce such dumps for you. We do not have coredumps yet but that could be added if there is demand.

chrisc11 7y ago
Yeah, minidump and breakpad/crashpad can be great when running native code on an OS! One challenge with embedded devices (i.e ARM Cortex-Ms) is that it is not always safe/viable to try and grab the thread contexts at the time of crash. These MCUs have none or very limited support for memory management (a MPU at best), the OS/application state usually lives in the same RAM area, and the stack dealing with exception handling is usually quite small. For these reasons, it's usually desirable to grab a very simple memory dump and offload as much of the processing as possible to the server. The "core dump" format that is used for linux is saved as an ELF with a few special PT_NOTE sections to convey thread information. This would be hard to generate on most embedded devices. The coredump we collect for embedded devices is closer to a mini-dump. It is basically just a raw memory dump and the current register state on the device. On the server side, we recover the thread contexts and backtraces by using the debug symbols, the RAM capture, and analyzers for the RTOS that was used (i.e FreeRTOS, Zephyr, ThreadX, Mbed OS)