This brings back memories.... At a previous employer we had a product of ours that would just randomly reboot. Standard Windows on an industrial PC, nothing yo

This brings back memories.... At a previous employer we had a product of ours that would just randomly reboot. Standard Windows on an industrial PC, nothing you’d think would be that esoteric. It’d look like a sudden power failure. No crashes, nothing in the logs to indicate anything at all. Similar hardware platform to some other products we had where this issue never arose. It literally took months to find the cause, while in the meantime the people that didn’t want us to succeed (think big-company politics) were scoring points against us. We did everything we could think of, fixed a lot of bugs, tried stress-testing the system, things like that. Nothing worked. We were met with skepticism when we started saying it was the hardware. They said it can’t be the hardware.... Except this time it was — and we found out almost purely by good luck. One of the hardware guys happened to have an oscilloscope hooked up to a system when the reboot happened. The reboot was preceded by ‘something strange’ on the scope (I’m likely missing stuff here since this is memory from a few years back). It turned out the motherboard had a bug which would manifest when the system went into a certain lower power state. That’s why stress-testing never caused it. In fact, the testing we did was actively preventing us from catching it!! Lesson well and truly learned. On a related note - is it just me or did Intel have a lot of trouble with their power-management implementation a few years back? I’ve since worked on another product that would randomly hard-freeze and sure enough after checking the Intel errata doc for the CPU/SoC in question (an Atom something-or-other), sure enough there were issues with S-states that meant we needed to limit them (to S1 I believe) in the BIOS, which did indeed fix the freeze issue. To anyone that wants to avoid the long road here, realise that modern Windows, Linux and macOS systems never freeze or kernel panic without a special reason. Part of what I had to unlearn many years ago was the notion that on Windows, freezes and blue-screens were things that ‘just happened’ in the course of otherwise regular operation. Low-level can get you, even if you aren’t doing low-level work.