The worst bug I've seen in (IBM) BMC The device in question didn't had NTP so to get accurate BMC logs timing it had to be synced from host OS BMC is them fuc

The worst bug I've seen in (IBM) BMC The device in question didn't had NTP so to get accurate BMC logs timing it had to be synced from host OS BMC is them fucking up the watchdog implementation, that worked on wall time. Which means if you changed time in-between OS sending watchdog ping and BMC checking for timeout, you got system reboot. But it was intermittent enough that you might set the cron to update the BMC clock (because you want accurate time on it for log correlation) and be lucky enough time to not connect to the cause when it finally happens. Also we generally observed that BMCs got better, 10 years server were lucky if it was working and not crashing every few weeks (all mostly on IBM gear), to the point we had to stop alerting on that, and it was unreliable as method for OOB access, so far anything newer than few years have been fine. Althought I have no idea how you make modern hardware take minutes to return list of sensors(I'm looking at you Lenovo)