17 ms·

> I worked from the executable provided by SIOU, first interactively under GDB (but it nearly drove me crazy, as I had to wait sometimes one hour to trigger the

> I worked from the executable provided by SIOU, first interactively under GDB (but it nearly drove me crazy, as I had to wait sometimes one hour to trigger the crash again), then using a little OCaml script that ran the program 1000 times and saved the core dumps produced at every crash. rr can often be a time-saver in situations by providing deterministic replays up to the point of a crash, whereas coredump analysis is a single retrospective snapshot. http://rr-project.org/ http://rr-project.org/

tdullien 9y ago
As much as I love RR, I am not sure it would've helped here, as the bug requires multiple threads to concurrently run? Also, RR is based on achieving deterministic replay IIRC, so I am not sure it'd be the first choice for a nondeterministic hardware bug?
- KenoFischer 9y ago
  You can do multiple concurrent rr recordings. A non-deterministic cpu bug would generally cause a divergence in the recording vs replay, so rr would be a decent way to go about this. The way I'd have probably used rr when faced with this is to bisect the recording to find which code is responsible.
nosefouratyou 9y ago
There's also UndoDB if you can pay for it. Not sure what the differences are between it and RR, but I know with UndoDB you can create a binary for the client that has the recorder built in, so it can automatically record failing states.
joshuata 9y ago
rr wouldn't help in this case. From the docs, it "emulates a single-core machine. So, parallel programs incur the slowdown of running on a single core." The skylake bug only occurred under heavily threaded loads.
jacquesm 9y ago
How will emulation trigger a hardware bug?
- AgentME 9y ago
  rr doesn't do any emulation. It records the inputs to a program so that it could be run again deterministically the same way again. However it doesn't actually run code as multi-threaded so it wouldn't be a help here.
testfailure 9y ago
Replaying the recorded code under either rr or UndoDB would have revealed that something odd was going on in the CPU when the recording replayed differently depending on whether HT was enabled. Obviously finding this would have required that the tester replayed the recording multiple times with and without HT enabled

tdullien 9y ago

As much as I love RR, I am not sure it would've helped here, as the bug requires multiple threads to concurrently run? Also, RR is based on achieving deterministic replay IIRC, so I am not sure it'd be the first choice for a nondeterministic hardware bug?

KenoFischer 9y ago
You can do multiple concurrent rr recordings. A non-deterministic cpu bug would generally cause a divergence in the recording vs replay, so rr would be a decent way to go about this. The way I'd have probably used rr when faced with this is to bisect the recording to find which code is responsible.

nosefouratyou 9y ago

There's also UndoDB if you can pay for it. Not sure what the differences are between it and RR, but I know with UndoDB you can create a binary for the client that has the recorder built in, so it can automatically record failing states.

joshuata 9y ago

rr wouldn't help in this case. From the docs, it "emulates a single-core machine. So, parallel programs incur the slowdown of running on a single core." The skylake bug only occurred under heavily threaded loads.

jacquesm 9y ago

How will emulation trigger a hardware bug?

AgentME 9y ago
rr doesn't do any emulation. It records the inputs to a program so that it could be run again deterministically the same way again. However it doesn't actually run code as multi-threaded so it wouldn't be a help here.

testfailure 9y ago

Replaying the recorded code under either rr or UndoDB would have revealed that something odd was going on in the CPU when the recording replayed differently depending on whether HT was enabled. Obviously finding this would have required that the tester replayed the recording multiple times with and without HT enabled