I recall that in either gdb or Qt Creator's gdb wrapper, parameters were sometimes being uninitialized when I breakpoint the function declaration line, then be

I recall that in either gdb or Qt Creator's gdb wrapper, parameters were sometimes being uninitialized when I breakpoint the function declaration line, then be initialized when I single-step to on/before the first line of code. I can't replicate it now though. ---- I'm interested in the promise of omniscient debugging to partially replace both debuggers, tracing tools, and logging (including printf debugging), for bugs in complex systems which you can reproduce in controlled environments (not in production). However I don't believe it will be perfect even there, because I've noticed that when debugging many binaries (especially non-symboled or worse-yet stripped ones) in gdb, it prints many variable values as "optimized out". This requires special debug builds which partly alleviates the problem (I still get better results with compiled-in debug prints). I've recently been tracing the execution of the pipewire daemon. Since it's a large library filled with virtual function calls constructed from C macros, it's painful to statically analyze by reading the code. I found a Callgrind profiling trace to be useful as a control flow analysis tool. Unlike perf, it logs every function called, counts the calls to each function, records all callers and how many times each caller called a function, and even records how many times each line of code calls another function (can distinguish between two calls to the same child in the same function). For example, I recently used KCacheGrind to identify that spa_alsa_open() was called 6 times, 4 times by one caller and once each by 2 other callers. And it returned early twice, and continued onto snd_pcm_open() 4 times. I also wanted to trace all ALSA functions being called across the trace (ltrace didn't work well since libasound.so.2 was loaded via dlopen, and I couldn't figure out how to make ltrace not burn CPU intercepting calls from one .so function to another). This was difficult to extract from the the Callgrind trace, and I had to rig together a script using `gprof2dot --format=callgrind -n 0 -e 0`, rg to convert the output XML file into a CSV of callers and callees, and csvsql to isolate all calls from non-snd-containing functions into snd-containing functions, so I could read the source code to see where these ALSA functions were being called. The result didn't tell me the order or parameters of the function calls, so I had to infer the order in some cases, and insert logging statements in other cases. I think Pernosco is promising in this area, but I haven't tried it yet because it's commercial and has limited free uploads. One idea (I don't know if it's implemented or not) is having the ability to run SQL-like queries to list all calls from one binary to another .so library. Another idea (on top of the caller-callee lists you already have) is creating control flow graphs like KCacheGrind.