3 ms·

> Since I refuse to manually annotate them, I've had to relentlessly improve my analyzers until they get every last corner case right. I've fallen into this tr

> Since I refuse to manually annotate them, I've had to relentlessly improve my analyzers until they get every last corner case right. I've fallen into this trap enough times to recognize it as a trap now. Good for you that it works for you, but in general, automation of loosely-defined problems just doesn't get 100% of cases right and once it works, you need to streamline the manual process instead.

boricj 1y ago
The context of this is quite peculiar. Given the amount of information loss that occurs at the linking stage, delinking is an extremely intricate puzzle to solve. It's also very hard to be sure that you got it right, because there are many, many ways it can go wrong in ways that are a nightmare to troubleshoot. The scale of the endeavor can be also staggering: one user has over 250 000 relocation spots in their artifact, pretty much all of them must be spot-on otherwise glitches and crashes will occur. The only way I've managed to pull it off is with an analyzer design that makes no assumptions, double-checks everything, logs anything that couldn't be successfully processed and heavily investing into regression testing. The manual process then consists of fixing inaccuracies within the Ghidra database, or (rarely nowadays) fix an unhandled corner case within the analyzer. Also, no manual annotations of relocations means they cannot go stale as the Ghidra database is modified. I also happened to attempt this first on possibly the worst architecture possible, MIPS. I won't go into the details, but split HI16/LO16 relocations, register dependency graphs and branch delay slots can produce some absolutely gnarly cases to untangle. Let's just say that my MIPS analyzer contains among other things one recursive function that takes 6 parameters, 4 of them shifts one place to the left at each iteration. I'm not exactly sure how it works.

boricj 1y ago

The context of this is quite peculiar. Given the amount of information loss that occurs at the linking stage, delinking is an extremely intricate puzzle to solve. It's also very hard to be sure that you got it right, because there are many, many ways it can go wrong in ways that are a nightmare to troubleshoot. The scale of the endeavor can be also staggering: one user has over 250 000 relocation spots in their artifact, pretty much all of them must be spot-on otherwise glitches and crashes will occur. The only way I've managed to pull it off is with an analyzer design that makes no assumptions, double-checks everything, logs anything that couldn't be successfully processed and heavily investing into regression testing. The manual process then consists of fixing inaccuracies within the Ghidra database, or (rarely nowadays) fix an unhandled corner case within the analyzer. Also, no manual annotations of relocations means they cannot go stale as the Ghidra database is modified. I also happened to attempt this first on possibly the worst architecture possible, MIPS. I won't go into the details, but split HI16/LO16 relocations, register dependency graphs and branch delay slots can produce some absolutely gnarly cases to untangle. Let's just say that my MIPS analyzer contains among other things one recursive function that takes 6 parameters, 4 of them shifts one place to the left at each iteration. I'm not exactly sure how it works.