3 ms·

Do people use LLMs to "improve" decompiled code? Just plugged the above into Gpt-4 and got: // The choice of PolygonKind can be PolygonStandard, PolygonSim

Do people use LLMs to "improve" decompiled code? Just plugged the above into Gpt-4 and got: // The choice of PolygonKind can be PolygonStandard, PolygonSimple, or BasicUnk10 - they are quite similar in functionality int intersectionResult; struct PolygonStandard *polygonStandard; int computationResult; struct BasicQueue *basicQueue; if ((edgeClipper1->clipMask & (uint16_t)(edgeClipper2->clipMask & edgeClipper3->clipMask) & 0x1F8) == 0 && (edgeClipper3->viewWidth - edgeClipper2->viewWidth) * (edgeClipper1->viewHeight - edgeClipper2->viewHeight) + (edgeClipper3->viewHeight - edgeClipper2->viewHeight) * (edgeClipper2->viewWidth - edgeClipper1->viewWidth) > 0) { Seems it didn't really understand "BucketKind". Neither do I to be honest.

qqluqq 3y ago
they became popular just a bit after I finished all the RE work, so no didn't use any for it
galangalalgol 3y ago
That is the first impressive thing I've seen a llm do with code. My company does not allow their use out of legal concerns so I haven't tried using them for anything since playiwith the initial gpt trial. I was not at all impressed then. Can you really just point it at your code base and ask it to improve the variable names or whatever?
- mkoryak 3y ago
  No don't do this. Once I asked chatgpt to make some obfuscated+minified js readable for me. It did. I was blown away by how well it chose the variable names. I started looking at this code to find the prod bug, but I found other bugs that I wasn't expecting... I took a closer look and realized that the code it made was almost right, but also 5% wrong. I wouldn't have noticed the incorrectness if I wasn't debugging for several hours before and had some general idea of what the pretty printed code did and did not do.
  - drxzcl 3y ago
    You can get better results by setting up a test harness that compares to the original and feeding back the results. But then you are doing boring work and letting the llm do the fun work, which is not the point the whole thing.
  - pjerem 3y ago
    Maybe you can get achieve it by asking it to just give you the name correspondances in a table and do the renaming yourself. That’d be hardly useful in a full file with colliding names but on a single function that may be useful.
- jerf 3y ago
  Ah, here we see the great danger of LLMs, which is that they have the ability to be confident enough to bullshit you completely successfully. I don't think this is clipping edges, although I could be wrong about that, but if all thomasahle was to put the exact above code into GPT-4 and did not go to the source and pull more out, GPT has absolutely no reason to label "z" as "intersectionResult" as it was never used in the rest of the code, same for labelling "v6" as "computationResult", and the guesses for polygonStandard and basicQueue are at best the first stab a decompiler might take, but there's probably a better name than simply reiterating the type. That's not better variable names; that's better-looking garbage. At least you know you don't know what "v7" is. With the post-LLM code you're having to poke through confident error.
araes 3y ago
They do now
taway1237 3y ago
Answering your question directly: RE is 60% of my daily work, and I don't use and don't know anyone who uses LLMs for RE. One reason is that you lose interactivity if you process code using something other than your decompiler of choice, and it's a huge problem - you want to stick to your RE "IDE". The other is that, at least in my line of work, I don't need perfect code, just to understand what's going on and maybe a few critical functions.

qqluqq 3y ago

they became popular just a bit after I finished all the RE work, so no didn't use any for it

galangalalgol 3y ago

That is the first impressive thing I've seen a llm do with code. My company does not allow their use out of legal concerns so I haven't tried using them for anything since playiwith the initial gpt trial. I was not at all impressed then. Can you really just point it at your code base and ask it to improve the variable names or whatever?

mkoryak 3y ago
No don't do this. Once I asked chatgpt to make some obfuscated+minified js readable for me. It did. I was blown away by how well it chose the variable names. I started looking at this code to find the prod bug, but I found other bugs that I wasn't expecting... I took a closer look and realized that the code it made was almost right, but also 5% wrong. I wouldn't have noticed the incorrectness if I wasn't debugging for several hours before and had some general idea of what the pretty printed code did and did not do.
- drxzcl 3y ago
  You can get better results by setting up a test harness that compares to the original and feeding back the results. But then you are doing boring work and letting the llm do the fun work, which is not the point the whole thing.
- pjerem 3y ago
  Maybe you can get achieve it by asking it to just give you the name correspondances in a table and do the renaming yourself. That’d be hardly useful in a full file with colliding names but on a single function that may be useful.
jerf 3y ago
Ah, here we see the great danger of LLMs, which is that they have the ability to be confident enough to bullshit you completely successfully. I don't think this is clipping edges, although I could be wrong about that, but if all thomasahle was to put the exact above code into GPT-4 and did not go to the source and pull more out, GPT has absolutely no reason to label "z" as "intersectionResult" as it was never used in the rest of the code, same for labelling "v6" as "computationResult", and the guesses for polygonStandard and basicQueue are at best the first stab a decompiler might take, but there's probably a better name than simply reiterating the type. That's not better variable names; that's better-looking garbage. At least you know you don't know what "v7" is. With the post-LLM code you're having to poke through confident error.

araes 3y ago

They do now

taway1237 3y ago

Answering your question directly: RE is 60% of my daily work, and I don't use and don't know anyone who uses LLMs for RE. One reason is that you lose interactivity if you process code using something other than your decompiler of choice, and it's a huge problem - you want to stick to your RE "IDE". The other is that, at least in my line of work, I don't need perfect code, just to understand what's going on and maybe a few critical functions.