5 ms·

Spotting Patterns in C Disassembly

54 points by donmcc 12y ago

pjmlp 12y ago
In the late 80's, when compilers for micro-computers still generated crappy code, it was quite easy to write tools that would map back to the source, if one knew which compiler was used. Most patterns were as easy as tracking down bytecodes.
- Someone 12y ago
  I think it was easier to spot which compiler was used. C vs (turbo) Pascal, of course, was trivial, but it wasn't that hard to discriminate between C compilers, either. Reverse engineering was only possible because programs were relatively small. After all, one still had to make up function and argument names.
- legulere 12y ago
  You can still see some patterns today. For instance you see interleaving in the code emitted by llvm (patterns like insa,insb,insa,insb,insa,insb)
jenandre 12y ago
awesome, ok, then you try to go reconstruct a vtable and want to kill yourself
- mschuster91 12y ago
  vtables for shit with inherited classes and interfaces \o/ sometimes I love reverse engineering. Keeps your mind active ;)
- jerryr 12y ago
  OP here. Hah! I was totally planning to follow up with a vtable example soon. :)
ndesaulniers 12y ago
Very nice, this is the blog post that I wanted to follow up mine with: http://nickdesaulniers.github.io/blog/2014/04/18/lets-write-some-x86-64/ http://nickdesaulniers.github.io/blog/2014/04/18/lets-write-... I guess I still could, for x86_64. Any interest? I believe the term for these patterns is called "lowering."
- jerryr 12y ago
  I really enjoyed your post. I actually know x86 better than ARM & I'd love to read a similar patterns post targeted at x86_64. Do you have a reference for the "lowering" term? I did a quick search & couldn't find anything--but it's a hard term to search for. Thanks! Edit: Finally found a few references for "lowering" in the context of compilers. Here's a SO reference: http://stackoverflow.com/questions/20252876/wanted-good-definition-of-the-term-lowering-in-the-context-of-compilers http://stackoverflow.com/questions/20252876/wanted-good-defi...
  - ndesaulniers 12y ago
    Thanks Jerry, I appreciate it. I wont let this post prevent me from self motivating to write the next post (and I have time now since I just finished my talk on WebGL (wow graphics is exciting))! I learned of the term from Dan, who reviewed my last post.
joncooper 12y ago
I'm working on the Coursera compilers class right now, which is excellent: https://class.coursera.org/compilers-004 https://class.coursera.org/compilers-004 The code generation portion of the class covers this material from the other end, i.e. at the point where the AST is transformed into assembly code, and it has been super interesting. This is obvious in retrospect but wasn't to me beforehand: if you're interested in reverse engineering, it's very helpful to study how the assembly was written in the first place.
- drudru11 12y ago
  Same here. I'm enjoying the class as well. However, I used to do reverse engineering, so the code gen was not that difficult. I enjoyed the part of the course that covered optimization. It was really interesting to get the prof's take on the space. Anyways, back to the homework :)
eldavido 12y ago
Interested in this stuff? Get a computer engineering degree. This is precisely what we studied in my first-year programming classes. I have fond memories of lecture halls with blackboards on every wall, divided into right and left columns, showing simplified x86 assembly on the left, and C code on the right. Another interesting topic in this area is calling conventions: the expectations of your compiler/platform on how the stack is arranged, to facilitate "linkage" between callers and callees. The "calling convention" dictates where the return value goes, where the processor registers get saved (so they can be restored when control transfers back from the called function to the caller), and the location of the return address. Also, it varies based on whether you're calling code in your own program or into the operating system -- OS calls always have to scrub all the registers, to avoid leaking protected state back to userspace. Nice article.
- xur17 12y ago
  It's also pretty fun to find spots where the compiler failed to fully optimize the code (possibly due to the optimization level chosen). For example, I recently saw a few cases in the code I was looking at where a function pushed several registers onto the stack that were never used in the function. I've found it reasonably interesting to look at at first, but it can get dull relatively quickly, especially when you're first learning.

54 points by donmcc 12y ago

pjmlp 12y ago

In the late 80's, when compilers for micro-computers still generated crappy code, it was quite easy to write tools that would map back to the source, if one knew which compiler was used. Most patterns were as easy as tracking down bytecodes.

Someone 12y ago
I think it was easier to spot which compiler was used. C vs (turbo) Pascal, of course, was trivial, but it wasn't that hard to discriminate between C compilers, either. Reverse engineering was only possible because programs were relatively small. After all, one still had to make up function and argument names.
legulere 12y ago
You can still see some patterns today. For instance you see interleaving in the code emitted by llvm (patterns like insa,insb,insa,insb,insa,insb)

jenandre 12y ago

awesome, ok, then you try to go reconstruct a vtable and want to kill yourself

mschuster91 12y ago
vtables for shit with inherited classes and interfaces \o/ sometimes I love reverse engineering. Keeps your mind active ;)
jerryr 12y ago
OP here. Hah! I was totally planning to follow up with a vtable example soon. :)

ndesaulniers 12y ago

Very nice, this is the blog post that I wanted to follow up mine with: http://nickdesaulniers.github.io/blog/2014/04/18/lets-write-some-x86-64/ http://nickdesaulniers.github.io/blog/2014/04/18/lets-write-... I guess I still could, for x86_64. Any interest? I believe the term for these patterns is called "lowering."

jerryr 12y ago
I really enjoyed your post. I actually know x86 better than ARM & I'd love to read a similar patterns post targeted at x86_64. Do you have a reference for the "lowering" term? I did a quick search & couldn't find anything--but it's a hard term to search for. Thanks! Edit: Finally found a few references for "lowering" in the context of compilers. Here's a SO reference: http://stackoverflow.com/questions/20252876/wanted-good-definition-of-the-term-lowering-in-the-context-of-compilers http://stackoverflow.com/questions/20252876/wanted-good-defi...
- ndesaulniers 12y ago
  Thanks Jerry, I appreciate it. I wont let this post prevent me from self motivating to write the next post (and I have time now since I just finished my talk on WebGL (wow graphics is exciting))! I learned of the term from Dan, who reviewed my last post.

joncooper 12y ago

I'm working on the Coursera compilers class right now, which is excellent: https://class.coursera.org/compilers-004 https://class.coursera.org/compilers-004 The code generation portion of the class covers this material from the other end, i.e. at the point where the AST is transformed into assembly code, and it has been super interesting. This is obvious in retrospect but wasn't to me beforehand: if you're interested in reverse engineering, it's very helpful to study how the assembly was written in the first place.

drudru11 12y ago
Same here. I'm enjoying the class as well. However, I used to do reverse engineering, so the code gen was not that difficult. I enjoyed the part of the course that covered optimization. It was really interesting to get the prof's take on the space. Anyways, back to the homework :)

eldavido 12y ago

Interested in this stuff? Get a computer engineering degree. This is precisely what we studied in my first-year programming classes. I have fond memories of lecture halls with blackboards on every wall, divided into right and left columns, showing simplified x86 assembly on the left, and C code on the right. Another interesting topic in this area is calling conventions: the expectations of your compiler/platform on how the stack is arranged, to facilitate "linkage" between callers and callees. The "calling convention" dictates where the return value goes, where the processor registers get saved (so they can be restored when control transfers back from the called function to the caller), and the location of the return address. Also, it varies based on whether you're calling code in your own program or into the operating system -- OS calls always have to scrub all the registers, to avoid leaking protected state back to userspace. Nice article.

xur17 12y ago
It's also pretty fun to find spots where the compiler failed to fully optimize the code (possibly due to the optimization level chosen). For example, I recently saw a few cases in the code I was looking at where a function pushed several registers onto the stack that were never used in the function. I've found it reasonably interesting to look at at first, but it can get dull relatively quickly, especially when you're first learning.