3 ms·

> Fernflower is the first actually working analytical decompiler for Java and probably for a high-level programming language in general. That really deserves a

> Fernflower is the first actually working analytical decompiler for Java and probably for a high-level programming language in general. That really deserves a link. What is an “analytical” decompiler?

jakewins 1y ago
Someone apparently had the exact same question in 2020: https://stackoverflow.com/questions/62298929/what-is-an-analytical-decompiler https://stackoverflow.com/questions/62298929/what-is-an-anal... Answer is pretty vague though, but sounds like it’s about not trying to “reverse” what the compiler did, but rather try and “analytically” work put what source code would likely have yielded the byte code it’s looking at?
- rhdunn 1y ago
  Yes, that's what it is doing. If you have a block of code a compiler will compile a language expression or statement into a particular set of assembly/bytecode instructions. For example converting `a + b` to `ADD a b`. A reversing decompiler will look at the `ADD a b` and produce `a + b` as the output. This is the simplest approach as it is effectively just a collection of these types of mapping. While this works, it can be harder to read and noisier than the actual code. This is because: 1. it does not handle annotations like @NotNull correctly -- these are shown as `if (arg == null) throw ...` instead of the annotation because the if/throw is what the compiler generated for that annotation; 2. it doesn't make complex expressions readable; 3. it doesn't detect optimizations like unrolling loops, reordering expressions, etc. For (1) an analytical decompiler can recognize the `if (arg == null) throw` expression at the start of the function and map that to a @NotNull annotation. Likewise, it could detect other optimizations like loop unrolling and produce better code for that.
  - vbezhenar 1y ago
    I'm not sure that @NotNull example is appropriate. Java compiler does not add any checks for @NotNull annotations. Those annotations exist for IDE and linting tools, compiler doesn't care. May be there are Java-like languages like Lombok or non-standard compilers which do add those checks, but I think that Java decompiler shouldn't do assumptions of these additional tools.
    - rhdunn 1y ago
      https://www.jetbrains.com/help/idea/annotating-source-code.html#disable-nullability-assertions https://www.jetbrains.com/help/idea/annotating-source-code.h... > When you compile your project with IntelliJ IDEA build tool, the IDE adds assertions to all code elements annotated with @NotNull. These assertions will throw an error if the elements happen to be null at runtime.
      - bingabingabinga 1y ago
        [dead]
      - vbezhenar 1y ago
        That's not java compiler. That's intellij compiler. I'd say that's very weird anti-feature, because your build in IDE and maven will work differently.
        wokkel 1y ago
        When using Lombok it will use a compiler plugin for this so maven builds have @nonnull generated as if-statements. I dont know if intellij uses their own plugin but they do support Lombok in maven projects, so maybe thats where this is coming from. Afaik intellij has no built in compiler but relies on java .
        Bjartr 1y ago
        Lombok hijacks the compiler to do it's own thing, and violates the contract Java compiler plugins are supposed to follow. See this comment by an OpenJDK tech lead: https://news.ycombinator.com/item?id=37666793 https://news.ycombinator.com/item?id=37666793
        lisbbb 1y ago
        I was initially impressed with Lombok and then ran into all the downsides of it and it was institutionally abandoned at one particular firm I was with (100s of devs).
lbalazscs 1y ago
The link about Stiver has some details: > Stiver decided to write his own decompiler as a side project. To overcome the weaknesses of existing alternatives, he took a different approach. After reading the bytecode, he constructed a control-flow graph in static single-assignment form, which is much better to express the program semantics abstracting the particular shape of bytecode. At the beginning of this project, Stiver knew little about static analysis and compiler design and had to learn a lot, but the effort was worth it. The resulting decompiler produced much better results than anything available at that time. It could even decompile the bytecode produced by some obfuscators without any explicit support. https://blog.jetbrains.com/idea/2024/11/in-memory-of-stiver/ https://blog.jetbrains.com/idea/2024/11/in-memory-of-stiver/
krackers 1y ago
As far as I can tell (although I"m a novice at RE), in the native world all non-trivial decompilers are "analytical", doing things like control-flow recovery and such. I guess the only reason why the first java decompiler was "non-analytical" is that the bytecode (at least in early days) was simple enough that you could basically pattern-match it back to instructions. So if I'd have to give a definition I pulled out of my ass: * non-analytical compiler: "local", works only at the instruction or basic-block level, probably done by just pattern matching templates * analytical: anything that does non-local transformations, working across basic-blocks to recover logic and control flow

jakewins 1y ago

Someone apparently had the exact same question in 2020: https://stackoverflow.com/questions/62298929/what-is-an-analytical-decompiler https://stackoverflow.com/questions/62298929/what-is-an-anal... Answer is pretty vague though, but sounds like it’s about not trying to “reverse” what the compiler did, but rather try and “analytically” work put what source code would likely have yielded the byte code it’s looking at?

rhdunn 1y ago
Yes, that's what it is doing. If you have a block of code a compiler will compile a language expression or statement into a particular set of assembly/bytecode instructions. For example converting `a + b` to `ADD a b`. A reversing decompiler will look at the `ADD a b` and produce `a + b` as the output. This is the simplest approach as it is effectively just a collection of these types of mapping. While this works, it can be harder to read and noisier than the actual code. This is because: 1. it does not handle annotations like @NotNull correctly -- these are shown as `if (arg == null) throw ...` instead of the annotation because the if/throw is what the compiler generated for that annotation; 2. it doesn't make complex expressions readable; 3. it doesn't detect optimizations like unrolling loops, reordering expressions, etc. For (1) an analytical decompiler can recognize the `if (arg == null) throw` expression at the start of the function and map that to a @NotNull annotation. Likewise, it could detect other optimizations like loop unrolling and produce better code for that.
- vbezhenar 1y ago
  I'm not sure that @NotNull example is appropriate. Java compiler does not add any checks for @NotNull annotations. Those annotations exist for IDE and linting tools, compiler doesn't care. May be there are Java-like languages like Lombok or non-standard compilers which do add those checks, but I think that Java decompiler shouldn't do assumptions of these additional tools.
  - rhdunn 1y ago
    https://www.jetbrains.com/help/idea/annotating-source-code.html#disable-nullability-assertions https://www.jetbrains.com/help/idea/annotating-source-code.h... > When you compile your project with IntelliJ IDEA build tool, the IDE adds assertions to all code elements annotated with @NotNull. These assertions will throw an error if the elements happen to be null at runtime.
    - bingabingabinga 1y ago
      [dead]
    - vbezhenar 1y ago
      That's not java compiler. That's intellij compiler. I'd say that's very weird anti-feature, because your build in IDE and maven will work differently.
      - wokkel 1y ago
        When using Lombok it will use a compiler plugin for this so maven builds have @nonnull generated as if-statements. I dont know if intellij uses their own plugin but they do support Lombok in maven projects, so maybe thats where this is coming from. Afaik intellij has no built in compiler but relies on java .
        Bjartr 1y ago
        Lombok hijacks the compiler to do it's own thing, and violates the contract Java compiler plugins are supposed to follow. See this comment by an OpenJDK tech lead: https://news.ycombinator.com/item?id=37666793 https://news.ycombinator.com/item?id=37666793
        lisbbb 1y ago
        I was initially impressed with Lombok and then ran into all the downsides of it and it was institutionally abandoned at one particular firm I was with (100s of devs).

lbalazscs 1y ago

The link about Stiver has some details: > Stiver decided to write his own decompiler as a side project. To overcome the weaknesses of existing alternatives, he took a different approach. After reading the bytecode, he constructed a control-flow graph in static single-assignment form, which is much better to express the program semantics abstracting the particular shape of bytecode. At the beginning of this project, Stiver knew little about static analysis and compiler design and had to learn a lot, but the effort was worth it. The resulting decompiler produced much better results than anything available at that time. It could even decompile the bytecode produced by some obfuscators without any explicit support. https://blog.jetbrains.com/idea/2024/11/in-memory-of-stiver/ https://blog.jetbrains.com/idea/2024/11/in-memory-of-stiver/

krackers 1y ago

As far as I can tell (although I"m a novice at RE), in the native world all non-trivial decompilers are "analytical", doing things like control-flow recovery and such. I guess the only reason why the first java decompiler was "non-analytical" is that the bytecode (at least in early days) was simple enough that you could basically pattern-match it back to instructions. So if I'd have to give a definition I pulled out of my ass: * non-analytical compiler: "local", works only at the instruction or basic-block level, probably done by just pattern matching templates * analytical: anything that does non-local transformations, working across basic-blocks to recover logic and control flow