5 ms·

Hacking java bytecode

93 points by onderkalaci 11y ago

th3iedkid 11y ago
very interesting ! >>There are some who may scoff at bytecode optimization, reasoning that it’s a worthless skill for modern computer science, especially those working in high-level languages. Understanding what the compiler emits however is a skill still very much alive and well in embedded programming, FPGA programming, and other performance-oriented disciplines. A few more use-cases are with building instrumentation modules, profilers besides byte-code analytic for automated-binary-audits,obfuscation amongst others.
kitd 11y ago
I've been writing Java for 15 years and I never knew about breaking out of arbitrary labelled blocks!
- _yosefk 11y ago
  It's funny how there's still a considered-harmful goto buried in the language, even though they don't call it that.
  - Alupis 11y ago
    > It's funny how there's still a considered-harmful goto buried in the language, even though they don't call it that. Not quite. GOTO allowed you to jump anywhere in the code, for any reason, at any time. Java's Label only really allows you to break out of a loop and jump to a label that was previously defined before that loop. It's mainly used for breaking out of a set of nested loops on some condition.
  - pkolaczk 11y ago
    It's also funny to sometimes see the same people who say goto is bad, use break and continue heavily in their loops. Which are only tad better than goto and often just as hard to read.
    - kuschku 11y ago
      Loops in general are a problem. Tail-Recursive functions would be a far better solution. And almost every loop can be replaced with a simple map, fold, or filter.
      - frik 11y ago
        Recursion can be expensive (call stack) depending on the language implementation. For example Lua 5.1 and recent JS implementations have a proper recursive tail call implementation. The Lua manual explains it: "Without proper tail calls, each user move would create a new stack level. After some number of moves, there would be a stack overflow. With proper tail calls, there is no limit to the number of moves that a user can make, because each move actually performs a goto to another function, not a conventional call." -- http://www.lua.org/pil/6.3.html http://www.lua.org/pil/6.3.html
- jfoutz 11y ago
  you can also abuse the syntax in an amusing way public class Foo{ public static void main(String[] args){ http://news.ycombinator.com/ System.out.println("hello kitd"); } } because http: is a label, and // starts a comment.
  - _lce0 11y ago
    haha that's evil AND genious!! public class OhLord { public static void main(String[] args){ free: { I: want: to: break free; } System.out.println("yeah!"); } }
    - hinkley 11y ago
      It's strange, but it's true.
userbinator 11y ago
contestants are required to turn in their source, rather than compiled code If there's nothing about the language in which the source must be written in but only that it compiles to JVM bytecode that meets the requirements, I'd be very tempted to write most or almost all of it in bytecode... java.lang.ClassLoader is allowed and it provides some useful methods for that purpose. ;-) At each block’s exit, a checkpoint is injected with the block’s total cost. During live execution, these checkpoints increment the AI’s internal total bytecode counter. If at any checkpoint the running tally exceeds GameConstants.BYTECODE_LIMIT, the AI’s execution is halted and execution of the next robot’s AI begins. That sounds to me like it might be possible to squeeze in some more extra instructions at the end.
- marktangotango 11y ago
  One might imagine this notion of limiting computational budget of arbitrary code would have applications beyond this particualar application.
- jdmichal 11y ago
  My immediate thought was that branchless coding could have a big effect if used right. But those functions usually use more operations to achieve the same effect, so the tradeoff would probably not be worth it.
- icefox 11y ago
  I was excited to give a shot at Battlecode until I heard about the broken solution to limit resources. >In order to hard limit each team’s total computation, yet guarantee equal computation resources to each team, each team is given a bytecode limit, and their code is instrumented and allowed to run only up to that limit before it is halted. They want to give each team equal cpu time, but when you have a bytecode that has instructions like invokevirtual and multianewarray the execution time isn't' exactly similar. And then there is the story about how in 2007 to make things easier the engine counted the bytecodes of various standard library methods as a predetermined fixed number no matter what happened. So of course someone put A* inside string.matches with a regular expression. (http://realgl.blogspot.com/2013/08/battlecode.html http://realgl.blogspot.com/2013/08/battlecode.html). Sadly whenever I hear about Battlecode I hear stories about how people abused and broke the bytecode limitation to get more cpu resources than everyone else not about how they solved the problem straight up better than everyone else.
sz4kerto 11y ago
I remember the good old days when I wrote a postcompiler for .Net, and I realized that I can jump back from a catch clause to the try block :)
- louthy 11y ago
  A retry feature in C#/F# would be nice for sure!
trun 11y ago
Cory's link to the instrumentation source doesn't seem to exist anymore, but here's the latest public version... https://github.com/battlecode/battlecode-server/tree/master/src/main/battlecode/engine/instrumenter https://github.com/battlecode/battlecode-server/tree/master/... Most of the interesting bits happen in RoboMethodTree.
- onderkalaci 11y ago
  Thanks for this one!
xvilka 11y ago
I think it is worth to mention that you can inspect Java bytecode [1][2] with radare2 tool (use the git version [3]) [1] http://dso.thecoverofnight.com/posts/2014/04/radare-java-intro/ http://dso.thecoverofnight.com/posts/2014/04/radare-java-int... [2] http://radare.org/get/radare_java.pdf http://radare.org/get/radare_java.pdf [3] https://github.com/radare/radare2 https://github.com/radare/radare2
j_s 11y ago
Dexer, a .NET-based tool to work with Android Dalvik bytecode in a manner similar to Mono.Cecil: http://sebastien.lebreton.free.fr/blog/index.php?/categories/4-Dexer http://sebastien.lebreton.free.fr/blog/index.php?/categories... The IKVM.NET blog covers a lot of low-level details as the author implements 'JVM on .NET CLR': http://weblog.ikvm.net/ http://weblog.ikvm.net/

93 points by onderkalaci 11y ago

very interesting ! >>There are some who may scoff at bytecode optimization, reasoning that it’s a worthless skill for modern computer science, especially those working in high-level languages. Understanding what the compiler emits however is a skill still very much alive and well in embedded programming, FPGA programming, and other performance-oriented disciplines. A few more use-cases are with building instrumentation modules, profilers besides byte-code analytic for automated-binary-audits,obfuscation amongst others.

kitd 11y ago

I've been writing Java for 15 years and I never knew about breaking out of arbitrary labelled blocks!

_yosefk 11y ago
It's funny how there's still a considered-harmful goto buried in the language, even though they don't call it that.
- Alupis 11y ago
  > It's funny how there's still a considered-harmful goto buried in the language, even though they don't call it that. Not quite. GOTO allowed you to jump anywhere in the code, for any reason, at any time. Java's Label only really allows you to break out of a loop and jump to a label that was previously defined before that loop. It's mainly used for breaking out of a set of nested loops on some condition.
- pkolaczk 11y ago
  It's also funny to sometimes see the same people who say goto is bad, use break and continue heavily in their loops. Which are only tad better than goto and often just as hard to read.
  - kuschku 11y ago
    Loops in general are a problem. Tail-Recursive functions would be a far better solution. And almost every loop can be replaced with a simple map, fold, or filter.
    - frik 11y ago
      Recursion can be expensive (call stack) depending on the language implementation. For example Lua 5.1 and recent JS implementations have a proper recursive tail call implementation. The Lua manual explains it: "Without proper tail calls, each user move would create a new stack level. After some number of moves, there would be a stack overflow. With proper tail calls, there is no limit to the number of moves that a user can make, because each move actually performs a goto to another function, not a conventional call." -- http://www.lua.org/pil/6.3.html http://www.lua.org/pil/6.3.html
jfoutz 11y ago
you can also abuse the syntax in an amusing way public class Foo{ public static void main(String[] args){ http://news.ycombinator.com/ System.out.println("hello kitd"); } } because http: is a label, and // starts a comment.
- _lce0 11y ago
  haha that's evil AND genious!! public class OhLord { public static void main(String[] args){ free: { I: want: to: break free; } System.out.println("yeah!"); } }
  - hinkley 11y ago
    It's strange, but it's true.

userbinator 11y ago

contestants are required to turn in their source, rather than compiled code If there's nothing about the language in which the source must be written in but only that it compiles to JVM bytecode that meets the requirements, I'd be very tempted to write most or almost all of it in bytecode... java.lang.ClassLoader is allowed and it provides some useful methods for that purpose. ;-) At each block’s exit, a checkpoint is injected with the block’s total cost. During live execution, these checkpoints increment the AI’s internal total bytecode counter. If at any checkpoint the running tally exceeds GameConstants.BYTECODE_LIMIT, the AI’s execution is halted and execution of the next robot’s AI begins. That sounds to me like it might be possible to squeeze in some more extra instructions at the end.

marktangotango 11y ago
One might imagine this notion of limiting computational budget of arbitrary code would have applications beyond this particualar application.
jdmichal 11y ago
My immediate thought was that branchless coding could have a big effect if used right. But those functions usually use more operations to achieve the same effect, so the tradeoff would probably not be worth it.
icefox 11y ago
I was excited to give a shot at Battlecode until I heard about the broken solution to limit resources. >In order to hard limit each team’s total computation, yet guarantee equal computation resources to each team, each team is given a bytecode limit, and their code is instrumented and allowed to run only up to that limit before it is halted. They want to give each team equal cpu time, but when you have a bytecode that has instructions like invokevirtual and multianewarray the execution time isn't' exactly similar. And then there is the story about how in 2007 to make things easier the engine counted the bytecodes of various standard library methods as a predetermined fixed number no matter what happened. So of course someone put A* inside string.matches with a regular expression. (http://realgl.blogspot.com/2013/08/battlecode.html http://realgl.blogspot.com/2013/08/battlecode.html). Sadly whenever I hear about Battlecode I hear stories about how people abused and broke the bytecode limitation to get more cpu resources than everyone else not about how they solved the problem straight up better than everyone else.

sz4kerto 11y ago

I remember the good old days when I wrote a postcompiler for .Net, and I realized that I can jump back from a catch clause to the try block :)

louthy 11y ago
A retry feature in C#/F# would be nice for sure!

trun 11y ago

Cory's link to the instrumentation source doesn't seem to exist anymore, but here's the latest public version... https://github.com/battlecode/battlecode-server/tree/master/src/main/battlecode/engine/instrumenter https://github.com/battlecode/battlecode-server/tree/master/... Most of the interesting bits happen in RoboMethodTree.

onderkalaci 11y ago
Thanks for this one!

xvilka 11y ago

I think it is worth to mention that you can inspect Java bytecode [1][2] with radare2 tool (use the git version [3]) [1] http://dso.thecoverofnight.com/posts/2014/04/radare-java-intro/ http://dso.thecoverofnight.com/posts/2014/04/radare-java-int... [2] http://radare.org/get/radare_java.pdf http://radare.org/get/radare_java.pdf [3] https://github.com/radare/radare2 https://github.com/radare/radare2

j_s 11y ago

Dexer, a .NET-based tool to work with Android Dalvik bytecode in a manner similar to Mono.Cecil: http://sebastien.lebreton.free.fr/blog/index.php?/categories/4-Dexer http://sebastien.lebreton.free.fr/blog/index.php?/categories... The IKVM.NET blog covers a lot of low-level details as the author implements 'JVM on .NET CLR': http://weblog.ikvm.net/ http://weblog.ikvm.net/