6 ms·

JRuby Creator's Short List of Key Missing JVM Features

124 points by gthank 16y ago

donw 16y ago
I've been wondering for awhile if the JVM really has a lot of runway left; Java 7 has been amazingly slow in coming, and it'll be at least six months, and more likely a year, before even the early adopters can really use it in production. Startup time is a very real problem, as is memory use compared to other, similar VMs (V8, LLVM). With closures (lambda expressions), Java will be a lot more useful, but my money is honestly on V8 and JavaScript at this point. It's moving faster, and the node.js guys are very right that JavaScript is the language of the web.
- sb 16y ago
  Hi, just for the record, V8 differs in one very important aspect from the JVM: it includes no interpreter. It directly compiles methods to native code before their first execution, whereas the JVM includes a highly optimized interpreter for the first couple of executions. This affects memory usage, too (a couple of things that come to mind are profiling, additional memory for de-optimization, etc.)
  - bad_user 16y ago
    I have a hard time believing that V8 doesn't do profiling / de-optimizations ... as these are crucial parts for optimizing runtime dispatch or boxing/unboxing of primitives. I know that V8 doesn't do tracing compiling, but still, it should at least have inline caches.
- noelwelsh 16y ago
  The JVM does a huge amount more than V8/node.js. If your world consists solely of shuffling bytes around a network then node.js may be a good solution, but it doesn't extend much further than that. For example, I wouldn't want to write a machine learning algorithm in Javascript, nor would I want to write a storage engine, nor a ... you get the idea. Furthermore, while node.js may be growing quickly (easy when you're small!) the development of the Javascript language is taking a rather torturous route as the various vendors play games at the Ecmascript table. To return to the article, what I really want from both the JVM and Javascript is tail recursion. I'd also like proper lexical scoping in Javascript, though that isn't so important. I don't view Java or Javascript as languages you write but rather languages you compile to (well, you compile to bytecode for the JVM, but hopefully you get my point). Scala and Clojure make mighty fine Java replacements; I haven't seen anything yet for Javascript that is much of an improvement.
  - sanderjd 16y ago
    Could you list some reasons why you wouldn't want to write a machine learning algorithm in Javascript, or a write storage engine nor a ...? Lack of libraries? Too high level? Or do you not like the syntax/semantics/...?
    - jshen 16y ago
      threading maybe?
    - noelwelsh 16y ago
      Performance -- Javascript VMs are not optimised for numeric code. Lack of libraries. I don't particularly like JS for involved projects (e.g. no module system, so you have to write your own).
- axod 16y ago
  Why is startup time relevant if you're doing backend server stuff? It's pretty much the ideal use case for Java. Java 7? meh. If you like chasing new shiny things all day, sure. Javascript is a great language, no doubt, but for me, it lacks a lot of the things Java does so well to ensure that things don't break, finding bugs easy, etc etc Why are so programmers so fashion conscious though? Do we write programs to solve problems, or do we write programs to be fashionable. If we write to solve problems, then use Java5. Use BASIC. Who cares if it solves the problem well. If you write to be fashionable, then sure, use <NEW SHINY THING THAT REINVENTS THE WHEEL BUT ISN'T UP TO MUCH YET>. People will marvel over your great fashion sense, but will it solve the problem even as well as a less fashionable approach? The amount of man hours wasted on reinventing the wheel in programming languages is staggering. So much time spent making a saw with a more comfortable handle that doesn't cut as well.
  - bad_user 16y ago
    > Why is startup time relevant if you're doing backend server stuff? It's pretty much the ideal use case for Java. Even on "backend server stuff" I pretty much prefer the FastCGI model, with multiple processes handling requests, as I don't have to worry about memory-leaks or about a rogue thread that can take out my whole server. With the JVM, this is impossible ... you need a monolithic process that takes care of everything (no support for forking, long startup time, high resource usage). And having multiple apps running in their own JVM (so they don't interfere with one another) is no panacea. You can also look at the problems Google's App Engine has with Java apps ... as resources are elastic, extra servers / VMs are loaded on demand, and the first requests hitting such a cold VM almost always ends up with a timeout error (although they are working to fix this, the magnitude of this issue is not the same with Python on GAE). And this is solely because of the slow warm-up time. > People will marvel over your great fashion sense, but will it solve the problem even as well as a less fashionable approach? I have ideas that I can prototype over a single weekend, which wouldn't be possible without the help of those people with "great fashion sense". > The amount of man hours wasted on reinventing the wheel in programming languages is staggering Why don't you teach people then how to spend their time better with constructive arguments? The easiest thing in the world is to dismiss / critique.
- swannodette 16y ago
  Counterpoints: 1) Javascript is the slowest moving language standard since Common Lisp 2) Node.js is a cool project that is heading very gradually towards adoptability for projects that aren't tech demos 3) JVM is mature enough to support other languages - can't say that about V8 and probably not for some time. JVM languages moving fast and in good/interesting directions. 4) Android 5) JVM startup time not an issue for web services, games, etc - only one-off shell scripts. 6) Memory use. Unless you're running on a mobile platform - who cares? Projects like Node.js will have a bright future with small agile teams. But no more so than Clojure or Scala. And unlike Node.js, I think Clojure and Scala certainly have the ability to scale up to heavy-lifting tasks that require concurrency, real perf, and mature libraries. I love JS in the browser. But on the server, I have yet to see anything that convinces me that JavaScript will ever really trump what the JVM offers.
  - 10ren 16y ago
    Aside on 5): if you make your shell scripts call an already-running JVM, it is amazingly fast (ie. it's a server). A very very simple hack for is to run the JVM in an xterm, inside rlwrap, and implement a rudimentary command-loop - then type your commands directly.
strlen 16y ago
First, I want to say that irrespective of its flaws, the JVM is an awesome piece of software. There's lot of really exciting work still going on: G1 collector, Azul's open source contributions, register based Dalvik VM. Ten years ago few people would have imagined distributed databases running on the JVM that are fast enough to saturate the network, industrial scale JVM based HPC and running 20gb sized heaps under heavy production load without any GC-related problems in sight. On another note, few people would have imagined Scala and Clojure either. So don't take what I am about to say as a "yet another rant from a developer bashing his tools" rant. There's great deal of optimization going into the JVM as a server-side VM, but unfortunately, there's less work going into the JVM as a client VM. Lot of the advanced options that exist on the server version of HotSpot aren't available on the client version. Azul's recently open sourced tweaks are also aimed at the server side market. I fully understand the reason for this: Microsoft dominates the "thick client" market, the money for Java is on the server side (with Javascript client code running in the browser). The few Java desktop apps that I use are development related: Yourkit profiler, JVisualVM, IntelliJ, Processing-based Arduino IDE and Eclipse CDT (for the excellent AVR plugin). They're actually excellent, but (with the exception of light-weight Processing/Arduino IDEs) they're aimed at developers who are familiar with the Java platform and don't mind occasional rough edges: I had to tune IntelliJ's GC settings (enabling CompressedOops on 64-bit Linux, using a 32-bit JVM and CMS collector on OS X, adjusting sizes of various generations etc...) to get it to stop locking up on me. Given I work on memory intensive Java and Scala applications, that's not a problem for me. Imagine, however, a word processor application that required this ("hey mom, it really is called `CompressedOops', one word, capital C and capital O!"). On the other hand Qt provides many utilities for C++ beyond the UI and has bindings for Ruby. It's cross platform between OS X, Linux and Windows. Qt and C++ would be the route I'd take were I to build a cross platform application I am skilled enough with valgrind and honestly don't find lack of a GC to be a major problem for me (especially for desktop programming, which doesn't frequently involve complex parallel algorithms that are difficult to implement with manual memory management). Qt's provides a great deal of what one expects from a high-level platform like the JVM: concurrency libraries beyond primitives, simplifications for building event/callback driven applications, additional collections and even an IoC container. If I didn't mind tying myself to Microsoft's standards, I'd also take a serious look at Mono: while Mono's GC is primitive compared to HotSpot when it comes to server side applications, I've yet to hear of people having to tune it in order to run Mono-based desktop apps such as (excellent) F-Spot; garbage collection is a long-ago solved problem, I find it hard to believe that client VMs couldn't come with sane defaults out of the box. In this way it reminds me of Linux in mid-90s/early 00s (before distributions like Ubuntu and extensive driver support): lot of work going into scaling to thousands of processors and new architectures, while getting support for desktop sound and video cards required recompiling a kernel. Finally, the author really strikes a point with lack of a POSIX API built in to the JVM. What's even more striking is that this can't even be explained by "focus on enterprise server market" theme. Server-side, Linux is almost a mono-culture with occasional use of Solaris or OS X (the latter frequently as development environment). The few non-POSIX platforms (Windows, mainframes) aren't really a large part of the Java market and have POSIX compatibility layers available for them (bonus point: why not support both, like Perl does -- having builtins for most system calls, but also an excellent Win32 module). POSIX-JNA is available and (from what I hear) is an excellent library, but its use of GPL makes it incompatible with Apache-licensed projects (ASL 2.0 being de-facto standard in the Java world). A minimal interface to POSIX, coupled with a "systemcall()" method (allowing easy use of Linux-specific extensions) should be the standard part of the JDK: Python and Perl offer this without sacrificing portability and safety, why can't Java?
stcredzero 16y ago
It seems like "tracing JIT" is really just "advanced compiler hints from runtime analysis". There's no reason this couldn't be gathered by tools and saved in a file to be used by compilers. Combine this with very advanced symbolic debugging and very rapid compilation, there would no longer be any need for late-binding dynamic languages. (Says the lover of Smalltalk, no less! It would have to be with debugging on the level of Smalltalk or IPython and then some.)
- headius 16y ago
  And true to form, on the JRuby project we're looking into ways to optimize around where the JVM doesn't quite serve us well. I've recently been experimenting with doing my own dynamic optimization passes, and that let me to think about how to save off this dynopt information to disk for instant gratification on future runs. I think part of the problem with the JVM is that its requirements have too long been driven by the big EE server folks, who have almost completely different requirements from day-to-day developers, client app developers, RIA developers, and so on. The key here might be making a good business case for those other domains, to help drive the JVM in the direction those domains need. I actually have hope that the (perhaps misguided) push for JavaFX at Sun/Oracle will bear fruit in the form of client-side and non-giant-server-app domains, since I know the JavaFX team have butted their heads up against a lot of the same problems we've faced in JRuby.
  - stcredzero 16y ago
    I was thinking about applying this to Go.
10ren 16y ago
When he says "object serialization" is broken, does he mean Java's built-in object serialization (JOS), or the concept of serialization, independent of any specific implementation? He later says "default serialization", which is what JOS's serialization of an object is called if its serialization hasn't been customized - but maybe he means JOS itself? Anyway: for JOS, you don't need to provide no-arg constructors and you don't need to un-final fields, because JOS extralinguistically both creates without constructors and sets final fields. JOS also provides hooks for you to initialize objects. It has many other other hooks that few people use. It's true that some aspects of JOS are ugly in how it implements what it does, but much of what it sets out to do is necessary for a full-featured serialization, that can work over the network. Serialization in other languages hasn't shown a "right" way to do it that I'm aware of. I'm interested to hear more about the problems Charles found with JOS, and whether he has misunderstandings about this arcane bit of java, or if it's me who's misunderstood his very brief aside on it here.
- headius 16y ago
  I mean the built-in serialization. Outside of the classloading and security hacks required to make it work, the fact is that it performs at its worst if you don't do things like provide a no-arg constructor and non-final fields. In those cases, the amount of reflective hackery required behind the scenes is absolutely dreadful, and in benchmarking a simple graph of objects recently I saw that 99% of the time was spent doing reflective access. That's absurd. Rewriting to use Externalizable was a painful process, but it was orders of magnitude faster than builtin serialization. The default serialization mechanism is basically unusably slow for any high-throughput purposes. I have not seen the hooks you describe for user-driven initialization of classes, and unfortunately most of the resources I consulted online while trying to write fast deserialization logic recently didn't mention them either. Got a link? I'm certainly willing to learn what I'm doing wrong.
  - 10ren 16y ago
    True, reflection is slower than regular access - though JOS does a fair bit of caching to avoid some of the cost (for when you serialize many instances of the same class). I think your experience with rewriting as Externalization shows that JOS is one of those performance vs. ease trade-offs. BTW: Are you sure that no-arg constructors and un-finaling fields makes a significant performance difference? The only instance I've come across for this showed the (surprising) result that JOS's setting final fields is actually faster than reflective setting of non-final fields - but I haven't profiled that explicitly. I would expect deserialization with a no-arg to be slower, because then it has to call the constructor in addition to actually creating the object (allocating memory etc). But I haven't profiled this either. You can customize the initialization of a class by creating a readObject() method for it, which is a sort of extra-linguistic constructor. It is confusingly named the same as the method that you call to start deserialization - but here it is a callback method that you write, that JOS itself will call: private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException { ois.defaultReadObject(); // "default" deserialization // ... your initialization here ... } You can also read data from the stream explicitly, and set the class fields yourself - if so, you need to write a corresponding writeObject() method that writes the data to the stream. By avoiding reflection, this should be faster than the "default" (but again I haven't profiled this). If you don't provide a readObject, it will call defaultReadObject() for you by, you know, default. Same for writeObject(). Googling for the specific methods turns up many tutorials. The following docs are for serialization in Java 1.3, but these "basics" haven't changed: for readObject() http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/input.doc4.html#2971 http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/i... (section 3.1 has the API, with defaultReadObject() in it). Here's the overall contents page: http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/serialTOC.doc.html http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/s... There's also hooks for validating the object graph after all of it has been deserialized; and for replacing (resolving) one object with another. I know something of JOS, but not much of its performance, which seems to be your crucial requirement. So I might not be much help, but let me know if you'd like more info or if I've misunderstood something (I might not reply til tomorrow - it's 3am here).

124 points by gthank 16y ago

donw 16y ago

I've been wondering for awhile if the JVM really has a lot of runway left; Java 7 has been amazingly slow in coming, and it'll be at least six months, and more likely a year, before even the early adopters can really use it in production. Startup time is a very real problem, as is memory use compared to other, similar VMs (V8, LLVM). With closures (lambda expressions), Java will be a lot more useful, but my money is honestly on V8 and JavaScript at this point. It's moving faster, and the node.js guys are very right that JavaScript is the language of the web.

sb 16y ago
Hi, just for the record, V8 differs in one very important aspect from the JVM: it includes no interpreter. It directly compiles methods to native code before their first execution, whereas the JVM includes a highly optimized interpreter for the first couple of executions. This affects memory usage, too (a couple of things that come to mind are profiling, additional memory for de-optimization, etc.)
- bad_user 16y ago
  I have a hard time believing that V8 doesn't do profiling / de-optimizations ... as these are crucial parts for optimizing runtime dispatch or boxing/unboxing of primitives. I know that V8 doesn't do tracing compiling, but still, it should at least have inline caches.
noelwelsh 16y ago
The JVM does a huge amount more than V8/node.js. If your world consists solely of shuffling bytes around a network then node.js may be a good solution, but it doesn't extend much further than that. For example, I wouldn't want to write a machine learning algorithm in Javascript, nor would I want to write a storage engine, nor a ... you get the idea. Furthermore, while node.js may be growing quickly (easy when you're small!) the development of the Javascript language is taking a rather torturous route as the various vendors play games at the Ecmascript table. To return to the article, what I really want from both the JVM and Javascript is tail recursion. I'd also like proper lexical scoping in Javascript, though that isn't so important. I don't view Java or Javascript as languages you write but rather languages you compile to (well, you compile to bytecode for the JVM, but hopefully you get my point). Scala and Clojure make mighty fine Java replacements; I haven't seen anything yet for Javascript that is much of an improvement.
- sanderjd 16y ago
  Could you list some reasons why you wouldn't want to write a machine learning algorithm in Javascript, or a write storage engine nor a ...? Lack of libraries? Too high level? Or do you not like the syntax/semantics/...?
  - jshen 16y ago
    threading maybe?
  - noelwelsh 16y ago
    Performance -- Javascript VMs are not optimised for numeric code. Lack of libraries. I don't particularly like JS for involved projects (e.g. no module system, so you have to write your own).
axod 16y ago
Why is startup time relevant if you're doing backend server stuff? It's pretty much the ideal use case for Java. Java 7? meh. If you like chasing new shiny things all day, sure. Javascript is a great language, no doubt, but for me, it lacks a lot of the things Java does so well to ensure that things don't break, finding bugs easy, etc etc Why are so programmers so fashion conscious though? Do we write programs to solve problems, or do we write programs to be fashionable. If we write to solve problems, then use Java5. Use BASIC. Who cares if it solves the problem well. If you write to be fashionable, then sure, use <NEW SHINY THING THAT REINVENTS THE WHEEL BUT ISN'T UP TO MUCH YET>. People will marvel over your great fashion sense, but will it solve the problem even as well as a less fashionable approach? The amount of man hours wasted on reinventing the wheel in programming languages is staggering. So much time spent making a saw with a more comfortable handle that doesn't cut as well.
- bad_user 16y ago
  > Why is startup time relevant if you're doing backend server stuff? It's pretty much the ideal use case for Java. Even on "backend server stuff" I pretty much prefer the FastCGI model, with multiple processes handling requests, as I don't have to worry about memory-leaks or about a rogue thread that can take out my whole server. With the JVM, this is impossible ... you need a monolithic process that takes care of everything (no support for forking, long startup time, high resource usage). And having multiple apps running in their own JVM (so they don't interfere with one another) is no panacea. You can also look at the problems Google's App Engine has with Java apps ... as resources are elastic, extra servers / VMs are loaded on demand, and the first requests hitting such a cold VM almost always ends up with a timeout error (although they are working to fix this, the magnitude of this issue is not the same with Python on GAE). And this is solely because of the slow warm-up time. > People will marvel over your great fashion sense, but will it solve the problem even as well as a less fashionable approach? I have ideas that I can prototype over a single weekend, which wouldn't be possible without the help of those people with "great fashion sense". > The amount of man hours wasted on reinventing the wheel in programming languages is staggering Why don't you teach people then how to spend their time better with constructive arguments? The easiest thing in the world is to dismiss / critique.
swannodette 16y ago
Counterpoints: 1) Javascript is the slowest moving language standard since Common Lisp 2) Node.js is a cool project that is heading very gradually towards adoptability for projects that aren't tech demos 3) JVM is mature enough to support other languages - can't say that about V8 and probably not for some time. JVM languages moving fast and in good/interesting directions. 4) Android 5) JVM startup time not an issue for web services, games, etc - only one-off shell scripts. 6) Memory use. Unless you're running on a mobile platform - who cares? Projects like Node.js will have a bright future with small agile teams. But no more so than Clojure or Scala. And unlike Node.js, I think Clojure and Scala certainly have the ability to scale up to heavy-lifting tasks that require concurrency, real perf, and mature libraries. I love JS in the browser. But on the server, I have yet to see anything that convinces me that JavaScript will ever really trump what the JVM offers.
- 10ren 16y ago
  Aside on 5): if you make your shell scripts call an already-running JVM, it is amazingly fast (ie. it's a server). A very very simple hack for is to run the JVM in an xterm, inside rlwrap, and implement a rudimentary command-loop - then type your commands directly.

strlen 16y ago

First, I want to say that irrespective of its flaws, the JVM is an awesome piece of software. There's lot of really exciting work still going on: G1 collector, Azul's open source contributions, register based Dalvik VM. Ten years ago few people would have imagined distributed databases running on the JVM that are fast enough to saturate the network, industrial scale JVM based HPC and running 20gb sized heaps under heavy production load without any GC-related problems in sight. On another note, few people would have imagined Scala and Clojure either. So don't take what I am about to say as a "yet another rant from a developer bashing his tools" rant. There's great deal of optimization going into the JVM as a server-side VM, but unfortunately, there's less work going into the JVM as a client VM. Lot of the advanced options that exist on the server version of HotSpot aren't available on the client version. Azul's recently open sourced tweaks are also aimed at the server side market. I fully understand the reason for this: Microsoft dominates the "thick client" market, the money for Java is on the server side (with Javascript client code running in the browser). The few Java desktop apps that I use are development related: Yourkit profiler, JVisualVM, IntelliJ, Processing-based Arduino IDE and Eclipse CDT (for the excellent AVR plugin). They're actually excellent, but (with the exception of light-weight Processing/Arduino IDEs) they're aimed at developers who are familiar with the Java platform and don't mind occasional rough edges: I had to tune IntelliJ's GC settings (enabling CompressedOops on 64-bit Linux, using a 32-bit JVM and CMS collector on OS X, adjusting sizes of various generations etc...) to get it to stop locking up on me. Given I work on memory intensive Java and Scala applications, that's not a problem for me. Imagine, however, a word processor application that required this ("hey mom, it really is called `CompressedOops', one word, capital C and capital O!"). On the other hand Qt provides many utilities for C++ beyond the UI and has bindings for Ruby. It's cross platform between OS X, Linux and Windows. Qt and C++ would be the route I'd take were I to build a cross platform application I am skilled enough with valgrind and honestly don't find lack of a GC to be a major problem for me (especially for desktop programming, which doesn't frequently involve complex parallel algorithms that are difficult to implement with manual memory management). Qt's provides a great deal of what one expects from a high-level platform like the JVM: concurrency libraries beyond primitives, simplifications for building event/callback driven applications, additional collections and even an IoC container. If I didn't mind tying myself to Microsoft's standards, I'd also take a serious look at Mono: while Mono's GC is primitive compared to HotSpot when it comes to server side applications, I've yet to hear of people having to tune it in order to run Mono-based desktop apps such as (excellent) F-Spot; garbage collection is a long-ago solved problem, I find it hard to believe that client VMs couldn't come with sane defaults out of the box. In this way it reminds me of Linux in mid-90s/early 00s (before distributions like Ubuntu and extensive driver support): lot of work going into scaling to thousands of processors and new architectures, while getting support for desktop sound and video cards required recompiling a kernel. Finally, the author really strikes a point with lack of a POSIX API built in to the JVM. What's even more striking is that this can't even be explained by "focus on enterprise server market" theme. Server-side, Linux is almost a mono-culture with occasional use of Solaris or OS X (the latter frequently as development environment). The few non-POSIX platforms (Windows, mainframes) aren't really a large part of the Java market and have POSIX compatibility layers available for them (bonus point: why not support both, like Perl does -- having builtins for most system calls, but also an excellent Win32 module). POSIX-JNA is available and (from what I hear) is an excellent library, but its use of GPL makes it incompatible with Apache-licensed projects (ASL 2.0 being de-facto standard in the Java world). A minimal interface to POSIX, coupled with a "systemcall()" method (allowing easy use of Linux-specific extensions) should be the standard part of the JDK: Python and Perl offer this without sacrificing portability and safety, why can't Java?

stcredzero 16y ago

It seems like "tracing JIT" is really just "advanced compiler hints from runtime analysis". There's no reason this couldn't be gathered by tools and saved in a file to be used by compilers. Combine this with very advanced symbolic debugging and very rapid compilation, there would no longer be any need for late-binding dynamic languages. (Says the lover of Smalltalk, no less! It would have to be with debugging on the level of Smalltalk or IPython and then some.)

headius 16y ago
And true to form, on the JRuby project we're looking into ways to optimize around where the JVM doesn't quite serve us well. I've recently been experimenting with doing my own dynamic optimization passes, and that let me to think about how to save off this dynopt information to disk for instant gratification on future runs. I think part of the problem with the JVM is that its requirements have too long been driven by the big EE server folks, who have almost completely different requirements from day-to-day developers, client app developers, RIA developers, and so on. The key here might be making a good business case for those other domains, to help drive the JVM in the direction those domains need. I actually have hope that the (perhaps misguided) push for JavaFX at Sun/Oracle will bear fruit in the form of client-side and non-giant-server-app domains, since I know the JavaFX team have butted their heads up against a lot of the same problems we've faced in JRuby.
- stcredzero 16y ago
  I was thinking about applying this to Go.

10ren 16y ago

When he says "object serialization" is broken, does he mean Java's built-in object serialization (JOS), or the concept of serialization, independent of any specific implementation? He later says "default serialization", which is what JOS's serialization of an object is called if its serialization hasn't been customized - but maybe he means JOS itself? Anyway: for JOS, you don't need to provide no-arg constructors and you don't need to un-final fields, because JOS extralinguistically both creates without constructors and sets final fields. JOS also provides hooks for you to initialize objects. It has many other other hooks that few people use. It's true that some aspects of JOS are ugly in how it implements what it does, but much of what it sets out to do is necessary for a full-featured serialization, that can work over the network. Serialization in other languages hasn't shown a "right" way to do it that I'm aware of. I'm interested to hear more about the problems Charles found with JOS, and whether he has misunderstandings about this arcane bit of java, or if it's me who's misunderstood his very brief aside on it here.

headius 16y ago
I mean the built-in serialization. Outside of the classloading and security hacks required to make it work, the fact is that it performs at its worst if you don't do things like provide a no-arg constructor and non-final fields. In those cases, the amount of reflective hackery required behind the scenes is absolutely dreadful, and in benchmarking a simple graph of objects recently I saw that 99% of the time was spent doing reflective access. That's absurd. Rewriting to use Externalizable was a painful process, but it was orders of magnitude faster than builtin serialization. The default serialization mechanism is basically unusably slow for any high-throughput purposes. I have not seen the hooks you describe for user-driven initialization of classes, and unfortunately most of the resources I consulted online while trying to write fast deserialization logic recently didn't mention them either. Got a link? I'm certainly willing to learn what I'm doing wrong.
- 10ren 16y ago
  True, reflection is slower than regular access - though JOS does a fair bit of caching to avoid some of the cost (for when you serialize many instances of the same class). I think your experience with rewriting as Externalization shows that JOS is one of those performance vs. ease trade-offs. BTW: Are you sure that no-arg constructors and un-finaling fields makes a significant performance difference? The only instance I've come across for this showed the (surprising) result that JOS's setting final fields is actually faster than reflective setting of non-final fields - but I haven't profiled that explicitly. I would expect deserialization with a no-arg to be slower, because then it has to call the constructor in addition to actually creating the object (allocating memory etc). But I haven't profiled this either. You can customize the initialization of a class by creating a readObject() method for it, which is a sort of extra-linguistic constructor. It is confusingly named the same as the method that you call to start deserialization - but here it is a callback method that you write, that JOS itself will call: private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException { ois.defaultReadObject(); // "default" deserialization // ... your initialization here ... } You can also read data from the stream explicitly, and set the class fields yourself - if so, you need to write a corresponding writeObject() method that writes the data to the stream. By avoiding reflection, this should be faster than the "default" (but again I haven't profiled this). If you don't provide a readObject, it will call defaultReadObject() for you by, you know, default. Same for writeObject(). Googling for the specific methods turns up many tutorials. The following docs are for serialization in Java 1.3, but these "basics" haven't changed: for readObject() http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/input.doc4.html#2971 http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/i... (section 3.1 has the API, with defaultReadObject() in it). Here's the overall contents page: http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/serialTOC.doc.html http://java.sun.com/j2se/1.3/docs/guide/serialization/spec/s... There's also hooks for validating the object graph after all of it has been deserialized; and for replacing (resolving) one object with another. I know something of JOS, but not much of its performance, which seems to be your crucial requirement. So I might not be much help, but let me know if you'd like more info or if I've misunderstood something (I might not reply til tomorrow - it's 3am here).