12 ms·

135M messages a second between processes in Java (2013)

262 points by matteuan 10y ago

atemerev 10y ago
Java in HFT environment is wonderfully perverse and beautifully abused in so many ways.
- oppositelock 10y ago
  Yeah, it's amazing how fast it goes when you give it tons of RAM and turn off GC.
  - bitmapbrother 10y ago
    Or you could just use Azul Systems pauseless JVM and not have to worry about the GC ever.
leeoniya 10y ago
(2013)
- biot 10y ago
  Has this metric been surpassed since?
  - srtjstjsj 10y ago
    Of course. Since this is a hardware-bound metric, newer faster harder can transmit more messages. A more timeless metric would be something like "30-instruction message-passing on x86"
    - tantalor 10y ago
      Is this sort of code purely bound by # CPU instructions? Surely other aspects of hardware could also improve.
    - biot 10y ago
      Rephrased: given the same hardware, has anyone found a technique that results in greater throughput?
      - geocar 10y ago
        Yes, absolutely. I use KDB to get billions-of-transactions-per-second, but then an interpreted language is clearly better than the JVM. Put grossly, the "technique" isn't well defined; Even things like "message" aren't well defined, and at the end of the day, are we really talking about a new-and-improved memcpy[1]? Or are we talking about something that's less-slow than other things (and if so, what?) Not saying this isn't interesting, just that I don't know if it's interesting. Something I want to know is when Java programmers struggle to get into 200k/sec HTTP requests, can I point them at this? [1]: http://www.codeproject.com/Articles/1110153/Apex-memmove-the-fastest-memcpy-memmove-on-x-x-EVE http://www.codeproject.com/Articles/1110153/Apex-memmove-the...
carsongross 10y ago
The JVM is a treasure just sitting there waiting to be rediscovered. It really is a shame that there is so much noise and unnecessary complexity around using it.
- taeric 10y ago
  This is true of most any environment. When speed is a requirement, people can make things fast. For most of us. Speed is not a requirement.
  - gozur88 10y ago
    Or it is but the slow part is the database. I wonder what percentage of business applications are some kind of RDB CRUD system - get some rows, put them in a table, let the user update or delete, send them on to the next system... When you're doing that kind of application the overhead the language adds is lost in the DB noise floor.
- mablap 10y ago
  For me, as an outsider, the JVM looks a bit like .NET: this huge, extremely complex codebase which does everything and anything, very optimized and comes with great tools. But to me that is a problem. Even though the JVM could do X or Y better than another platform, just the time spent learning about it feels like a waste of time, as I could already be productive in another language which is less optimized and less complex, but much faster to develop with. It seems just learning to use a Java IDE is a job in itself. I do mathematical modeling, by the way. If I need speed, I use optimised libraries. So I might not be the target user of Java, although I've often heard good things about it for scientific computing.
  - formula1 10y ago
    Java really isnt that bad. I think the biggest issue surrounding it is a package manager, better ui/templating language and a more obvious/documented import and linking api. Outside of that it isnt too bad Edit : also events were annoying in 7. I dont have a particular interest in creating a class just to act as a listener. I believe 8 introduced lambdas which is nice
xchip 10y ago
Moving blocks of memory around is something I am expecting java to do quite fast, just because the VM has intrinsic instructions to do that. You could do the same thing in any other language.
- kasey_junk 10y ago
  The thing to remember though Is that these libraries are type safe, fast & general purpose. I bemoan the lack of a similar library in go for instance because the lack of generics. You can certainly do the same thing in C as well, but I can't easily mix it with code that has automatic memory management. As for dynamic languages, can you actually do this? I'm not saying its impossible, I've just never seen it.
  - srtjstjsj 10y ago
    > these libraries are type safe, not in this blog post: UnsafeAccess.unsafe UnsafeDirectByteBuffer.getAddress
    - kasey_junk 10y ago
      Those are wrapped in type safe methods though (all type safety cross process has this step regardless of language/lib). In nontrivial examples you tend to wrap more than Java longs but the techniques are the same. EDIT: I realize now that I was bringing a lot of context with me that is not obvious just from the blog post. Martin Thompson and Todd Montgomery who are mentioned in this post (and Richard Warburton who isn't). Are the main authors of the real-logic libraries, which are extremely well known in JVM performance circles. Aeron in particular is a great IPC lib. Other libraries that are similar and in the same circle are Open-HFT Chronicle and the LMAX Disruptor. For that matter the author of this blog post @nitsanw is pretty well known in that arena.
    - slantedview 10y ago
      Unsafe is usually used for allocating memory directly rather than having it be managed by the runtime. This has nothing to do with type safety.
srtjstjsj 10y ago
Is there any reason someone would want to run multiple Java processes communicating at high speed? It seems useless, since the massive overhead of each process makes IPC useless in a high-performance environment.
- kasey_junk 10y ago
  One nice thing is you can segment GC sensitive tasks from non sensitive tasks. Another is core affinity. Another is 1 process that is "native" and another is JVM.
- noselasd 10y ago
  I'm not sure why you think there is massive overhead, can you explain a bit more about that ?
  - ketralnis 10y ago
    Not the parent but the biggest overheads are going to be startup time (which is easy to work around by just bringing up the procs and keeping them there) and memory overhead. There's some extra memory used for keeping all of the code or other shared resources around but let's assume that's negligible. Because the JVM never gives memory back to the OS, you have to give it the maximum amount of memory that it could ever want, plus enough to last you between GC runs. Often this means that you have to give it at least twice the memory it actually needs. If you have more than one process, you end up having to multiply that extra "slack" memory by the number of processes that you have, which means that you have all of this wasted memory sitting around all of the time. "Memory is cheap" is a common attitude but in my experience in web stuff it's the only resource that's anywhere close to 100% used on an app server. So things with high memory overhead mean that you need more machines or bigger ones. Of course, whether that matters depends on the actual numbers of how much memory is needed, how much is "wasted", whether that's actually your bottleneck, etc.
    - kasey_junk 10y ago
      In my experience on the web its all IO bound. If you are using multiple JVMs and high performance IPC libraries you have largely already solved the memory bound problem on the JVM with standard techniques like arena allocation.
bogomipz 10y ago
Can someone elaborate on this: "IPC, what's the problem? Inter Process Communication is an old problem and there are many ways to solve it (which I will not discuss here)." How is IPC an old problem and how was it solved?
- matmo 10y ago
  I think the two main solutions are pipes and shared memory.
- tlb 10y ago
  It was a big focus of the Mach kernel in the early 90s. It was a microkernel, meaning that they moved as much functionality as possible out of the kernel and into user processes. For example, to access the file system each call (open, read, write) was really a message to a file system server process. So IPC performance mattered a lot. Here's an example some heroic attempts to make it fast, getting down to about 50 uS on a 1991 CPU: http://homes.cs.washington.edu/~bershad/Papers/p175-bershad.pdf http://homes.cs.washington.edu/~bershad/Papers/p175-bershad....
  - bogomipz 10y ago
    Thanks for the link.
jpalomaki 10y ago
There's also more light weight alternatives to "XML heavy traditional frameworks" on Java side. Check for example Play[1], which makes the "write some code, test in browser, write more, test" cycle possible. https://www.playframework.com https://www.playframework.com
option_greek 10y ago
With all those memory accesses marked as unsafe, I'm wondering why not just use C++.

262 points by matteuan 10y ago

atemerev 10y ago

Java in HFT environment is wonderfully perverse and beautifully abused in so many ways.

oppositelock 10y ago
Yeah, it's amazing how fast it goes when you give it tons of RAM and turn off GC.
- bitmapbrother 10y ago
  Or you could just use Azul Systems pauseless JVM and not have to worry about the GC ever.

leeoniya 10y ago

(2013)

biot 10y ago
Has this metric been surpassed since?
- srtjstjsj 10y ago
  Of course. Since this is a hardware-bound metric, newer faster harder can transmit more messages. A more timeless metric would be something like "30-instruction message-passing on x86"
  - tantalor 10y ago
    Is this sort of code purely bound by # CPU instructions? Surely other aspects of hardware could also improve.
  - biot 10y ago
    Rephrased: given the same hardware, has anyone found a technique that results in greater throughput?
    - geocar 10y ago
      Yes, absolutely. I use KDB to get billions-of-transactions-per-second, but then an interpreted language is clearly better than the JVM. Put grossly, the "technique" isn't well defined; Even things like "message" aren't well defined, and at the end of the day, are we really talking about a new-and-improved memcpy[1]? Or are we talking about something that's less-slow than other things (and if so, what?) Not saying this isn't interesting, just that I don't know if it's interesting. Something I want to know is when Java programmers struggle to get into 200k/sec HTTP requests, can I point them at this? [1]: http://www.codeproject.com/Articles/1110153/Apex-memmove-the-fastest-memcpy-memmove-on-x-x-EVE http://www.codeproject.com/Articles/1110153/Apex-memmove-the...

carsongross 10y ago

The JVM is a treasure just sitting there waiting to be rediscovered. It really is a shame that there is so much noise and unnecessary complexity around using it.

taeric 10y ago
This is true of most any environment. When speed is a requirement, people can make things fast. For most of us. Speed is not a requirement.
- gozur88 10y ago
  Or it is but the slow part is the database. I wonder what percentage of business applications are some kind of RDB CRUD system - get some rows, put them in a table, let the user update or delete, send them on to the next system... When you're doing that kind of application the overhead the language adds is lost in the DB noise floor.
mablap 10y ago
For me, as an outsider, the JVM looks a bit like .NET: this huge, extremely complex codebase which does everything and anything, very optimized and comes with great tools. But to me that is a problem. Even though the JVM could do X or Y better than another platform, just the time spent learning about it feels like a waste of time, as I could already be productive in another language which is less optimized and less complex, but much faster to develop with. It seems just learning to use a Java IDE is a job in itself. I do mathematical modeling, by the way. If I need speed, I use optimised libraries. So I might not be the target user of Java, although I've often heard good things about it for scientific computing.
- formula1 10y ago
  Java really isnt that bad. I think the biggest issue surrounding it is a package manager, better ui/templating language and a more obvious/documented import and linking api. Outside of that it isnt too bad Edit : also events were annoying in 7. I dont have a particular interest in creating a class just to act as a listener. I believe 8 introduced lambdas which is nice

xchip 10y ago

Moving blocks of memory around is something I am expecting java to do quite fast, just because the VM has intrinsic instructions to do that. You could do the same thing in any other language.

kasey_junk 10y ago
The thing to remember though Is that these libraries are type safe, fast & general purpose. I bemoan the lack of a similar library in go for instance because the lack of generics. You can certainly do the same thing in C as well, but I can't easily mix it with code that has automatic memory management. As for dynamic languages, can you actually do this? I'm not saying its impossible, I've just never seen it.
- srtjstjsj 10y ago
  > these libraries are type safe, not in this blog post: UnsafeAccess.unsafe UnsafeDirectByteBuffer.getAddress
  - kasey_junk 10y ago
    Those are wrapped in type safe methods though (all type safety cross process has this step regardless of language/lib). In nontrivial examples you tend to wrap more than Java longs but the techniques are the same. EDIT: I realize now that I was bringing a lot of context with me that is not obvious just from the blog post. Martin Thompson and Todd Montgomery who are mentioned in this post (and Richard Warburton who isn't). Are the main authors of the real-logic libraries, which are extremely well known in JVM performance circles. Aeron in particular is a great IPC lib. Other libraries that are similar and in the same circle are Open-HFT Chronicle and the LMAX Disruptor. For that matter the author of this blog post @nitsanw is pretty well known in that arena.
  - slantedview 10y ago
    Unsafe is usually used for allocating memory directly rather than having it be managed by the runtime. This has nothing to do with type safety.

srtjstjsj 10y ago

Is there any reason someone would want to run multiple Java processes communicating at high speed? It seems useless, since the massive overhead of each process makes IPC useless in a high-performance environment.

kasey_junk 10y ago
One nice thing is you can segment GC sensitive tasks from non sensitive tasks. Another is core affinity. Another is 1 process that is "native" and another is JVM.
noselasd 10y ago
I'm not sure why you think there is massive overhead, can you explain a bit more about that ?
- ketralnis 10y ago
  Not the parent but the biggest overheads are going to be startup time (which is easy to work around by just bringing up the procs and keeping them there) and memory overhead. There's some extra memory used for keeping all of the code or other shared resources around but let's assume that's negligible. Because the JVM never gives memory back to the OS, you have to give it the maximum amount of memory that it could ever want, plus enough to last you between GC runs. Often this means that you have to give it at least twice the memory it actually needs. If you have more than one process, you end up having to multiply that extra "slack" memory by the number of processes that you have, which means that you have all of this wasted memory sitting around all of the time. "Memory is cheap" is a common attitude but in my experience in web stuff it's the only resource that's anywhere close to 100% used on an app server. So things with high memory overhead mean that you need more machines or bigger ones. Of course, whether that matters depends on the actual numbers of how much memory is needed, how much is "wasted", whether that's actually your bottleneck, etc.
  - kasey_junk 10y ago
    In my experience on the web its all IO bound. If you are using multiple JVMs and high performance IPC libraries you have largely already solved the memory bound problem on the JVM with standard techniques like arena allocation.

bogomipz 10y ago

Can someone elaborate on this: "IPC, what's the problem? Inter Process Communication is an old problem and there are many ways to solve it (which I will not discuss here)." How is IPC an old problem and how was it solved?

matmo 10y ago
I think the two main solutions are pipes and shared memory.
tlb 10y ago
It was a big focus of the Mach kernel in the early 90s. It was a microkernel, meaning that they moved as much functionality as possible out of the kernel and into user processes. For example, to access the file system each call (open, read, write) was really a message to a file system server process. So IPC performance mattered a lot. Here's an example some heroic attempts to make it fast, getting down to about 50 uS on a 1991 CPU: http://homes.cs.washington.edu/~bershad/Papers/p175-bershad.pdf http://homes.cs.washington.edu/~bershad/Papers/p175-bershad....
- bogomipz 10y ago
  Thanks for the link.

jpalomaki 10y ago

There's also more light weight alternatives to "XML heavy traditional frameworks" on Java side. Check for example Play[1], which makes the "write some code, test in browser, write more, test" cycle possible. https://www.playframework.com https://www.playframework.com

option_greek 10y ago

With all those memory accesses marked as unsafe, I'm wondering why not just use C++.