3 ms·

Except for the fact that JNI is such a piece of utter... garbage ... as far as performance is concerned. One has to think twice, thrice ...countless times befor

by srean 11y ago

Except for the fact that JNI is such a piece of utter... garbage ... as far as performance is concerned. One has to think twice, thrice ...countless times before jumping back and forth over the runtime bridge of JVM and native. Not that Python is that good at it either, but better than JNI, almost everything is. The best I have seen is Lua's FFI. I consider it an accident of history that Numpy, Scipy, Pandas, Scikits got written for Python and not Lua. Thanks to Luajit I think Lua would have been a better choice. Now that cause has been taken up by Torch and Julia. JVM by itself is terrible for reaching close to the FLOPS that the CPU is capable of. Try sparse matrix multiply with it and see it for yourself.

AlphaSite 11y ago
I think Swift's seamless bridging is pretty damn good as well.
- astrafoo 11y ago
  A bridge is always easier to build if there is no river in between.
eonwe 11y ago
Could you elaborate a bit on how JVM hinders reaching maximum FLOPS when multiplying sparse matrices? I could think some examples where lacking SSE/AVX support would hinder it, but I don't see the connection with sparse matrices.
- srean 11y ago
  What make sparse matrices harder to JIT is proving that the loop bounds will not be exceeded, so all accesses re bound checked. In any case in my experience array bounds checking and escape analysis never gave the boost that theory and JVM fans promise. So even normal matrix multiply will trail behind. That said Hotspot JVM is possibly one of the most optimized VMs we have got. A structural problem of JVM is that its runtime semantics is over-specified, there is very little room for the JIT to do its stuff. For example function arguments are evaluated right to left, there goes an opportunity for parallelism.
  - eonwe 11y ago
    For normal dense matrix laid out as a double[] and accessed directly as i* N_ROW + j probably won't get its loop check elided. For double[][] I would _think_ that happens more easily. But how are sparse matrices then generally laid out? A naive approach would be some hash map, perhaps with some locality, in which I don't see JIT problems.
    - srean 11y ago
      There are some defacto standard formats such as CRS, CSC, list of tuples etc. Layout of the third should be obvious and it is not used much for cases where speed matters because one loses locality in this layout. For the other two they are laid out column after column (or row by row), row ids, and offsets to indicate the start and end of columns (rows).
      - eonwe 11y ago
        Thanks, this helped. Using CSC and CRS is probably quite problematic with JVM bounds check elimination (or the lack of it). So if they would need to be used on JVM, I think it would be wise to drop the safety and use _sun.misc.Unsafe_ for unchecked array access.
bipin_nag 11y ago
Correct me if I am wrong but Spark does not use JNI for Python. It uses py4j to allow Python programs to access Java Objects. All the core functionality is implemented in Scala/Java. py4j uses sockets to communicate. See https://www.py4j.org/about.html https://www.py4j.org/about.html
- srean 11y ago
  That's my understanding as well. This is the reason why you are caught between the rock and a hard place on the Hadoop stack. Java code wont be efficient in terms of FLOPs and a popular and often effective escape hatch: code number crunching parts in C, C++ or Fortran, is also not very effective / convenient. Particularly so if it requires going back and forth over the bridge frequently because crossing the bridge has significant overhead. So typically you have to move majority of the core into the high performant language of choice and what remains is glue. If gluing is what I want to do, there are other languages that can give Java stiff competition. The core capability thing going for Java in this domain is HDFS, and its not that great a file system for big data.
  - lmm 11y ago
    When we say JVM code is not efficient in terms of FLOPs we're talking about a factor of 2 or so though, not really the same as the factor of 20+ you lose by implementing numeric code in pure Python. So Scala gives you a 10x faster cycle during early development. Maybe it's fast enough that you don't need that last factor of 2, or maybe just JNIing a few core operations is enough (you can also do some tricks with the advanced type system; with NumPy if it looks like Python iteration then it is, whereas with something like Breeze you can potentially have something that looks like Scala iteration but won't actually copy the data back and forth). But in the cases where you do need to push things right down into Fortran, Scala is just as good a glue language as Python is.
    - srean 11y ago
      > When we say JVM code is not efficient in terms of FLOPs we're talking about a factor of 2 or so though, not really the same as the factor of 20+ Not so in my experience. Write a matrix multiply in pure Java, and pure C, C++, Fortran. The difference would easily be north of 5X, typically more. Regarding JNI, please see the root of the comment tree. Rarely have I seen more stinky garbage. Its hopeless if you have to go back and forth across the bridge. If that be so I might as well be on the other side of the bridge. JVM is fantastic, but if you are doing number crunching and performance matters, then JVM is a wrong choice.

by srean 11y ago

AlphaSite 11y ago

I think Swift's seamless bridging is pretty damn good as well.

astrafoo 11y ago
A bridge is always easier to build if there is no river in between.

eonwe 11y ago

Could you elaborate a bit on how JVM hinders reaching maximum FLOPS when multiplying sparse matrices? I could think some examples where lacking SSE/AVX support would hinder it, but I don't see the connection with sparse matrices.

srean 11y ago
What make sparse matrices harder to JIT is proving that the loop bounds will not be exceeded, so all accesses re bound checked. In any case in my experience array bounds checking and escape analysis never gave the boost that theory and JVM fans promise. So even normal matrix multiply will trail behind. That said Hotspot JVM is possibly one of the most optimized VMs we have got. A structural problem of JVM is that its runtime semantics is over-specified, there is very little room for the JIT to do its stuff. For example function arguments are evaluated right to left, there goes an opportunity for parallelism.
- eonwe 11y ago
  For normal dense matrix laid out as a double[] and accessed directly as i* N_ROW + j probably won't get its loop check elided. For double[][] I would _think_ that happens more easily. But how are sparse matrices then generally laid out? A naive approach would be some hash map, perhaps with some locality, in which I don't see JIT problems.
  - srean 11y ago
    There are some defacto standard formats such as CRS, CSC, list of tuples etc. Layout of the third should be obvious and it is not used much for cases where speed matters because one loses locality in this layout. For the other two they are laid out column after column (or row by row), row ids, and offsets to indicate the start and end of columns (rows).
    - eonwe 11y ago
      Thanks, this helped. Using CSC and CRS is probably quite problematic with JVM bounds check elimination (or the lack of it). So if they would need to be used on JVM, I think it would be wise to drop the safety and use _sun.misc.Unsafe_ for unchecked array access.

bipin_nag 11y ago

Correct me if I am wrong but Spark does not use JNI for Python. It uses py4j to allow Python programs to access Java Objects. All the core functionality is implemented in Scala/Java. py4j uses sockets to communicate. See https://www.py4j.org/about.html https://www.py4j.org/about.html

srean 11y ago
That's my understanding as well. This is the reason why you are caught between the rock and a hard place on the Hadoop stack. Java code wont be efficient in terms of FLOPs and a popular and often effective escape hatch: code number crunching parts in C, C++ or Fortran, is also not very effective / convenient. Particularly so if it requires going back and forth over the bridge frequently because crossing the bridge has significant overhead. So typically you have to move majority of the core into the high performant language of choice and what remains is glue. If gluing is what I want to do, there are other languages that can give Java stiff competition. The core capability thing going for Java in this domain is HDFS, and its not that great a file system for big data.
- lmm 11y ago
  When we say JVM code is not efficient in terms of FLOPs we're talking about a factor of 2 or so though, not really the same as the factor of 20+ you lose by implementing numeric code in pure Python. So Scala gives you a 10x faster cycle during early development. Maybe it's fast enough that you don't need that last factor of 2, or maybe just JNIing a few core operations is enough (you can also do some tricks with the advanced type system; with NumPy if it looks like Python iteration then it is, whereas with something like Breeze you can potentially have something that looks like Scala iteration but won't actually copy the data back and forth). But in the cases where you do need to push things right down into Fortran, Scala is just as good a glue language as Python is.
  - srean 11y ago
    > When we say JVM code is not efficient in terms of FLOPs we're talking about a factor of 2 or so though, not really the same as the factor of 20+ Not so in my experience. Write a matrix multiply in pure Java, and pure C, C++, Fortran. The difference would easily be north of 5X, typically more. Regarding JNI, please see the root of the comment tree. Rarely have I seen more stinky garbage. Its hopeless if you have to go back and forth across the bridge. If that be so I might as well be on the other side of the bridge. JVM is fantastic, but if you are doing number crunching and performance matters, then JVM is a wrong choice.