4 ms·

DL4J contributor here. I spoke with our team so the differences in performance are likely explained by array ordering. ND4J, for good reason, requires F orderin

DL4J contributor here. I spoke with our team so the differences in performance are likely explained by array ordering. ND4J, for good reason, requires F ordering because of limitations in cuBLAS. While I haven't had the opportunity to closely examine the Neanderthal comparison (I'm also not a clojure user), the likely explanation is that there's an implicit ordering impacting this. Deeplearning4j is written around F ordering and ND4J supports this. Admittedly, our ordering API is not obvious to the average user. Here's an example test you can run yourself that demonstrates ordering: https://gist.github.com/raver119/92b615704ca1bf169aa23a6a6e7d9880 https://gist.github.com/raver119/92b615704ca1bf169aa23a6a6e7... o.n.i.TensorFlowImportTest - Orders: CCC; Time: 11532 ns; o.n.i.TensorFlowImportTest - Orders: CCF; Time: 2101 ns; o.n.i.TensorFlowImportTest - Orders: CFC; Time: 10202 ns; o.n.i.TensorFlowImportTest - Orders: CFF; Time: 1960 ns; o.n.i.TensorFlowImportTest - Orders: FCC; Time: 10744 ns; o.n.i.TensorFlowImportTest - Orders: FCF; Time: 1717 ns; o.n.i.TensorFlowImportTest - Orders: FFC; Time: 10097 ns; o.n.i.TensorFlowImportTest - Orders: FFF; Time: 1716 ns; We also profiled the above test and confirmed that F -> C ordering adds significant overhead. I can share screenshots if anyone is interested.

dragandj 8y ago
While this is a good explanation, keep in mind that: 1. This is the benchmark that you provided, so while this might be not obvious to the average user, it also seems to not be obvious to the above average user that wrote the benchmark. I was just using what you proposed, assuming that you used the right thing in your library. 2. It still does not explain how you got better performance with ND4J with the same non-optimal call, which was what started the discussion, and inspired this post. 3. Neanderthal supports both Row and Column oriented order with cuBLAS with the same performance, and won't have those problems that you mention for ND4J. I'm, of course, interested in following up on this. Please decide on what cases you'd like to compare, post the (optimal) code, and the ND4J and Neanderthal numbers that you get, and I'll respond with my comments.
- treo 8y ago
  It looks like while converting from my benchmarking code you've dropped the 'f' when creating the resulting array. https://github.com/treo/benchmarking_nd4j/blob/master/src/main/java/com/example/neanderthal/NeanderthalComparision_1024x1024.java#L17 https://github.com/treo/benchmarking_nd4j/blob/master/src/ma... The difference is rather huge with the newer versions of nd4j. While the numbers in the following gists do not contain the measurements I took for neanderthal, they do contain the numbers that I got for ND4J. Without f ordering: https://gist.github.com/treo/1fab39f213da26255cf4f75e383ff908 https://gist.github.com/treo/1fab39f213da26255cf4f75e383ff90... With f ordering: https://gist.github.com/treo/94fe92c9417b5c8b24baa12924a35b04 https://gist.github.com/treo/94fe92c9417b5c8b24baa12924a35b0... As you can see something happened in the time between the 0.4 release (I took that as the comparison point since that was when I ran my own benchmarks the last time) and the 0.9.1 release that introduced additional overhead. Originally I planned to create my own write-up on this, but I wanted to first to find out what happened there. Given that ND4J is mainly used inside of DL4J and the matrix sizes it is used with usually are rather large, the performance overhead difference that I've observed there for tiny multiplications isn't necessarily that bad, as the newer version performs much better on larger matrices.
  - dragandj 8y ago
    You're right. In that particular case, ND4J comes to Neanderthal's speed. But only in that particular case; and even then ND4J is still not faster than Neanderthal. My initial quest was to find out whether ND4J can be faster than Neanderthal, and I still couldn't find a case where it is. Although, to my defense, the option in question here is very poorly documented. I've found the ND4J tutorial page where it's mentioned, and even after re-reading the sentence multiple times, I still do not connect its description to what it (seems to) actually do. It also does not mention that it affects computation speed. Anyway, I'm looking forward to reading your detailed analysis, and especially seeing your Neanderthal numbers.
    - treo 8y ago
      Do you have any pointer on how you've profiled Neanderthal during development? When I originally set out to compare ND4J and Neanderthal, I've ran into the issue that I bottomed out at: they basically both call MKL (or Openblas) for BLAS operations.
zz34 8y ago
When will you just admit the initial claim about ND4J being faster than Neanderthal was bogus?
vonnik 8y ago
Here's our updated benchmarks. Thanks to Dragan for the cooperation! Benchmarking ND4J and Neanderthal (Scientific Computing in Java and Clojure) https://www.dubs.tech/blog/benchmarking-nd4j-and-neanderthal/ https://www.dubs.tech/blog/benchmarking-nd4j-and-neanderthal...

dragandj 8y ago

While this is a good explanation, keep in mind that: 1. This is the benchmark that you provided, so while this might be not obvious to the average user, it also seems to not be obvious to the above average user that wrote the benchmark. I was just using what you proposed, assuming that you used the right thing in your library. 2. It still does not explain how you got better performance with ND4J with the same non-optimal call, which was what started the discussion, and inspired this post. 3. Neanderthal supports both Row and Column oriented order with cuBLAS with the same performance, and won't have those problems that you mention for ND4J. I'm, of course, interested in following up on this. Please decide on what cases you'd like to compare, post the (optimal) code, and the ND4J and Neanderthal numbers that you get, and I'll respond with my comments.

treo 8y ago
It looks like while converting from my benchmarking code you've dropped the 'f' when creating the resulting array. https://github.com/treo/benchmarking_nd4j/blob/master/src/main/java/com/example/neanderthal/NeanderthalComparision_1024x1024.java#L17 https://github.com/treo/benchmarking_nd4j/blob/master/src/ma... The difference is rather huge with the newer versions of nd4j. While the numbers in the following gists do not contain the measurements I took for neanderthal, they do contain the numbers that I got for ND4J. Without f ordering: https://gist.github.com/treo/1fab39f213da26255cf4f75e383ff908 https://gist.github.com/treo/1fab39f213da26255cf4f75e383ff90... With f ordering: https://gist.github.com/treo/94fe92c9417b5c8b24baa12924a35b04 https://gist.github.com/treo/94fe92c9417b5c8b24baa12924a35b0... As you can see something happened in the time between the 0.4 release (I took that as the comparison point since that was when I ran my own benchmarks the last time) and the 0.9.1 release that introduced additional overhead. Originally I planned to create my own write-up on this, but I wanted to first to find out what happened there. Given that ND4J is mainly used inside of DL4J and the matrix sizes it is used with usually are rather large, the performance overhead difference that I've observed there for tiny multiplications isn't necessarily that bad, as the newer version performs much better on larger matrices.
- dragandj 8y ago
  You're right. In that particular case, ND4J comes to Neanderthal's speed. But only in that particular case; and even then ND4J is still not faster than Neanderthal. My initial quest was to find out whether ND4J can be faster than Neanderthal, and I still couldn't find a case where it is. Although, to my defense, the option in question here is very poorly documented. I've found the ND4J tutorial page where it's mentioned, and even after re-reading the sentence multiple times, I still do not connect its description to what it (seems to) actually do. It also does not mention that it affects computation speed. Anyway, I'm looking forward to reading your detailed analysis, and especially seeing your Neanderthal numbers.
  - treo 8y ago
    Do you have any pointer on how you've profiled Neanderthal during development? When I originally set out to compare ND4J and Neanderthal, I've ran into the issue that I bottomed out at: they basically both call MKL (or Openblas) for BLAS operations.

zz34 8y ago

When will you just admit the initial claim about ND4J being faster than Neanderthal was bogus?

vonnik 8y ago

Here's our updated benchmarks. Thanks to Dragan for the cooperation! Benchmarking ND4J and Neanderthal (Scientific Computing in Java and Clojure) https://www.dubs.tech/blog/benchmarking-nd4j-and-neanderthal/ https://www.dubs.tech/blog/benchmarking-nd4j-and-neanderthal...