4 ms·

Thanks. Does it map to native routines like numpy? If yes, what about the copy overhead from JVM to native memory?

• The_rationalist 5y ago
  Nd4j (another project) enable to use cuda/MKL sota backends. It does native calls but I don't think the overhead is high. Besides some of those overhead are being optimized in the JNI successor.
• emergentorder 5y ago
  Yes, it uses ONNX Runtime / MLAS (their native BLAS lib) under the hood. And yes, there is copy overhead but you can eliminate it internally to a single function/graph by compiling it down to a single ONNX model. The end result is within ~15% run time of PyTorch w/ MKL when training a reasonably-sized MLP. And ORT also provides support for CUDA and a number of other "execution providers".