3 ms·

AMD needs to do something like https://github.com/intel/mkl-dnn https://github.com/intel/mkl-dnn and https://github.com/opencv/dldt https://github.com/opencv/dl

by m0zg 7y ago

AMD needs to do something like https://github.com/intel/mkl-dnn https://github.com/intel/mkl-dnn and https://github.com/opencv/dldt https://github.com/opencv/dldt, either by talking to Intel to get them to accept contributions (vastly preferable), or, failing that, by forking the libs and implementing their own support. With recent work on efficient NN architectures the CPUs are pretty viable for most _inference_ (not training) tasks. DLDT/OpenVINO perf is particularly impressive. It does, however, extensively use JIT kernel generation through https://github.com/herumi/xbyak https://github.com/herumi/xbyak depending on the supported features for a particular chip (search for "mayiuse()" in the code to see examples). It's anyone's guess what it will detect and generate on an AMD chip however. Now that AMD is raking in more cash, it'd be a great time to invest some of that back into their dev ecosystem.

gnufx 7y ago
The relevant free low-level library in this area is libxsmm, which drove MKL to improve for small matrix work by being faster a while ago. I don't know how it performs on AMD CPUs, or how much a priori detailed micro-architecture knowledge it requires, but I doubt the maintainer would refuse changes for AMD. It was referenced with no interest in https://news.ycombinator.com/item?id=16600347 https://news.ycombinator.com/item?id=16600347 How specific to Intel CPUs is MKL-DNN (now renamed)? When I looked at it, the CPU code seemed fairly generic SIMD.
- m0zg 7y ago
  MKL-DNN is not MKL though. I think MKL-DNN is fairly specific to Intel CPUs though if you want maximum performance. It'll necessarily make certain assumptions about instruction throughput and latency, something that's pretty much certain to be different on AMD in some cases. In super tight kernels such minute differences often boil down to much lower performance. So if AMD wants to do a good job, they'll need, at least in some cases, to detect AMD CPUs and JIT their own specialized kernels that perform well on AMD.
  - gnufx 7y ago
    Sure, but it wasn't immediately obvious it was that specific when I looked, hence the question.

by m0zg 7y ago

gnufx 7y ago

The relevant free low-level library in this area is libxsmm, which drove MKL to improve for small matrix work by being faster a while ago. I don't know how it performs on AMD CPUs, or how much a priori detailed micro-architecture knowledge it requires, but I doubt the maintainer would refuse changes for AMD. It was referenced with no interest in https://news.ycombinator.com/item?id=16600347 https://news.ycombinator.com/item?id=16600347 How specific to Intel CPUs is MKL-DNN (now renamed)? When I looked at it, the CPU code seemed fairly generic SIMD.

m0zg 7y ago
MKL-DNN is not MKL though. I think MKL-DNN is fairly specific to Intel CPUs though if you want maximum performance. It'll necessarily make certain assumptions about instruction throughput and latency, something that's pretty much certain to be different on AMD in some cases. In super tight kernels such minute differences often boil down to much lower performance. So if AMD wants to do a good job, they'll need, at least in some cases, to detect AMD CPUs and JIT their own specialized kernels that perform well on AMD.
- gnufx 7y ago
  Sure, but it wasn't immediately obvious it was that specific when I looked, hence the question.