5 ms·

Assuming this is your code: https://github.com/antirez/neural-redis/blob/master/nn.c https://github.com/antirez/neural-redis/blob/master/nn.c You're really jus

by gok 10y ago

Assuming this is your code: https://github.com/antirez/neural-redis/blob/master/nn.c https://github.com/antirez/neural-redis/blob/master/nn.c You're really just implementing a vector*matrix multiply. You probably want to just use BLAS's sgemv routine. On macOS, link Accelerate and use cblas_sgemv(); on Linux consider installing Intel MKL or OpenBLAS. If you're just looking to learn what a reasonably state-of-the-art SGEMV kernel looks like for a modern chip like Haswell, check out OpenBLAS's code: https://github.com/xianyi/OpenBLAS/blob/develop/kernel/x86_64/sgemv_n_microk_haswell-4.c https://github.com/xianyi/OpenBLAS/blob/develop/kernel/x86_6...

deleted 10y ago
[deleted]
ajtulloch 10y ago
https://github.com/Maratyszcza/NNPACK/blob/master/src/x86_64-fma/blas/sgemm.py https://github.com/Maratyszcza/NNPACK/blob/master/src/x86_64... (from https://github.com/Maratyszcza/NNPACK https://github.com/Maratyszcza/NNPACK) is also a pretty reasonable AVX2 GEMM implementation written in a Python assembler, it's a bit easier to follow than the OpenBLAS kernels and has very reasonable performance vs OpenBLAS/MKL. I second the guidance to just use a BLAS library. I'd guess you'd see a 2-10x speedup at least. It looks like you're only implementing a classical MLP library (which is fine for learning, but not the best choice for something like this, integrating VW into Redis would be a much better choice IMO).
- genericpseudo 10y ago
  VW is the business. Strongly recommended.
antirez 10y ago
That sounds a great advice, I'll check this library. I've the feeling that the fact I've non aligned addresses in the current weights scheme will be a problem and padding will be required. That's why I used AVX "loadu" that deals with non aligned addresses, but perhaps I'm paying a lot of performances because of this. Thanks. EDIT: apparently on modern CPUs that's not the case and magically LOADU can be as fast as LOAD.
- gok 10y ago
  Actually if you have a Mac running Sierra, check out BasicNeuralNetworkSubroutines. Highly optimized primitives for exactly this purpose.
- gens 10y ago
  Don't do unaligned memory access, whatever your cpu flags say. Another thing you can do, if you don't plan on using the results immediately, is to use non-temporal store (movntps for SSE2). If you do plan to use the results right away, then just use them without storing in main memory. And you can do the usual unrolling of loop.
  - nkurz 10y ago
    Don't do unaligned memory access, whatever your cpu flags say. I don't think this is good advice. Do you have an example of poor performance of unaligned memory access on a processor that supports AVX? I think the better advice is: 1) Prefer aligned access if possible, favoring writes over reads. 2) A single unaligned access per cycle usually does not cause a slowdown. 3) Accesses that cross 64B cachelines essentially act as two accesses, thus may add a cycle. 4) You can only sustain multiple accesses per cycle if you avoid crossing cachelines. 5) Writes that cross 4KB pages have a 100+ cycle penalty on pre-Skylake Intel (down to 5 cycles on Skylake).
    - gens 10y ago
      Does anybody have an example of good performance of unaligned memory access on modern cpus ? And note that it doesn't matter if the cpu supports AVX, but if it has a flag that says it can do fast unaligned memory access (i don't remember, is it misalignsse ?). Common sense says that unaligned access can't be faster then aligned. And if you have data that fits into a ymm register, then you might as well use aligned access (a neural network is usually an example of such). I did test it a while ago. Problem is that i don't remember if it was on this, modern, cpu or the older one. I could test if i cared enough for other peoples opinion, but alas i don't (only usage of unaligned AVX access i found to be from newbies to SIMD). An example, that you request, would be to look at glibc memcpy, that uses ssse3 [0] so that it could always get aligned access (ssse3 has per-byte operations). In other words, how about that the people who claim that operations that do extra work are as fast as the ones that don't prove it ? Instead of the burden of proof falling on people that don't have such an opinion/experience ? Then i will bow my head and say "You are right. Thank you for pointing that out". But alas google-ing for 10min and have found no such benchmark anywhere. And writing such a test isn't hard, not in the slightest. [0]https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/multiarch/memcpy-ssse3.S https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/...

by gok 10y ago

deleted 10y ago

[deleted]

ajtulloch 10y ago

https://github.com/Maratyszcza/NNPACK/blob/master/src/x86_64-fma/blas/sgemm.py https://github.com/Maratyszcza/NNPACK/blob/master/src/x86_64... (from https://github.com/Maratyszcza/NNPACK https://github.com/Maratyszcza/NNPACK) is also a pretty reasonable AVX2 GEMM implementation written in a Python assembler, it's a bit easier to follow than the OpenBLAS kernels and has very reasonable performance vs OpenBLAS/MKL. I second the guidance to just use a BLAS library. I'd guess you'd see a 2-10x speedup at least. It looks like you're only implementing a classical MLP library (which is fine for learning, but not the best choice for something like this, integrating VW into Redis would be a much better choice IMO).

genericpseudo 10y ago
VW is the business. Strongly recommended.

antirez 10y ago

That sounds a great advice, I'll check this library. I've the feeling that the fact I've non aligned addresses in the current weights scheme will be a problem and padding will be required. That's why I used AVX "loadu" that deals with non aligned addresses, but perhaps I'm paying a lot of performances because of this. Thanks. EDIT: apparently on modern CPUs that's not the case and magically LOADU can be as fast as LOAD.

gok 10y ago
Actually if you have a Mac running Sierra, check out BasicNeuralNetworkSubroutines. Highly optimized primitives for exactly this purpose.
gens 10y ago
Don't do unaligned memory access, whatever your cpu flags say. Another thing you can do, if you don't plan on using the results immediately, is to use non-temporal store (movntps for SSE2). If you do plan to use the results right away, then just use them without storing in main memory. And you can do the usual unrolling of loop.
- nkurz 10y ago
  Don't do unaligned memory access, whatever your cpu flags say. I don't think this is good advice. Do you have an example of poor performance of unaligned memory access on a processor that supports AVX? I think the better advice is: 1) Prefer aligned access if possible, favoring writes over reads. 2) A single unaligned access per cycle usually does not cause a slowdown. 3) Accesses that cross 64B cachelines essentially act as two accesses, thus may add a cycle. 4) You can only sustain multiple accesses per cycle if you avoid crossing cachelines. 5) Writes that cross 4KB pages have a 100+ cycle penalty on pre-Skylake Intel (down to 5 cycles on Skylake).
  - gens 10y ago
    Does anybody have an example of good performance of unaligned memory access on modern cpus ? And note that it doesn't matter if the cpu supports AVX, but if it has a flag that says it can do fast unaligned memory access (i don't remember, is it misalignsse ?). Common sense says that unaligned access can't be faster then aligned. And if you have data that fits into a ymm register, then you might as well use aligned access (a neural network is usually an example of such). I did test it a while ago. Problem is that i don't remember if it was on this, modern, cpu or the older one. I could test if i cared enough for other peoples opinion, but alas i don't (only usage of unaligned AVX access i found to be from newbies to SIMD). An example, that you request, would be to look at glibc memcpy, that uses ssse3 [0] so that it could always get aligned access (ssse3 has per-byte operations). In other words, how about that the people who claim that operations that do extra work are as fast as the ones that don't prove it ? Instead of the burden of proof falling on people that don't have such an opinion/experience ? Then i will bow my head and say "You are right. Thank you for pointing that out". But alas google-ing for 10min and have found no such benchmark anywhere. And writing such a test isn't hard, not in the slightest. [0]https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/multiarch/memcpy-ssse3.S https://github.com/lattera/glibc/blob/master/sysdeps/x86_64/...