3 ms·

Use a domain-specific compiler to generate custom, stand-alone, massively multi-threaded AVX-512 inference C code: https://NN-512.com https://NN-512.com The g

Use a domain-specific compiler to generate custom, stand-alone, massively multi-threaded AVX-512 inference C code: https://NN-512.com https://NN-512.com The generated code is easily twice as fast as TensorFlow's AVX-512 kernels (Intel's oneAPI).

etaioinshrdlu 5y ago
Your project is well engineered but no matter how many times you post it here, you won’t get real traction without a different development model, such as building on github. This is especially true in a high churn, high cost field like ML. I also think you are being anonymous unnecessarily.
johndough 5y ago
Some user feedback: I tried to get the ResNet50 example to work, but I gave up after 2 hours. There really should be an end-to-end example like "./resnet50_example.py monkey.jpg". A few points where I struggled: - What are the names in the ResNet50Params struct? I looked at a few popular ResNet50 implementations, but I could not find any correspondence and since the names have been sorted, the order of the members might as well be random. I thought about matching the parameters by shape instead, but the chance of getting that correct is almost zero. Why no natural sort? Or even better, a naming scheme that sorts in the same order as execution order, for example with zero-prefixed numbers like "layer05" - 6 MB C files are somewhat ridiculous. There should at least be a download link somewhere. Copy & pasting from the website would take forever because of scrolling. Also it is impossible to verify that that code monstrosity is not doing anything evil. - Go seems like an odd choice when almost every machine learning project these days is developed in Python. - GCC throws a few hundred warnings: ResNet50.c: In function ‘ResNet50ThreeArrangeDats7Callee1’: ResNet50.c:57009:15: warning: unused variable ‘rel27’ [-Wunused-variable] 57009 | ptrdiff_t rel27 = j62-0; | ^~~~~ and ResNet50.c: In function ‘ResNet50NetCreate’: ResNet50.c:77852:20: warning: taking address of packed member of ‘struct ResNet50Params’ may result in an unaligned pointer value [-Waddress-of-packed-member] 77852 | params1->bn1Means, | ~~~~~~~^~~~~~~~~~ - You would get more feedback if there was a platform to discuss such issues (e.g. GitHub) instead of hijacking random threads on HackerNews. - EDIT: I just realized I could probably just write a parser for that graph file format. This would be much easier if it was something standard like JSON instead. - EDIT2: I think some shapes in the graph file definition are incorrect (possibly reversed per block?). For example, tensor "one1" should have "ToChannels=64" instead of "ToChannels=256".
- 37ef_ced3 5y ago
  Here is a guy who worked it out: https://github.com/jonatron/test_nn512 https://github.com/jonatron/test_nn512 That's a full ResNet50 conversion from the corresponding Caffe protobuf. He also wrote a small script to convert TensorFlow networks. GCC warns about perfectly valid, correct C. It even warns about switch case fallthough. All of the generated code is fully correct. Turn the warnings off. Please don't copy and paste from the website! Download the NN-512 executable and run it. Email me if you still have difficulties.
  - johndough 5y ago
    Thanks, that code does indeed work! And it is significantly faster than PyTorch (15 vs 60 milliseconds). Maybe I will use it in the future if AVX512 should become more widespread.
    - 37ef_ced3 5y ago
      Awesome. Send me email any time and I will assist you.
    - 37ef_ced3 5y ago
      The new NN-512 release (version 35) changes the build instructions to suppress GCC's warnings while building the generated code (-w switch): // To build an object file: // gcc -c -w -std=c99 -pthread -Ofast -mavx512f Example1.c Thanks for your feedback.

etaioinshrdlu 5y ago

Your project is well engineered but no matter how many times you post it here, you won’t get real traction without a different development model, such as building on github. This is especially true in a high churn, high cost field like ML. I also think you are being anonymous unnecessarily.

johndough 5y ago

Some user feedback: I tried to get the ResNet50 example to work, but I gave up after 2 hours. There really should be an end-to-end example like "./resnet50_example.py monkey.jpg". A few points where I struggled: - What are the names in the ResNet50Params struct? I looked at a few popular ResNet50 implementations, but I could not find any correspondence and since the names have been sorted, the order of the members might as well be random. I thought about matching the parameters by shape instead, but the chance of getting that correct is almost zero. Why no natural sort? Or even better, a naming scheme that sorts in the same order as execution order, for example with zero-prefixed numbers like "layer05" - 6 MB C files are somewhat ridiculous. There should at least be a download link somewhere. Copy & pasting from the website would take forever because of scrolling. Also it is impossible to verify that that code monstrosity is not doing anything evil. - Go seems like an odd choice when almost every machine learning project these days is developed in Python. - GCC throws a few hundred warnings: ResNet50.c: In function ‘ResNet50ThreeArrangeDats7Callee1’: ResNet50.c:57009:15: warning: unused variable ‘rel27’ [-Wunused-variable] 57009 | ptrdiff_t rel27 = j62-0; | ^~~~~ and ResNet50.c: In function ‘ResNet50NetCreate’: ResNet50.c:77852:20: warning: taking address of packed member of ‘struct ResNet50Params’ may result in an unaligned pointer value [-Waddress-of-packed-member] 77852 | params1->bn1Means, | ~~~~~~~^~~~~~~~~~ - You would get more feedback if there was a platform to discuss such issues (e.g. GitHub) instead of hijacking random threads on HackerNews. - EDIT: I just realized I could probably just write a parser for that graph file format. This would be much easier if it was something standard like JSON instead. - EDIT2: I think some shapes in the graph file definition are incorrect (possibly reversed per block?). For example, tensor "one1" should have "ToChannels=64" instead of "ToChannels=256".

37ef_ced3 5y ago
Here is a guy who worked it out: https://github.com/jonatron/test_nn512 https://github.com/jonatron/test_nn512 That's a full ResNet50 conversion from the corresponding Caffe protobuf. He also wrote a small script to convert TensorFlow networks. GCC warns about perfectly valid, correct C. It even warns about switch case fallthough. All of the generated code is fully correct. Turn the warnings off. Please don't copy and paste from the website! Download the NN-512 executable and run it. Email me if you still have difficulties.
- johndough 5y ago
  Thanks, that code does indeed work! And it is significantly faster than PyTorch (15 vs 60 milliseconds). Maybe I will use it in the future if AVX512 should become more widespread.
  - 37ef_ced3 5y ago
    Awesome. Send me email any time and I will assist you.
  - 37ef_ced3 5y ago
    The new NN-512 release (version 35) changes the build instructions to suppress GCC's warnings while building the generated code (-w switch): // To build an object file: // gcc -c -w -std=c99 -pthread -Ofast -mavx512f Example1.c Thanks for your feedback.