I mentioned theory and experts, you said enough theory. I switched to practical applications and walk the talk showing my code, and then you back off and want

I mentioned theory and experts, you said enough theory. I switched to practical applications and walk the talk showing my code, and then you back off and want to go back to opinions. I see now that you want to back myself with experts since reproducible code and runnable benchmarks is not enough. Apparently you recognize Nvidia as an expert so let's talk about CuDNN where optimizing convolution is all about memory layout, source: https://github.com/soumith/convnet-benchmarks/issues/93#issuecomment-192627065 https://github.com/soumith/convnet-benchmarks/issues/93#issu... and it's not about C vs C++ vs PTX. Or let's hear about what Nvidia says about optimizing GEMM: https://github.com/NVIDIA/cutlass/blob/master/media/docs/efficient_gemm.md https://github.com/NVIDIA/cutlass/blob/master/media/docs/eff..., it's all about memory locality and tiling. Or maybe Stanford, the US government and Nvidia Research are also wrong when pouring significant research in Legion? https://legion.stanford.edu/ https://legion.stanford.edu/ > Legion is a data-centric parallel programming system for writing portable high performance programs targeted at distributed heterogeneous architectures. Legion presents abstractions which allow programmers to describe properties of program data (e.g. independence, locality). By making the Legion programming system aware of the structure of program data, it can automate many of the tedious tasks programmers currently face, including correctly extracting task- and data-level parallelism and moving data around complex memory hierarchies. A novel mapping interface provides explicit programmer controlled placement of data in the memory hierarchy and assignment of tasks to processors in a way that is orthogonal to correctness, thereby enabling easy porting and tuning of Legion applications to new architectures. Are you saying they should have just called it a day once they were done with C++? Or you can read the DaCE paper on how to beat CuBLAS and CuDNN: https://arxiv.org/pdf/1902.10345.pdf https://arxiv.org/pdf/1902.10345.pdf, it's all about data movement. 6.4 Case Study III: Quantum Transport to optimize transistors heat dissipation, Nvidia strided matrix multiplication was improved upon by over 30%, and this part is pure Assembly, the improvement was about better utilizing the hardware caches.