2 ms·

I think we are talking past each other. sillysaurusx is talking about OpenGL/Direct3D as a portability layer for hardwares. API vs AD is not a hardware portabil

I think we are talking past each other. sillysaurusx is talking about OpenGL/Direct3D as a portability layer for hardwares. API vs AD is not a hardware portability layer concern. The proposal is to standardize OpenGL/Direct3D-like hardware portability layer defined by API. API vs AD you are talking about happens far above that and it is irrelevant.

oivey 6y ago
Today, things like PyTorch are AD systems, optimization systems, libraries of high performance interoperable common blocks, and more. This massive complexity is why there is no standard. If you have a compiler front end that can AD arbitrary code, then the framework no longer has to worry about that. If you have a compiler backend that can target multiple devices, then the framework doesn’t have to do that, either. The size and complexity of PyTorch/TensorFlow is exactly why there isn’t a standard hardware portable NN API. A sane API of hardware independent common blocks can’t happen without all the front and backend compiler stuff first. A PyTorch/TensorFlow-like API is always going to be massive, complicated, and hard to port to new hardware targets. Additionally, the complexity and reliance on C alone will make integrating exotic new concepts like say, differential equation solvers, extremely laborious.