Today, things like PyTorch are AD systems, optimization systems, libraries of high performance interoperable common blocks, and more. This massive complexity is

Today, things like PyTorch are AD systems, optimization systems, libraries of high performance interoperable common blocks, and more. This massive complexity is why there is no standard. If you have a compiler front end that can AD arbitrary code, then the framework no longer has to worry about that. If you have a compiler backend that can target multiple devices, then the framework doesn’t have to do that, either. The size and complexity of PyTorch/TensorFlow is exactly why there isn’t a standard hardware portable NN API. A sane API of hardware independent common blocks can’t happen without all the front and backend compiler stuff first. A PyTorch/TensorFlow-like API is always going to be massive, complicated, and hard to port to new hardware targets. Additionally, the complexity and reliance on C alone will make integrating exotic new concepts like say, differential equation solvers, extremely laborious.