A componentized DL inference hardware library + compiler suite + inference runtime. Overall already appears a solid foundation for a production system ATM. Anot

A componentized DL inference hardware library + compiler suite + inference runtime. Overall already appears a solid foundation for a production system ATM. Another example of nVidia's unchallenged prowess in DL software ecosystem. The hardware design library could be seen as a prominent example of "software defined hardware". Specifically, the compnentized hardware library dissects DL inference into 4 high-level operations: convolution, pooling, activation, normalization. Hardware description code is provided for these 4 operations. Allowing practitioners to tailor their chip design to their application. For instance, one could remove an operation from the chip, and adjust the size and throughput of any one operation. Software: compiler suite, and runtime are provided for building and running actual DL models. Compiler suite also operate based on the 4 operations, and produce an IR format that models around the operations; and then optimized towards the target chip. Runtime include kernel module and user space driver to handle the setup of the chip runtime and accept user request of loading model and inference requests. 2 questions I did not find explicit answer: * Is there an inference-aware training capability? * Does the compiler suite also consider the actual hardware spec during its optimization?