5 ms·
The unique property is the ability to just pick any code or library that is unaware of the differentiation library (unlike tf.function as it needs to specifical
by ddragon 6y ago
The unique property is the ability to just pick any code or library that is unaware of the differentiation library (unlike tf.function as it needs to specifically use tf methods) and get the gradient. In a language like Julia this is immediately useful as it has a massive ecosystem of numerical code that make sense to get gradients (like differential equations and the SciML project [1], or less conventional stuff like raytracers), but in a language like Swift (as there is no meaning to gradient of GUI libraries or frontend stuff) it is more of a "if you build they'll come" faith from Google.
But regardless unique features, it's a have cake and eat it too type of interface. You don't need to learn a second language within the language like tensorflow's tf.* making it even more natural and flexiblethan pytorch, including all debug mechanisms of the host language itself, but you still get compile time graph creation like tensorflow, including all kinds of optimizations. It makes other approaches seem primitive by comparison, but creating it is much more complex, and the main audience is already more than used to using language within language solutions (like numpy) which can provide something almost as good even if less elegantly, so it's not easy to convince people as well (when it involves changing programming languages).
[1] https://sciml.ai/ https://sciml.ai/