Admittedly it is a bit hard to see why Julia is so different in this area if you have not spent some with it. It took me a bit time as well to see. But there r

Admittedly it is a bit hard to see why Julia is so different in this area if you have not spent some with it. It took me a bit time as well to see. But there really is a profound difference. The thing is that Python is really just a language to rearrange networks of nodes written in C++. A node written for one ML framework will be incompatible with a node written in another language. If you want them to work together you need to spend a lot of effort gluing them together, and most likely the performance benefits are lost. This is very different in Julia because you are using pure Julia code. What is specialized C++ nodes in Python, are for the most part just plain functions in Julia. There is nothing special about them. That is why they can be used anywhere. In Python frameworks each of these C++ nodes need to contain information about how auto-differentiation is done. That is why they are tailor made for one specific ML library. In Julia the whole automatic differentiation stuff is perform code transformations on regular Julia code. Are you familiar with LISP, and the idea of code as data? That is what we are talking about here. A regular syntax graph of regular Julia code gets manipulated to do autodiff. You are not organizing C++ nodes in a network.