Cool that tiny models have better performance on some Julia frameworks then on pytorch but till Julia get some framework that allows for seamless multi-gpu trai

Cool that tiny models have better performance on some Julia frameworks then on pytorch but till Julia get some framework that allows for seamless multi-gpu training a lot of DL people just wont use it sadly. Its for sure easier to just optimize your small model on device exporting it from pytorch then write your own multi-gpu bridge using Nvidia software to train it :/