3 ms·
This seems to be comparing a hand implemented neural network in python (and numpy) and one in rust. Even in this simple case, the author discovers that in the p
by archgoon 7y ago
This seems to be comparing a hand implemented neural network in python (and numpy) and one in rust. Even in this simple case, the author discovers that in the python case, most of the time is spent in non-python linear algebra libraries.
Most of the major deep learning frameworks for python (tensorflow, keras, torch, mxnet, etc) will not normally be spending the majority of their time in python. Typically, the strategy is to use python to declare the overall structure of the net, and where to load data from, and then the actual heavy lifting will be done in optimized libraries written probably in C++ (or fortran, I seem to recall BLAS used fortran).
- sgillen 7y agoI think BLAS is a spec rather than one library. So your version may or may not be fortran. I do think the original "reference implementation" was written in fortran, which is sometimes called "The BLAS library" but I think most BLAS you see in the wild are not that.
- the_svd_doctor 7y agoYes. BLAS was originally specified in Fortran. But many BLAS implementation (like cuBLAS, the CUDA/Nvidia version for GPU's) don't use Fortran at all.
- archgoon 7y ago> I think BLAS is a spec rather than one library. Yep; poor wording on my part. Thanks. :)
- axaxs 7y agoThis is the strength of python, though, and one you cannot ignore. Python is old, and has fast C ops for everything you may want to accomplish. There's no shame in that method in benchmarks.