3 ms·
"Go fast" has multiple dimensions. For one example, I had a command-line tool which needed to compute something related to hypergeometric distribution. (It's b
by eesmith 4y ago
"Go fast" has multiple dimensions.
For one example, I had a command-line tool which needed to compute something related to hypergeometric distribution. (It's been a few years; I forget the details.)
This available in scipy, which need numpy. Most of my program's run-time was spent importing numpy. (The following timings are best-of-3.)
% time python -c pass
0.027u 0.011s 0:00.04 75.0% 0+0k 0+0io 0pf+0w
% time python -c "import numpy"
0.212u 0.059s 0:00.17 152.9% 0+0k 0+0io 14pf+0w
% time python -c "import scipy"
0.252u 0.077s 0:00.32 100.0% 0+0k 0+0io 14pf+0w
While 0.2 seconds doesn't seem like much to people used to spending hours developing a notebook, or running some large matrix computation, I could make my program 8x faster by writing the dozen or so lines I needed to evaluate that function myself.
Numpy is not about making short-lived programs fast.
In any case, this specific choice of importing all submodules is not to cache things up "for obvious [performance] reasons", because there is no machine performance improvements.
Instead, it's to make the API easier/faster to use. When you're in a notebook and you need numpy.foo.bar() you can just use it, and not have to go up and use an import statement first.
- urthor 4y agoYeah exactly. However, what I'd say is that a short launch from rest is an even more obscure use case than the one numpy supports. numpy has made the correct compromise IMO, optimizing for long lived programs and notebooks.
- eesmith 4y agoWhile my observation is that most other Python packages don't import all subpackages. "import urllib" does not also import urllib.parse, so you can't do: import urllib urllib.parse.quote("A&W") but instead must explicitly import urllib.parse. scikit-learn supports essentially the same use cases as NumPy and it doesn't import all its subpackages: >>> import sklearn >>> sklearn.linear_model Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'sklearn' has no attribute 'linear_model' >>> import sklearn.linear_model >>> sklearn.linear_model <module 'sklearn.linear_model' from '[...]sklearn/linear_model/__init__.py'> Thus leading to my earlier comment: > The NumPy developers think "practicality beats purity" is more important, and their main use-case is long-running programs where startup costs are slow, which I interpret as meaning their special cases is special enough to break the rules.