4 ms·

With it being a very popular python implementation, one wonders why such optimizations haven't been implemented yet. 60% sounds huge.

by pferde 7y ago

mikepurvis 7y ago
Probably because overall Python startup time is still dominated by site.py/pkg_resources spidering the filesystem and importing half the world; as long as that exists for most users, it's hard to be motivated to make speedups anywhere else. cf. https://github.com/pypa/setuptools/issues/510 https://github.com/pypa/setuptools/issues/510 The problem is so bad that it's one of the top quoted reasons that hg is considering a gradual rewrite to rust: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan
- pferde 7y ago
  Thanks for this, it makes for an interesting read, especially for someone who only dips his toe in python programming lightly once in a while.
- icebraining 7y ago
  The pkg_resources problem is specific for programs that use entry points (and it can be avoided), not a general CPython issue, and AFAICT is not related to Mercurial's plan to include Rust.
  - mikepurvis 7y ago
    Interesting; I hadn't realised this, thanks for the clarification. I'm aware of yapsy as a lightweight plugin system, but entry_points really is the gold standard in terms of Python extensibility. It's a shame it can't be made more performant.
- cozzyd 7y ago
  Hilariously, my pentadactyl shortcut for the oxidation plan was HG. I took that as a sign to read the article. Has hg ever considered a long-running "daemon-like" process instead? It could be instantiated automatically by the first call to hg and then it can die after it hasn't been used for a while. It would probably be racy and hard to support on multiple OS's though...
  - mikepurvis 7y ago
    They do kind of have this: https://www.mercurial-scm.org/wiki/CommandServer https://www.mercurial-scm.org/wiki/CommandServer Though it looks like it's more about supplying a way to embed hq in a proprietary (or non-Python) application via subprocess communication rather than directly importing the GPL modules or having to string-parse responses. There doesn't seem to be provision for interacting with the server directly from the command line, which is odd, since this pattern is well established from Java land (bazel, for example).

With it being a very popular python implementation, one wonders why such optimizations haven't been implemented yet. 60% sounds huge.

by pferde 7y ago

With it being a very popular python implementation, one wonders why such optimizations haven't been implemented yet. 60% sounds huge.

mikepurvis 7y ago

Probably because overall Python startup time is still dominated by site.py/pkg_resources spidering the filesystem and importing half the world; as long as that exists for most users, it's hard to be motivated to make speedups anywhere else. cf. https://github.com/pypa/setuptools/issues/510 https://github.com/pypa/setuptools/issues/510 The problem is so bad that it's one of the top quoted reasons that hg is considering a gradual rewrite to rust: https://www.mercurial-scm.org/wiki/OxidationPlan https://www.mercurial-scm.org/wiki/OxidationPlan

pferde 7y ago
Thanks for this, it makes for an interesting read, especially for someone who only dips his toe in python programming lightly once in a while.
icebraining 7y ago
The pkg_resources problem is specific for programs that use entry points (and it can be avoided), not a general CPython issue, and AFAICT is not related to Mercurial's plan to include Rust.
- mikepurvis 7y ago
  Interesting; I hadn't realised this, thanks for the clarification. I'm aware of yapsy as a lightweight plugin system, but entry_points really is the gold standard in terms of Python extensibility. It's a shame it can't be made more performant.
cozzyd 7y ago
Hilariously, my pentadactyl shortcut for the oxidation plan was HG. I took that as a sign to read the article. Has hg ever considered a long-running "daemon-like" process instead? It could be instantiated automatically by the first call to hg and then it can die after it hasn't been used for a while. It would probably be racy and hard to support on multiple OS's though...
- mikepurvis 7y ago
  They do kind of have this: https://www.mercurial-scm.org/wiki/CommandServer https://www.mercurial-scm.org/wiki/CommandServer Though it looks like it's more about supplying a way to embed hq in a proprietary (or non-Python) application via subprocess communication rather than directly importing the GPL modules or having to string-parse responses. There doesn't seem to be provision for interacting with the server directly from the command line, which is odd, since this pattern is well established from Java land (bazel, for example).