4 ms·

Python Entry Points Explained

167 points by Nurdok 9y ago

rajathagasthya 9y ago
I highly recommend reading Stevedore package's excellent documentation [1] to see practical examples of how entry points can be used as a plugin manager in applications. [1] - https://docs.openstack.org/stevedore/latest/index.html https://docs.openstack.org/stevedore/latest/index.html
pmoriarty 9y ago
Way too much ink wasted on cuteness and too little on explanation. I got stumped at: "In particular, the magic happens in get_sneks. The call to pkg_resources.iter_entry_points('snek_types') iterates over all the entry points that were registered under the name "snek_types". So, external packages can define an entry point called "snek_types" in their setup.py, and snek will dynamically load it at runtime." Wait. What entry points were registered under the name "snek_types"? Where were they regeistered as such? I think I must be missing something, or maybe that registration was hidden somewhere among all the fancy snakes. Can someone help explain this in a clearer way?
- greenshackle2 9y ago
  You have to keep reading, further down they show the updated setup.py with the snek_types entry points. I agree it's not super well organized, they should have printed the new setup.py before that bit.
- Karunamon 9y ago
  The registration is handled by the contents of the tuple in setup.py. Specifically: from setuptools import setup setup( name='snek', entry_points={ 'console_scripts': [ 'snek = snek:main', ], } ) It's using the setuptools module, and passing a tuple to it - which is used to create executables for the individual scripts once installed. https://packaging.python.org/tutorials/distributing-packages/#setup-args https://packaging.python.org/tutorials/distributing-packages... Specifically, "entry_points" in the setup tuple is what sets the name of the created files and links them to your code. In the example, the executable 'snek' points to main() in snek.py. Once your package is installed, pip or easy_install handles the magic of putting those files in place. There are boatloads of other options available (much like any packaging system), but this is a minimum viable example.
- lftl 9y ago
  That seemed like magic to me at first too. I think though that is happens during the installation of the package, which is why he had to reinstall after refactoring. I initially just skimmed over his log dump from the install, but I think the details are actually important there: > writing entry points to cute_snek.egg-info\entry_points.txt
- antoncohen 9y ago
  "Snek Pro Solutions" distributes a package (pip installed) called cute-snek. In the cute-snek setup.py it has: entry_points={ 'snek_types': [ 'cute = cute_snek:cute_snek', ], } The main snek codebase has: for entry_point in pkg_resources.iter_entry_points('snek_types'): sneks[entry_point.name] = entry_point.load() pkg_resources.iter_entry_points() iterates through all the snek_types an end user has installed on their system. When the end user does `pip install snek` they will only have the built-in sneks, when they also do `pip install cute-snek` the will have the "cute" snek installed, allowing them to run `snek --type cute` to see it.
jwilk 9y ago
Entry points are excellent at making your scripts start slooooooooow. Compare $ time python -m pyflakes /dev/null real 0m0.225s user 0m0.103s sys 0m0.008s with $ time pyflakes /dev/null real 0m2.468s user 0m1.738s sys 0m0.055s
- gizmo385 9y ago
  I've noticed that using argparse makes my scripts start very slow, especially if you expose a large number of option. I have a script that I wrote for work that has quite a variable number of available options and it can sometimes take several seconds for the script to startup and actually begin doing anything useful.
  - yes_or_gnome 9y ago
    Argparse's source is possibly the most difficult lib to grow from the stdlib. https://github.com/python/cpython/blob/master/Lib/argparse.py https://github.com/python/cpython/blob/master/Lib/argparse.p... For me, it is not debugable. I wanted to propose and start a reference implementation for supporting --[no-]bool-opt, but I quickly gave up. There's some solutions on SO, but none of them are sufficient. The best solution makes --bool-opt mutually exclusive to --no-bool-opt which will cause a hard failure which is not okay for shell aliases.
- ericfrederich 9y ago
  I did not get as dramatic of results as you. # python3 -m venv example # source example/bin/activate (example) # pip install pyflakes Collecting pyflakes Downloading pyflakes-1.5.0-py2.py3-none-any.whl (225kB) 100% || 225kB 892kB/s Installing collected packages: pyflakes Successfully installed pyflakes-1.5.0 (example) # time for i in {00..99}; do python -m pyflakes /dev/null; done real 0m7.167s user 0m6.407s sys 0m0.750s (example) # time for i in {00..99}; do pyflakes /dev/null; done real 0m7.661s user 0m6.894s sys 0m0.753s
  - jwilk 9y ago
    I guess it gets worse when you have more stuff installed in the environment.
sjburt 9y ago
What is the actual value of entry points over just marking a script +x and putting somewhere on the path? All I see is a bunch of added complexity.
- rcthompson 9y ago
  Answering this question was pretty much the entire point of the article. I felt that each section gave a concrete example of a situation where the solution in the previous section fell short and then introducing some new complexity to handle it.

167 points by Nurdok 9y ago

I highly recommend reading Stevedore package's excellent documentation [1] to see practical examples of how entry points can be used as a plugin manager in applications. [1] - https://docs.openstack.org/stevedore/latest/index.html https://docs.openstack.org/stevedore/latest/index.html

pmoriarty 9y ago

Way too much ink wasted on cuteness and too little on explanation. I got stumped at: "In particular, the magic happens in get_sneks. The call to pkg_resources.iter_entry_points('snek_types') iterates over all the entry points that were registered under the name "snek_types". So, external packages can define an entry point called "snek_types" in their setup.py, and snek will dynamically load it at runtime." Wait. What entry points were registered under the name "snek_types"? Where were they regeistered as such? I think I must be missing something, or maybe that registration was hidden somewhere among all the fancy snakes. Can someone help explain this in a clearer way?

greenshackle2 9y ago
You have to keep reading, further down they show the updated setup.py with the snek_types entry points. I agree it's not super well organized, they should have printed the new setup.py before that bit.
Karunamon 9y ago
The registration is handled by the contents of the tuple in setup.py. Specifically: from setuptools import setup setup( name='snek', entry_points={ 'console_scripts': [ 'snek = snek:main', ], } ) It's using the setuptools module, and passing a tuple to it - which is used to create executables for the individual scripts once installed. https://packaging.python.org/tutorials/distributing-packages/#setup-args https://packaging.python.org/tutorials/distributing-packages... Specifically, "entry_points" in the setup tuple is what sets the name of the created files and links them to your code. In the example, the executable 'snek' points to main() in snek.py. Once your package is installed, pip or easy_install handles the magic of putting those files in place. There are boatloads of other options available (much like any packaging system), but this is a minimum viable example.
lftl 9y ago
That seemed like magic to me at first too. I think though that is happens during the installation of the package, which is why he had to reinstall after refactoring. I initially just skimmed over his log dump from the install, but I think the details are actually important there: > writing entry points to cute_snek.egg-info\entry_points.txt
antoncohen 9y ago
"Snek Pro Solutions" distributes a package (pip installed) called cute-snek. In the cute-snek setup.py it has: entry_points={ 'snek_types': [ 'cute = cute_snek:cute_snek', ], } The main snek codebase has: for entry_point in pkg_resources.iter_entry_points('snek_types'): sneks[entry_point.name] = entry_point.load() pkg_resources.iter_entry_points() iterates through all the snek_types an end user has installed on their system. When the end user does `pip install snek` they will only have the built-in sneks, when they also do `pip install cute-snek` the will have the "cute" snek installed, allowing them to run `snek --type cute` to see it.

jwilk 9y ago

Entry points are excellent at making your scripts start slooooooooow. Compare $ time python -m pyflakes /dev/null real 0m0.225s user 0m0.103s sys 0m0.008s with $ time pyflakes /dev/null real 0m2.468s user 0m1.738s sys 0m0.055s

gizmo385 9y ago
I've noticed that using argparse makes my scripts start very slow, especially if you expose a large number of option. I have a script that I wrote for work that has quite a variable number of available options and it can sometimes take several seconds for the script to startup and actually begin doing anything useful.
- yes_or_gnome 9y ago
  Argparse's source is possibly the most difficult lib to grow from the stdlib. https://github.com/python/cpython/blob/master/Lib/argparse.py https://github.com/python/cpython/blob/master/Lib/argparse.p... For me, it is not debugable. I wanted to propose and start a reference implementation for supporting --[no-]bool-opt, but I quickly gave up. There's some solutions on SO, but none of them are sufficient. The best solution makes --bool-opt mutually exclusive to --no-bool-opt which will cause a hard failure which is not okay for shell aliases.
ericfrederich 9y ago
I did not get as dramatic of results as you. # python3 -m venv example # source example/bin/activate (example) # pip install pyflakes Collecting pyflakes Downloading pyflakes-1.5.0-py2.py3-none-any.whl (225kB) 100% || 225kB 892kB/s Installing collected packages: pyflakes Successfully installed pyflakes-1.5.0 (example) # time for i in {00..99}; do python -m pyflakes /dev/null; done real 0m7.167s user 0m6.407s sys 0m0.750s (example) # time for i in {00..99}; do pyflakes /dev/null; done real 0m7.661s user 0m6.894s sys 0m0.753s
- jwilk 9y ago
  I guess it gets worse when you have more stuff installed in the environment.

sjburt 9y ago

What is the actual value of entry points over just marking a script +x and putting somewhere on the path? All I see is a bunch of added complexity.

rcthompson 9y ago
Answering this question was pretty much the entire point of the article. I felt that each section gave a concrete example of a situation where the solution in the previous section fell short and then introducing some new complexity to handle it.