5 ms·

Writing Makefiles for Python Projects

31 points by smitty1e 6y ago

smitty1e 6y ago
I've used the former to good effect. A phony help target for a default can print a menu, so that the only thing to know/remember is to type: `make`. However, for setting up the whole environment, I'm looking for a cookiecutter-based or -integrated solution.
sneak 6y ago
One of my policy for all employees or contractors is that "make" (and optionally "make test") in the root of a repo should always "do the thing". We have a de facto, decades old standard for a "run the tests" or a "build the damn thing" big red button. We should use it, even if it's just a few line wrapper over npm run or docker build or whatever. (The fact that it could be either is the reason that there should be a standardized entrypoint.)
- wahern 6y ago
  I wish more people would read https://www.gnu.org/software/make/manual/html_node/Standard-Targets.html https://www.gnu.org/software/make/manual/html_node/Standard-.... Once upon a time anybody using a Unix-like OS was familiar with most of those standard targets as a consequence of the ubiquity of autotools, especially when package repositories weren't so comprehensive and it was more common to download and build tarballs locally. But at my current job, there's no rhyme or reason to make targets. Something like `make build` (where build may or may not be the default target, likely accidentally if it is) is more likely to install Go toolchain dependencies than to build any of the project source code. And of course little if any attempt is made to actually declare dependencies, correct or broken, as for so many programmers these days a Makefile is just a container for ad hoc shell code. I don't often make use of autotools in my own projects as I can usually get good mileage from plain POSIX make, switching to GNU make in a bind, but I religiously try to adhere to the standard targets and other build and distribution conventions outlined at https://www.gnu.org/software/make/manual/html_node/Makefile-Conventions.html https://www.gnu.org/software/make/manual/html_node/Makefile-... and https://www.gnu.org/prep/standards/standards.html#Managing-Releases https://www.gnu.org/prep/standards/standards.html#Managing-R...
  - sneak 6y ago
    Autotools are hot garbage stuck in the 1980s, and I can't fault anyone for avoiding the writing or advice of the organization that has not only failed to modernize them but has also stolen decades or centuries of person-hours of life from people due to the terrible inefficiency of this piece of critical infrastructure.
    - deleted 6y ago
      [deleted]
  - ForHackernews 6y ago
    > for so many programmers these days a Makefile is just a container for ad hoc shell code. Not gonna lie, that's completely what it is for me. I don't work in any compiled languages, and I don't have any kind of a "build target" (unless you count a Docker image?) in my workflow. Some old timers insist that "make test" (what are you 'making'?) is more intuitive than "run_tests.sh" so I indulge them. But it's just a list of 'phony' and some weirdly-formatted bash with tabs.
- canadianfella 6y ago
  Policies
guitarbill 6y ago
Poetry solves a lot of these problems; though not the test or lint command. For example, no more messing around with virtualenvs, sensibly declare normal and dev dependencies. The commands are useful, getting support for declaring easy to use test and lint commands in pyproject.toml shouldn't be so hard and something I'd love to see. In the meantime, you can use shell scripts (or a makefile sans the venv stuff) and do e.g. `poetry run ./test`. Tbh, there's nothing wrong with a makefile, and poetry isn't perfect. I just feel poetry works better than trying to work around the limitation of existing tooling. An so far, I'm happy with pyproject.toml. Putting dev tool configuration in there is a lot nicer than having several different files with different syntax for e.g. isort, pylint, etc
- npage97 6y ago
  take a look at taskipy https://github.com/illBeRoy/taskipy https://github.com/illBeRoy/taskipy as it resembles npm scripts.
  - guitarbill 6y ago
    that's awesome, pretty much exactly what i wanted! thanks
hansvm 6y ago
I've gotten in the habit lately of using SHELL and SHELLFLAGS (usually with ONESHELL) to just use a virtual environment for all the commands. You can't possibly forget to enter the virtualenv, and you also don't have to litter your makefile with extra variables. Does anyone else have other approaches beyond the two from the article?
- venthur 6y ago
  Can you show an example?
snicker7 6y ago
Using make as a command runner (instead of an actual build system) feels bitter-sweet. I do that. I no longer trust make as an actual build system (too easy to be incorrect, dynamic targets requires gymnastics or makefile generators). But, as a user, make feels very familiar.

31 points by smitty1e 6y ago

I've used the former to good effect. A phony help target for a default can print a menu, so that the only thing to know/remember is to type: `make`. However, for setting up the whole environment, I'm looking for a cookiecutter-based or -integrated solution.

sneak 6y ago

One of my policy for all employees or contractors is that "make" (and optionally "make test") in the root of a repo should always "do the thing". We have a de facto, decades old standard for a "run the tests" or a "build the damn thing" big red button. We should use it, even if it's just a few line wrapper over npm run or docker build or whatever. (The fact that it could be either is the reason that there should be a standardized entrypoint.)

wahern 6y ago
I wish more people would read https://www.gnu.org/software/make/manual/html_node/Standard-Targets.html https://www.gnu.org/software/make/manual/html_node/Standard-.... Once upon a time anybody using a Unix-like OS was familiar with most of those standard targets as a consequence of the ubiquity of autotools, especially when package repositories weren't so comprehensive and it was more common to download and build tarballs locally. But at my current job, there's no rhyme or reason to make targets. Something like `make build` (where build may or may not be the default target, likely accidentally if it is) is more likely to install Go toolchain dependencies than to build any of the project source code. And of course little if any attempt is made to actually declare dependencies, correct or broken, as for so many programmers these days a Makefile is just a container for ad hoc shell code. I don't often make use of autotools in my own projects as I can usually get good mileage from plain POSIX make, switching to GNU make in a bind, but I religiously try to adhere to the standard targets and other build and distribution conventions outlined at https://www.gnu.org/software/make/manual/html_node/Makefile-Conventions.html https://www.gnu.org/software/make/manual/html_node/Makefile-... and https://www.gnu.org/prep/standards/standards.html#Managing-Releases https://www.gnu.org/prep/standards/standards.html#Managing-R...
- sneak 6y ago
  Autotools are hot garbage stuck in the 1980s, and I can't fault anyone for avoiding the writing or advice of the organization that has not only failed to modernize them but has also stolen decades or centuries of person-hours of life from people due to the terrible inefficiency of this piece of critical infrastructure.
  - deleted 6y ago
    [deleted]
- ForHackernews 6y ago
  > for so many programmers these days a Makefile is just a container for ad hoc shell code. Not gonna lie, that's completely what it is for me. I don't work in any compiled languages, and I don't have any kind of a "build target" (unless you count a Docker image?) in my workflow. Some old timers insist that "make test" (what are you 'making'?) is more intuitive than "run_tests.sh" so I indulge them. But it's just a list of 'phony' and some weirdly-formatted bash with tabs.
canadianfella 6y ago
Policies

guitarbill 6y ago

Poetry solves a lot of these problems; though not the test or lint command. For example, no more messing around with virtualenvs, sensibly declare normal and dev dependencies. The commands are useful, getting support for declaring easy to use test and lint commands in pyproject.toml shouldn't be so hard and something I'd love to see. In the meantime, you can use shell scripts (or a makefile sans the venv stuff) and do e.g. `poetry run ./test`. Tbh, there's nothing wrong with a makefile, and poetry isn't perfect. I just feel poetry works better than trying to work around the limitation of existing tooling. An so far, I'm happy with pyproject.toml. Putting dev tool configuration in there is a lot nicer than having several different files with different syntax for e.g. isort, pylint, etc

npage97 6y ago
take a look at taskipy https://github.com/illBeRoy/taskipy https://github.com/illBeRoy/taskipy as it resembles npm scripts.
- guitarbill 6y ago
  that's awesome, pretty much exactly what i wanted! thanks

hansvm 6y ago

I've gotten in the habit lately of using SHELL and SHELLFLAGS (usually with ONESHELL) to just use a virtual environment for all the commands. You can't possibly forget to enter the virtualenv, and you also don't have to litter your makefile with extra variables. Does anyone else have other approaches beyond the two from the article?

venthur 6y ago
Can you show an example?

snicker7 6y ago

Using make as a command runner (instead of an actual build system) feels bitter-sweet. I do that. I no longer trust make as an actual build system (too easy to be incorrect, dynamic targets requires gymnastics or makefile generators). But, as a user, make feels very familiar.