4 ms·

Just FYI, but you do not need to switch away from setuptools if you do not want to. `setup.py` isn't going away, rather it is no longer the interface that "buil

Just FYI, but you do not need to switch away from setuptools if you do not want to. `setup.py` isn't going away, rather it is no longer the interface that "build frontends" (i.e. pip, but also pypa/build) use to build the project (there is a well defined interface made up of a number of task specific functions).

eesmith 3y ago
Thanks. I do understand that, and even if PyPA and the setuptools maintainers wanted to get rid of it, there's such a large installed base that any transition would take at least a decade. Plus, as I learned yesterday, the cmake-based backend also depends on setuptools. Thing is, my setup.py is full of hacks and is a nuisance to use, so I would prefer a better solution. What kind of hacks? In the "python setup.py install" days I processed sys.argv to handle extra compile-time parameters, like "don't compile for OpenMP". When direct use of setup.py became unsupported, I switched to environment variables to pass in the configuration. While it works, I don't know if it's a long-term viable solution. I read about the push towards isolated environments and reproducible builds, which is not something you can get if your build depends on ephemeral environment variables which are not captured in the pyproject.toml or dependency hashes. (Remember, I distribute a source package, not a precompiled wheel.) The longest component in my distribution takes a minute to compile. Setuptools does not handle dependency tracking well, so even if I touch C code which only takes a second to compile, it still rebuilds the entire component. I have a hacked-up Makefile for when I really need to work on that part of the code and want faster turn-arounds, but I would prefer to have a unified build system that really understands build dependencies. I subclass build_ext to handle the per-file compilation args, per-compiler link args, and to add an '$ORIGIN' rpath for non-Darwin builds as I generate shared libraries which depend on other shared libraries. To make this work I had to monkeypatch setuptools implementation internals. I know how fragile it is, and would like something better, which I could depend on being around for another 15 years. But when I dip into the Packaging world, I feel like my needs are in a forgotten corner, with the main impetuous being the needs of enterprise systems with engineering staff - a far cry from the packaging world I started with 20 years ago where building C extensions was one of the primary concerns.

eesmith 3y ago

Thanks. I do understand that, and even if PyPA and the setuptools maintainers wanted to get rid of it, there's such a large installed base that any transition would take at least a decade. Plus, as I learned yesterday, the cmake-based backend also depends on setuptools. Thing is, my setup.py is full of hacks and is a nuisance to use, so I would prefer a better solution. What kind of hacks? In the "python setup.py install" days I processed sys.argv to handle extra compile-time parameters, like "don't compile for OpenMP". When direct use of setup.py became unsupported, I switched to environment variables to pass in the configuration. While it works, I don't know if it's a long-term viable solution. I read about the push towards isolated environments and reproducible builds, which is not something you can get if your build depends on ephemeral environment variables which are not captured in the pyproject.toml or dependency hashes. (Remember, I distribute a source package, not a precompiled wheel.) The longest component in my distribution takes a minute to compile. Setuptools does not handle dependency tracking well, so even if I touch C code which only takes a second to compile, it still rebuilds the entire component. I have a hacked-up Makefile for when I really need to work on that part of the code and want faster turn-arounds, but I would prefer to have a unified build system that really understands build dependencies. I subclass build_ext to handle the per-file compilation args, per-compiler link args, and to add an '$ORIGIN' rpath for non-Darwin builds as I generate shared libraries which depend on other shared libraries. To make this work I had to monkeypatch setuptools implementation internals. I know how fragile it is, and would like something better, which I could depend on being around for another 15 years. But when I dip into the Packaging world, I feel like my needs are in a forgotten corner, with the main impetuous being the needs of enterprise systems with engineering staff - a far cry from the packaging world I started with 20 years ago where building C extensions was one of the primary concerns.