4 ms·

How well does this work for projects that include dependencies that are still reliant on the old setup.py and don't have wheels published?

droelf 3y ago
It doesn't work well for those yet. rip only deals with wheel files for now. The way (afaik) pip works for source dist packages is that it locally builds the wheel, and then extracts the metadata from that locally built wheel. This is also the approach that we want to take with rip. As you can imagine, the performance will be quite bad (but nothing we can do about that). Ideally, almost all packages should ship wheels.
- glandium 3y ago
  > all packages should ship wheels. And then a new python version is released and virtually no packages with native modules have wheels for that new python version. (too few use abi3) Also, good luck finding wheels for e.g. s390x linux.
- jborean93 3y ago
  > Ideally, almost all packages should ship wheels. It would be ideal but it's not always possible. I maintain a GSSAPI/KRB5 library that wraps the C libs but due to PyPI wheel policies I cannot upload a wheel without embedding those C libs which then opens up a whole bunch of problems around deps and lib conflicts :(
- stuaxo 3y ago
  Ideally - however there are points where it's not possible yet. Bindings like pycairo are a good example. On windows a WHL can be provided. On Linux it can't - Cairo (which pycairo binds to) is shipped by the distro, and distros are free to enable or disable different features. Since the C bindings are linked to the cairo shared library, and that includes all the backends you can't build that and know if would work in a distro. There are two use cases for pycairo - in one you might not care what the system provides and if the pycairo who provided its own Cairo shared object, that would be fine. In the second use case you want to use the system provides Cairo, e.g. to work with Gtk. It's impossible to resolve this workout changing how Cairo itself works and it's API. The result is no pycairo whl on Linux, and users who have to install all the dependencies (which once go to pango you pull in Gtk, X, freetype etc). Cairo isn't the only example but it's one I know. A lot of these are libraries that bind system libraries and predate virtualenv / venv - when they were created compiling things was fine.
westurner 3y ago
pypa/cibuildwheel: https://github.com/pypa/cibuildwheel https://github.com/pypa/cibuildwheel : > Example setup: To build manylinux, musllinux, macOS, and Windows wheels on GitHub Actions, you could use this .github/workflows/wheels.yml
- ArchOversight 3y ago
  That's great and all for packages I control, but I publish wheels already.
  - westurner 3y ago
    https://pythonwheels.com/ https://pythonwheels.com/ now lists only three packages that are not yet published to PyPI as wheels. But do you or others build them with SLSA 3? https://slsa.dev/get-started#slsa-3 https://slsa.dev/get-started#slsa-3
    - ArchOversight 3y ago
      There's thousands more packages than listed on that website. Also now you are moving the goalposts. Python wheels exist but not all Python packages are distributed as wheels. If I am going through the effort to build wheels myself, then I don't need a resolver since I can just make sure to pick all the versions I need. Then I don't need rip to install anything.

droelf 3y ago

It doesn't work well for those yet. rip only deals with wheel files for now. The way (afaik) pip works for source dist packages is that it locally builds the wheel, and then extracts the metadata from that locally built wheel. This is also the approach that we want to take with rip. As you can imagine, the performance will be quite bad (but nothing we can do about that). Ideally, almost all packages should ship wheels.

glandium 3y ago
> all packages should ship wheels. And then a new python version is released and virtually no packages with native modules have wheels for that new python version. (too few use abi3) Also, good luck finding wheels for e.g. s390x linux.
jborean93 3y ago
> Ideally, almost all packages should ship wheels. It would be ideal but it's not always possible. I maintain a GSSAPI/KRB5 library that wraps the C libs but due to PyPI wheel policies I cannot upload a wheel without embedding those C libs which then opens up a whole bunch of problems around deps and lib conflicts :(
stuaxo 3y ago
Ideally - however there are points where it's not possible yet. Bindings like pycairo are a good example. On windows a WHL can be provided. On Linux it can't - Cairo (which pycairo binds to) is shipped by the distro, and distros are free to enable or disable different features. Since the C bindings are linked to the cairo shared library, and that includes all the backends you can't build that and know if would work in a distro. There are two use cases for pycairo - in one you might not care what the system provides and if the pycairo who provided its own Cairo shared object, that would be fine. In the second use case you want to use the system provides Cairo, e.g. to work with Gtk. It's impossible to resolve this workout changing how Cairo itself works and it's API. The result is no pycairo whl on Linux, and users who have to install all the dependencies (which once go to pango you pull in Gtk, X, freetype etc). Cairo isn't the only example but it's one I know. A lot of these are libraries that bind system libraries and predate virtualenv / venv - when they were created compiling things was fine.

westurner 3y ago

pypa/cibuildwheel: https://github.com/pypa/cibuildwheel https://github.com/pypa/cibuildwheel : > Example setup: To build manylinux, musllinux, macOS, and Windows wheels on GitHub Actions, you could use this .github/workflows/wheels.yml

ArchOversight 3y ago
That's great and all for packages I control, but I publish wheels already.
- westurner 3y ago
  https://pythonwheels.com/ https://pythonwheels.com/ now lists only three packages that are not yet published to PyPI as wheels. But do you or others build them with SLSA 3? https://slsa.dev/get-started#slsa-3 https://slsa.dev/get-started#slsa-3
  - ArchOversight 3y ago
    There's thousands more packages than listed on that website. Also now you are moving the goalposts. Python wheels exist but not all Python packages are distributed as wheels. If I am going through the effort to build wheels myself, then I don't need a resolver since I can just make sure to pick all the versions I need. Then I don't need rip to install anything.