3 ms·

Particularly sad that even projects that attempt to pin their dependencies are running into this issue: if a dependency doesn’t have a pre-built wheel, then it

Particularly sad that even projects that attempt to pin their dependencies are running into this issue: if a dependency doesn’t have a pre-built wheel, then it will be built in an “isolated” environment, which means it won’t inherit a pinned setuptools version from the root project, and will instead pull the latest (unless the dependency itself is pinning setuptools). Reproducible builds can’t come too soon.

pjc50 2y ago
Having "install package" and "build package from source" be indistinguishable command lines is yet another massive design failure in the Python packaging system. It may sometimes, vaguely, work on the original programmer's machine. But the build environment isn't reproducible, and on Windows is often not present at all so installing the package simply falls over. (yes, we're using GRPC-on-Windows-Python, which means we're tied to the release schedule of wheels for that)
- physicsguy 2y ago
  Until about 7 or 8 years ago, install pretty much always meant build from source; wheels only came in fairly recently in the history of Python and are still not universal, as you've found out. You'll typically have problems on: * ARM Linux * POWER9/10 * Alpine distros which use MUSL for the C library (https://rpep.dev/posts/alpine-python-antipattern/ https://rpep.dev/posts/alpine-python-antipattern/) * Some packages which depend on C libraries that are difficult to build on Windows (fault of the C libraries rather than Python really). Conda (and previous tools before it like Enthought Canopy) were designed to try to fix this problem, since the core Python packaging tooling just wasn't good enough. Wheels were proposed as a PEP extension and adapted but it took years for common packages to be built as wheels and even now there's nowhere near universal coverage for them even on common platforms.
  - eesmith 2y ago
    I only have source releases for macOS as I can't figure out how to include OpenMP for the binary wheels of my proprietary package.
    - physicsguy 2y ago
      You can't really do it, the best way is to build it with Clang installed with Homebrew and then put in the installation instructions to ask users to install OpenMP with `brew install libomp`. It really frustrates me that Apple strip it out of their build of Clang in favour of pushing their own Grand Central Dispatch.

pjc50 2y ago

Having "install package" and "build package from source" be indistinguishable command lines is yet another massive design failure in the Python packaging system. It may sometimes, vaguely, work on the original programmer's machine. But the build environment isn't reproducible, and on Windows is often not present at all so installing the package simply falls over. (yes, we're using GRPC-on-Windows-Python, which means we're tied to the release schedule of wheels for that)

physicsguy 2y ago
Until about 7 or 8 years ago, install pretty much always meant build from source; wheels only came in fairly recently in the history of Python and are still not universal, as you've found out. You'll typically have problems on: * ARM Linux * POWER9/10 * Alpine distros which use MUSL for the C library (https://rpep.dev/posts/alpine-python-antipattern/ https://rpep.dev/posts/alpine-python-antipattern/) * Some packages which depend on C libraries that are difficult to build on Windows (fault of the C libraries rather than Python really). Conda (and previous tools before it like Enthought Canopy) were designed to try to fix this problem, since the core Python packaging tooling just wasn't good enough. Wheels were proposed as a PEP extension and adapted but it took years for common packages to be built as wheels and even now there's nowhere near universal coverage for them even on common platforms.
- eesmith 2y ago
  I only have source releases for macOS as I can't figure out how to include OpenMP for the binary wheels of my proprietary package.
  - physicsguy 2y ago
    You can't really do it, the best way is to build it with Clang installed with Homebrew and then put in the installation instructions to ask users to install OpenMP with `brew install libomp`. It really frustrates me that Apple strip it out of their build of Clang in favour of pushing their own Grand Central Dispatch.