4 ms·

And chaos ensued. Unable to install aws-cli via pip: https://github.com/aws/aws-cli/issues/8036 https://github.com/aws/aws-cli/issues/8036

• x3n0ph3n3 3y ago
  Mostly because of PyYAML: https://github.com/yaml/pyyaml/issues/724 https://github.com/yaml/pyyaml/issues/724 I was pretty annoyed to discover that it had a separate build-time dependency on cython that you can't control from your requirements.txt.
  • jborean93 3y ago
    That's unfortunately a pip limitation. You can either disable build isolation so that the build requirements are using your current environment or you can use the PIP_CONSTRAINTS env var which will be passed through to the isolated build invocation allowing you to constrain the build requirements. In this case PyYAML did release a 6.0.1 version which contains an upper bound constraint on Cython.
    • raverbashing 3y ago
      Wonder if Cython didn't think of checking with their biggest "customers" for compatibility
      • roblabla 3y ago
        PyYAML knew about the breakage since january 2022[0], and nothing really happened. After a year and a half with lots of alphas and betas, I don't think there is much cython could do, short of fixing PyYAML themselves. [0]: https://github.com/yaml/pyyaml/issues/601 https://github.com/yaml/pyyaml/issues/601
    • x3n0ph3n3 3y ago
      That's what I ended up doing just for the pyyaml installation.
  • stabbles 3y ago
    In Spack [1] we can express all these constraints for the dependency solver, and we also try to always re-cythonize sources [2]. The latter is because bundled cythonized files are sometimes forward incompatible with Python, so it's better to just regenerate those with an up to date cython. [1] https://github.com/spack/spack/ https://github.com/spack/spack/ [2] https://github.com/spack/spack/pull/35995 https://github.com/spack/spack/pull/35995
    • x3n0ph3n3 3y ago
      Oh great, yet _another_ Python package manager.
      • zincfingers 3y ago
        Spack is a lot more than just python package manager :)
      • cowsandmilk 3y ago
        It’s not a python package manager. It is a generalized package manager written in python.
    • gyrovagueGeist 3y ago
      Neat! I had no idea that Spack can be used for cythonize dependencies. I use it to manage my local compiler/CUDA/Trilinos stack but never considered it for non C++ things.
    • jborean93 3y ago
      That's what PyYAML does as well. It uses PEP 518 [1] to specify the dependencies for the build which PyYAML has included Cython [2]. It's just that for previous releases there was no upper bound here so pip and other tools just selected the latest version which was incompatible. In the past PyYAML included the cythonised .c files in the sdist but as of 5.4.0 they went the PEP 518 route and ensures the client will cythonise them if installing from the sdist. [1] - https://peps.python.org/pep-0518/ https://peps.python.org/pep-0518/ [2] - https://github.com/yaml/pyyaml/blob/release/6.0/pyproject.toml#L2 https://github.com/yaml/pyyaml/blob/release/6.0/pyproject.to...
  • formerly_proven 3y ago
    I like how the log in the issue also has another warning for breaking changes in it The license_file parameter is deprecated, use license_files instead. By 2023-Oct-30, you need to update your project and remove deprecated calls or your builds will no longer be supported. See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details. The level of churn and breakage in these basal python packaging packages is honestly insane.
• gchamonlive 3y ago
  I am reluctant to refer you to aws-cli V2 because it is way more involved to install. No idea why aws moved away from their original distribution method... However, the V2 is going to be the way to go moving forwards :/
  • garblegarble 3y ago
    >No idea why aws moved away from their original distribution method Didn't they move away exactly because of issues like this? As I understand it, aws-cli v2 is still written in Python but ships everything it needs self-contained so it can avoid Python dependency hell[1] 1: https://www.youtube.com/watch?v=U5y7JI_mHk8#t=3m40 https://www.youtube.com/watch?v=U5y7JI_mHk8#t=3m40
    • gchamonlive 3y ago
      Indeed!
    • PartiallyTyped 3y ago
      Funnily enough, I didn’t have issues with Python’s dependency hell before working for AWS, not because we have more dependencies, but because… reasons.
      • RandomThrow321 3y ago
        Is it special files that begin with a capital letter, version sets, version filter instances, and massive symlink farms?
        • PartiallyTyped 3y ago
          If you already know that much then you don’t need a response ;)
  • takeda 3y ago
    cat -n shell.nix { pkgs ? import <nixpkgs> {} }: pkgs.mkShell { name = "dev-env"; buildInputs = with pkgs; [ awscli2 ]; } And then you can have it available when entering shell with nix-shell.
• BiteCode_dev 3y ago
  And this is why you pin your dependencies + never install the latest version of anything.
  • stabbles 3y ago
    The lesson here is different: package developers must put upperbounds on major versions of dependencies that follow semver.
    • pdhborges 3y ago
      Should this be the take away? It looks to me that people are building pipelines without taking into account that they change roles every time they have to build a package, from a package consumer to a package packager. Even if the package developers set an upper bound on the build dependency it is the packagers responsibility to provide a deterministic build environment.