5 ms·

This is such a strange and annoying compromise to make. It's somewhat antithetical to the concept of RHEL versions being a standardized set of packages such th

This is such a strange and annoying compromise to make. It's somewhat antithetical to the concept of RHEL versions being a standardized set of packages such that when someone says "I have RHEL 7", you can say "My software will work on that". Now in order to support RHEL 8, you have to tell them to also have their sys admin install your preferred python. A better and less bespoke to RHEL solution imo would have been to make an anaconda install or virtualenv the default python for each user.

mavhc 8y ago
Wouldn't you just specify the dependencies in your installer so it just works?
- CoolGuySteve 8y ago
  Most people in my field don't use installers or rpms, we just share the git repo directly. I'm kind of pissed off just thinking about how many build systems will break when '/usr/bin/env python' doesn't return anything by default.
  - Spivak 8y ago
    Which is fine but how do you specify dependencies right now? Surely a minimal install of RHEL doesn't have everything you need.
    - mroche 8y ago
      Not meant to be snarky, just a tad confused on what exactly you’re asking. Have you ever made an RPM yourself? Dependencies are specified there. Or are you asking about dependencies that don’t exist in standard RHEL repositories?
      - Spivak 8y ago
        The parent was saying that specifying dependencies with rpm wasn't possible because they deploy with git but surely they must have something to specify/install deps since a RHEL base likely doesn't have everything they need especially if they're using the system python.
        mroche 8y ago
        I think at that point we’re kind of stuck with virtual environments for Python dependencies. I’m not too sure what the best solution is.
        isostatic 8y ago
        No doubt they have one of those "curl http://site.com/script.sh http://site.com/script.sh && sudo bash script.sh" installation instructions. Sigh.
  - KerrickStaley 8y ago
    Note: You shouldn't use #!/usr/bin/env python. Instead you should explicitly specify python2 or python3, unless your script is compatible with both.
    - throwaway98434 8y ago
      You say "you" as if you're talking to the person you're replying to. Most scripts in the world aren't authored by the person you're replying to. They're powerless to change the ocean of python out in the world. For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin I can't find a single one that runs on python3. No individual user is going to be fixing this. It's a very silly and shortsighted move on RedHat's part.
      - isostatic 8y ago
        Of the 50 /usr/bin/ python files on my Ubuntu 16.04 desktop, most are versioned in some way. None are called from "env". 37 /usr/bin/python3 4 /usr/bin/python3.5 4 /usr/bin/python 3 /usr/bin/python2.7 2 /usr/bin/python2 The 4 that aren't versioned are speedtest, speedtest-cli, dh_python2, apt-offline So two from ubuntu/debian, and two from a package
        throwaway98434 8y ago
        Right, someone working in ubuntu distro development has done this work for those packages specifically. Above, we're discussing some build system where someone hasn't done this work. Most collections of python code in the world (I'm going to hazard a guess significantly upward of 90%) just use /usr/bin/python or /usr/bin/env python. That's the nature of the problem.
        isostatic 8y ago
        You're discussing RHEL7's /usr/bin. > For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin I posted Ubuntu 1604's in comparison.
  - rterzi 8y ago
    #!/usr/bin/env python is almost guaranteed to give you a surprise at some point. If someone has a virtualenv or conda env active in that shell it will be a different python environment (with different installed modules) than in other shells.
    - CoolGuySteve 8y ago
      At least in the build-system glue language use case, the 'batteries included' modules like os, sh, glob, etc will all be there. Similarly, if you require pandas like I do, there's a fairly comprehensive baseline of functionality you can expect regardless of what environment it comes from.
  - hedora 8y ago
    I’m pissed almost daily at how many build systems break because they need 3.5.3 vs 3.5.4 or whatever, and at how many only work because they create a new virtual environment for every single script (Virtual environment ~= copy of python auto-downloaded from some random places). Also, FFS, why can’t pyc files have a bytecode format version number embedded in them, or at least be called .pyc3 by python 3? The whole ecosystem needs to solve the packaging problem, or get replaced by some other language ASAP. Instead, it’s breaking entire distros with its packaging tire fire. For many python projects I’ve dealt with, it is easier to reimplement that dang thing in some other language than make the python build reliable, secure and redistributable.
    - umanwizard 8y ago
      Maybe Python should stop changing so often? With C++ for example you are usually either targeting C++11, C++14, or C++17 (or if you're really unfortunate, C++03). The set of features in the language only changes every few years. Granted it's still possible that someone could have an old compiler that doesn't implement the advertised standard correctly, but that's a bug, not a deliberate design choice.
      - antoinealb 8y ago
        Thats not really fair. Python 3.2 was release in 2011 and Python 3.4 in 2014. Only 3.3 was released in the middle, I would not say it is only "so often" compared to c++. To be more general, C++ had 3 major versions from 2011 to 2017, while python had 5. Its more, but not significantly so (especially since very little compatibility is broken between 3.4 and 3.5 for example). And if you are talking about which version is shipped with a distro, then RHEL7 ships with gcc 4.8, which support only parts of c++14, while for example the latest Ubuntu LTS ships GCC 7,3, meaning you get full C++17 support. Don't get me wrong, the Python packaging situation is not very clean. But it is far from being an exception, and people complaining about it are just spreading FUD.
        StillBored 8y ago
        But the compiler is the same, so you specify the dialect you expect to use rather than running a different binary. Although to be fair, at least one distro decided to start shipping gcc like gcc-X where X is the version, so you can have multiple GCC's installed. I'm not sure that was the best idea, but its what they did.
  - solarkraft 8y ago
    Can't you just symlink your preferred python?
    - rterzi 8y ago
      No need to symlink manually, the `alternatives` mechanism will handle that for you.
collinmanderson 8y ago
My guess is a lot of sysadmins will just end up installing both 2 and 3.
madhadron 8y ago
The best solution I am aware of is to package your Python apps statically with the version they expect. For example, Facebook's XAR format (https://github.com/facebookincubator/xar/ https://github.com/facebookincubator/xar/) does this very nicely.
- badsectoracula 8y ago
  This looks remarkably similar to AppImage (which i suppose can also be used to package your own Python).

mavhc 8y ago

Wouldn't you just specify the dependencies in your installer so it just works?

CoolGuySteve 8y ago
Most people in my field don't use installers or rpms, we just share the git repo directly. I'm kind of pissed off just thinking about how many build systems will break when '/usr/bin/env python' doesn't return anything by default.
- Spivak 8y ago
  Which is fine but how do you specify dependencies right now? Surely a minimal install of RHEL doesn't have everything you need.
  - mroche 8y ago
    Not meant to be snarky, just a tad confused on what exactly you’re asking. Have you ever made an RPM yourself? Dependencies are specified there. Or are you asking about dependencies that don’t exist in standard RHEL repositories?
    - Spivak 8y ago
      The parent was saying that specifying dependencies with rpm wasn't possible because they deploy with git but surely they must have something to specify/install deps since a RHEL base likely doesn't have everything they need especially if they're using the system python.
      - mroche 8y ago
        I think at that point we’re kind of stuck with virtual environments for Python dependencies. I’m not too sure what the best solution is.
      - isostatic 8y ago
        No doubt they have one of those "curl http://site.com/script.sh http://site.com/script.sh && sudo bash script.sh" installation instructions. Sigh.
- KerrickStaley 8y ago
  Note: You shouldn't use #!/usr/bin/env python. Instead you should explicitly specify python2 or python3, unless your script is compatible with both.
  - throwaway98434 8y ago
    You say "you" as if you're talking to the person you're replying to. Most scripts in the world aren't authored by the person you're replying to. They're powerless to change the ocean of python out in the world. For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin I can't find a single one that runs on python3. No individual user is going to be fixing this. It's a very silly and shortsighted move on RedHat's part.
    - isostatic 8y ago
      Of the 50 /usr/bin/ python files on my Ubuntu 16.04 desktop, most are versioned in some way. None are called from "env". 37 /usr/bin/python3 4 /usr/bin/python3.5 4 /usr/bin/python 3 /usr/bin/python2.7 2 /usr/bin/python2 The 4 that aren't versioned are speedtest, speedtest-cli, dh_python2, apt-offline So two from ubuntu/debian, and two from a package
      - throwaway98434 8y ago
        Right, someone working in ubuntu distro development has done this work for those packages specifically. Above, we're discussing some build system where someone hasn't done this work. Most collections of python code in the world (I'm going to hazard a guess significantly upward of 90%) just use /usr/bin/python or /usr/bin/env python. That's the nature of the problem.
        isostatic 8y ago
        You're discussing RHEL7's /usr/bin. > For example, out of about 50 or so /usr/bin/python scripts I see in a RHEL7's /usr/bin I posted Ubuntu 1604's in comparison.
- rterzi 8y ago
  #!/usr/bin/env python is almost guaranteed to give you a surprise at some point. If someone has a virtualenv or conda env active in that shell it will be a different python environment (with different installed modules) than in other shells.
  - CoolGuySteve 8y ago
    At least in the build-system glue language use case, the 'batteries included' modules like os, sh, glob, etc will all be there. Similarly, if you require pandas like I do, there's a fairly comprehensive baseline of functionality you can expect regardless of what environment it comes from.
- hedora 8y ago
  I’m pissed almost daily at how many build systems break because they need 3.5.3 vs 3.5.4 or whatever, and at how many only work because they create a new virtual environment for every single script (Virtual environment ~= copy of python auto-downloaded from some random places). Also, FFS, why can’t pyc files have a bytecode format version number embedded in them, or at least be called .pyc3 by python 3? The whole ecosystem needs to solve the packaging problem, or get replaced by some other language ASAP. Instead, it’s breaking entire distros with its packaging tire fire. For many python projects I’ve dealt with, it is easier to reimplement that dang thing in some other language than make the python build reliable, secure and redistributable.
  - umanwizard 8y ago
    Maybe Python should stop changing so often? With C++ for example you are usually either targeting C++11, C++14, or C++17 (or if you're really unfortunate, C++03). The set of features in the language only changes every few years. Granted it's still possible that someone could have an old compiler that doesn't implement the advertised standard correctly, but that's a bug, not a deliberate design choice.
    - antoinealb 8y ago
      Thats not really fair. Python 3.2 was release in 2011 and Python 3.4 in 2014. Only 3.3 was released in the middle, I would not say it is only "so often" compared to c++. To be more general, C++ had 3 major versions from 2011 to 2017, while python had 5. Its more, but not significantly so (especially since very little compatibility is broken between 3.4 and 3.5 for example). And if you are talking about which version is shipped with a distro, then RHEL7 ships with gcc 4.8, which support only parts of c++14, while for example the latest Ubuntu LTS ships GCC 7,3, meaning you get full C++17 support. Don't get me wrong, the Python packaging situation is not very clean. But it is far from being an exception, and people complaining about it are just spreading FUD.
      - StillBored 8y ago
        But the compiler is the same, so you specify the dialect you expect to use rather than running a different binary. Although to be fair, at least one distro decided to start shipping gcc like gcc-X where X is the version, so you can have multiple GCC's installed. I'm not sure that was the best idea, but its what they did.
- solarkraft 8y ago
  Can't you just symlink your preferred python?
  - rterzi 8y ago
    No need to symlink manually, the `alternatives` mechanism will handle that for you.

collinmanderson 8y ago

My guess is a lot of sysadmins will just end up installing both 2 and 3.

madhadron 8y ago

The best solution I am aware of is to package your Python apps statically with the version they expect. For example, Facebook's XAR format (https://github.com/facebookincubator/xar/ https://github.com/facebookincubator/xar/) does this very nicely.

badsectoracula 8y ago
This looks remarkably similar to AppImage (which i suppose can also be used to package your own Python).