16 ms·

python packages that include stuff like scripts hardcode the full, explicit prefix in the #! lines. Such scripts are broken. One should fix the scripts instead

by moe 17y ago

python packages that include stuff like scripts hardcode the full, explicit prefix in the #! lines. Such scripts are broken. One should fix the scripts instead of mangling the environment. It's a one-liner: find /opt/myproject -type f -exec sed -i 's/#!\/usr\/bin\/python/#!\/usr\/bin\/env python/' {} \; but if you move it, everything breaks in unexpected ways. If you move it then PYTHONPATH is your friend. Once you have a python build for each project (which you want to have anyways) you'll notice that you don't need virtualenv or other kludges anymore either. Yep, it's really that easy. 4 lines of code. How many lines was yours again?

jnoller 17y ago
Did you read the comment above this? Sed is the initial hack to fix it, but it's still a hack; not a fix. Eventually I punted on it for the Linux dist, but I'm probably going to be stuck with the sed fix for OS/X. As for pythonpath - that has nothing to do with moving the entire directory path, including the interpreter, /bin dir, etc.
- moe 17y ago
  Sorry, I just can't follow where your exact problem is. I have 9 python projects under /opt/foo* right now. Each of them has their own python in /opt/foo*/sys/python. I su to the respective user who has PATH and PYTHONPATH set appropiately in its profile - and everything works as expected. I also don't follow why you consider sed a "hack" here. A hardcoded hashbang line is clearly a Bug - but gladly one that is trivially fixed with that one-liner. On the other side we have your "fix" that involves permanently mangling the environment in strange ways. In my book you are the one proposing the hack here.
  - jnoller 17y ago
    I'm not proposing mangling anything here; in fact I dropped the mangled environment (as it was a hack, and added too much space) and swapped to a simpler (and cleaner) virtualenv built in /opt. I still haven't fixed the build something and upload it for people, because of the hardcoded shebang, and yes - sed could fix that.
    - moe 17y ago
      Also, you have to run the sed command over the tree every time you install a new package which installs a binary How often do your projects depend on something else, written in python, that installs a binary? For me that number is zero. iirc all of my python dependencies are libraries. And if I stumbled into something that comes with a binary, with a broken hashbang line, then I'd be rather motivated to not only fix it, but to also feed the fix upstream. If the project is useful enough for me to use it then those 10 minutes of my time for a brown-paper-bag micropatch are a no-brainer. Well, to each their own. - But I still think this is a solution looking for a problem, with the potential to create more problems than it solves.
      - jnoller 17y ago
        Not exactly - for example, I depend on fabric, PIP, easy_install, nose, pylint and others. In this case, it's not a matter of fixing upstream on a per-project basis, it's fixing distutils as this is "magical" behavior I (and others) have seen, and that's not a pile of gators I (or I suspect you) would want to wade into. For example read the following threads: http://mail.python.org/pipermail/python-dev/2009-July/090259.html http://mail.python.org/pipermail/python-dev/2009-July/090259... http://mail.python.org/pipermail/python-dev/2009-July/090356.html http://mail.python.org/pipermail/python-dev/2009-July/090356... And so on. I agree wholeheartedly though - library-only dependencies are stupid simple.
        moe 17y ago
        I just checked and interestingly I also depend on easy_install (which I don't even use, but put it in there someday), nose and pylint. None of them gave me problems. I say if fabric and PIP are broken then fix them. easy_install is irrelevant anyways, nobody in their right mind would use that on a production system.
        jnoller 17y ago
        I'm surprised having the hardcoded paths didn't bite you - of course, how frequently are you moving the interpreter, or installing multiple versions of the interpreter (with the same dependencies) with a shared bin dir?
  - jnoller 17y ago
    Also, you have to run the sed command over the tree every time you install a new package which installs a binary

by moe 17y ago

jnoller 17y ago

Did you read the comment above this? Sed is the initial hack to fix it, but it's still a hack; not a fix. Eventually I punted on it for the Linux dist, but I'm probably going to be stuck with the sed fix for OS/X. As for pythonpath - that has nothing to do with moving the entire directory path, including the interpreter, /bin dir, etc.

moe 17y ago
Sorry, I just can't follow where your exact problem is. I have 9 python projects under /opt/foo* right now. Each of them has their own python in /opt/foo*/sys/python. I su to the respective user who has PATH and PYTHONPATH set appropiately in its profile - and everything works as expected. I also don't follow why you consider sed a "hack" here. A hardcoded hashbang line is clearly a Bug - but gladly one that is trivially fixed with that one-liner. On the other side we have your "fix" that involves permanently mangling the environment in strange ways. In my book you are the one proposing the hack here.
- jnoller 17y ago
  I'm not proposing mangling anything here; in fact I dropped the mangled environment (as it was a hack, and added too much space) and swapped to a simpler (and cleaner) virtualenv built in /opt. I still haven't fixed the build something and upload it for people, because of the hardcoded shebang, and yes - sed could fix that.
  - moe 17y ago
    Also, you have to run the sed command over the tree every time you install a new package which installs a binary How often do your projects depend on something else, written in python, that installs a binary? For me that number is zero. iirc all of my python dependencies are libraries. And if I stumbled into something that comes with a binary, with a broken hashbang line, then I'd be rather motivated to not only fix it, but to also feed the fix upstream. If the project is useful enough for me to use it then those 10 minutes of my time for a brown-paper-bag micropatch are a no-brainer. Well, to each their own. - But I still think this is a solution looking for a problem, with the potential to create more problems than it solves.
    - jnoller 17y ago
      Not exactly - for example, I depend on fabric, PIP, easy_install, nose, pylint and others. In this case, it's not a matter of fixing upstream on a per-project basis, it's fixing distutils as this is "magical" behavior I (and others) have seen, and that's not a pile of gators I (or I suspect you) would want to wade into. For example read the following threads: http://mail.python.org/pipermail/python-dev/2009-July/090259.html http://mail.python.org/pipermail/python-dev/2009-July/090259... http://mail.python.org/pipermail/python-dev/2009-July/090356.html http://mail.python.org/pipermail/python-dev/2009-July/090356... And so on. I agree wholeheartedly though - library-only dependencies are stupid simple.
      - moe 17y ago
        I just checked and interestingly I also depend on easy_install (which I don't even use, but put it in there someday), nose and pylint. None of them gave me problems. I say if fabric and PIP are broken then fix them. easy_install is irrelevant anyways, nobody in their right mind would use that on a production system.
        jnoller 17y ago
        I'm surprised having the hardcoded paths didn't bite you - of course, how frequently are you moving the interpreter, or installing multiple versions of the interpreter (with the same dependencies) with a shared bin dir?
- jnoller 17y ago
  Also, you have to run the sed command over the tree every time you install a new package which installs a binary