3 ms·

I think it is fair, actually. This is not unique to hours-old python projects, it's a common theme among almost all python tools I've used. I have a suspicion

I think it is fair, actually. This is not unique to hours-old python projects, it's a common theme among almost all python tools I've used. I have a suspicion that that something written in Julia, Go, Rust, or possibly even C wouldn't have nearly this many issues. I'm not talking about debugging the actual functionality of the software, but rather the environment and tooling surrounding the language and software built with it. This project in particular should be an easy case because you know the hardware you'll be running on ahead of time. I'm ranting a bit, but I've tried so many tools based on python and almost _none_ of them built/installed/ran correctly on the happy path laid out in each project's readme. Anyway, sorry, rant over.

nl 4y ago
Yes there is a fair amount of truth in that. I do think that experience helps here. I have a recipe for installing Python that works on most python projects most of the time. git clone <project> python3 -m venv ./venv source ./venv/bin/activate pip install -r requirements.txt deactiviate # need to do this to include the correct command line tools in path (eg Jupyter) source ./venv/bin/activate Done. On a Linux or Intel Mac system this works with pretty much every reasonable Python project. On M1 Macs the situation isn't great at the moment, though.
- nerdponx 4y ago
  As if compiling any given C program wasn't also a crap shoot. People also tend to forget that these ML packages are ridiculously complicated, and have a lot of dependencies not just on other libraries but on particularities of your system. That, and ML researchers can't also be expected to be good at everything. They are busy doing ML research and waiting to be given a recipe to follow. Meanwhile I can put together a Python package in my sleep that works perfectly on pretty much any system, but I don't know a damn thing about Autotools and would probably make a total mess if I tried use it. Or CMake. Or whatever Java uses. This "look Python bad!!" stuff has some merit (if only historical), but it mostly amounts to FUD and does a big disservice to the people who have worked hard over the past few years to get everything fixed up. Plus at least one of the issues I see here is because people disregarded the instructions and used Python 3.9 even though it says to use 3.10 because the code assumes it's running under 3.10.
- bschwindHN 4y ago
  Noted! I'll try this out next time I have to use a python project and see how it goes.
  - nl 4y ago
    Notably, it works perfectly on the linked repo for this story (unlike the conda version which is the first comment and is a complete mess).
- nerdponx 4y ago
  Also, sorry to double post, but you don't actually need to activate the venv. You can just invoke `venv/bin/pip`. That should hopefully save you a bit of time and annoyance.
  - nl 4y ago
    That's bad advice for two reasons: Firstly, non-experiences Python users (the kind who need recipes to follow) will also blindly follow installation instructions on websites. These all say "pip install xxx" and users will inevitably forget they need to change the path to make that work for their environment. By getting into the habit of activating the environment this problem goes away. Secondly, you do need to activate the venv so that command line tools in ./venv/bin are used (instead of whatever is in your default path). This includes both the correct version of python (if you set it when you setup the virtual environment) and importantly jupyter. From personal experience there's a whole set of pain when you don't realize your jupyter isn't the one inside the virtual environment and so is using other libraries. Because many shells cache the location of executables, if you don't deactivate and reactivate your virtual environment after installing Jupyter into it then it may (in some circumstances) use a globally installed version.

nl 4y ago

Yes there is a fair amount of truth in that. I do think that experience helps here. I have a recipe for installing Python that works on most python projects most of the time. git clone <project> python3 -m venv ./venv source ./venv/bin/activate pip install -r requirements.txt deactiviate # need to do this to include the correct command line tools in path (eg Jupyter) source ./venv/bin/activate Done. On a Linux or Intel Mac system this works with pretty much every reasonable Python project. On M1 Macs the situation isn't great at the moment, though.

nerdponx 4y ago
As if compiling any given C program wasn't also a crap shoot. People also tend to forget that these ML packages are ridiculously complicated, and have a lot of dependencies not just on other libraries but on particularities of your system. That, and ML researchers can't also be expected to be good at everything. They are busy doing ML research and waiting to be given a recipe to follow. Meanwhile I can put together a Python package in my sleep that works perfectly on pretty much any system, but I don't know a damn thing about Autotools and would probably make a total mess if I tried use it. Or CMake. Or whatever Java uses. This "look Python bad!!" stuff has some merit (if only historical), but it mostly amounts to FUD and does a big disservice to the people who have worked hard over the past few years to get everything fixed up. Plus at least one of the issues I see here is because people disregarded the instructions and used Python 3.9 even though it says to use 3.10 because the code assumes it's running under 3.10.
bschwindHN 4y ago
Noted! I'll try this out next time I have to use a python project and see how it goes.
- nl 4y ago
  Notably, it works perfectly on the linked repo for this story (unlike the conda version which is the first comment and is a complete mess).
nerdponx 4y ago
Also, sorry to double post, but you don't actually need to activate the venv. You can just invoke `venv/bin/pip`. That should hopefully save you a bit of time and annoyance.
- nl 4y ago
  That's bad advice for two reasons: Firstly, non-experiences Python users (the kind who need recipes to follow) will also blindly follow installation instructions on websites. These all say "pip install xxx" and users will inevitably forget they need to change the path to make that work for their environment. By getting into the habit of activating the environment this problem goes away. Secondly, you do need to activate the venv so that command line tools in ./venv/bin are used (instead of whatever is in your default path). This includes both the correct version of python (if you set it when you setup the virtual environment) and importantly jupyter. From personal experience there's a whole set of pain when you don't realize your jupyter isn't the one inside the virtual environment and so is using other libraries. Because many shells cache the location of executables, if you don't deactivate and reactivate your virtual environment after installing Jupyter into it then it may (in some circumstances) use a globally installed version.