5 ms·

It sounds mean to say it, but it's 100% true. I moved away from using python wherever I can. I've had colleagues struggle for days to install well used packages

It sounds mean to say it, but it's 100% true. I moved away from using python wherever I can. I've had colleagues struggle for days to install well used packages like pandas and numpy in conda.

danielvaughn 4y ago
I just began writing Python a few months ago. For years prior, I'd been a JS dev, and while NPM can be frustrating at times, I never encountered so many issues as I have in Python. It's crazy. I'm now curious whether there are languages out there that do have a really nice packaging system.
- dvlsg 4y ago
  Cargo (Rust) is pretty solid. Most of my minor complaints (like being unable to add packages from the CLI) have been resolved with time, as well.
- Nadya 4y ago
  Personally, zero complaints about Cargo (Rust) and very minimal complaints about NuGet (C#/.NET). My issues around NuGet are probably self-created because I refuse to learn the CLI [0] for it and I've had occasional issues with Visual Studio's UI for managing things. https://learn.microsoft.com/en-us/nuget/reference/nuget-exe-cli-reference https://learn.microsoft.com/en-us/nuget/reference/nuget-exe-...
  - evntdrvn 4y ago
    In a lot of ways, Paket is significantly better than NuGet, if you ever want to try something new :) It uses a lockfile approach like Cargo, has better dependency resolution, etc https://fsprojects.github.io/Paket/index.html https://fsprojects.github.io/Paket/index.html
- cstrahan 4y ago
  FWIW, I find Cargo to be one of the biggest reasons I like Rust so much — maybe even more than anything to do with Rust itself or safe code. I’ll often look for command line tools written in Rust, but not because of Rust fanboyism, but because I know I can just git clone the project and immediately start hacking on a new feature I need or a quick bug fix. In almost every other language I have to jump through one million hoops before I can build and run whatever it is, let alone have a nice developer experience (autocomplete, go to definition, etc).
  - adgjlsfhk1 4y ago
    Yeah, one of Julia's best decisions was taking heavy inspiration from Rust for the package manager. Rust was 100% the first language to get dependency management right.
    - okeuro49 4y ago
      > Rust was 100% the first language to get dependency management right. In my experience, Java, Go, PHP, NodeJS have all got similar package management that works.
      - fiddlerwoaroof 4y ago
        Yeah, npm was the first good package manager. It gets a lot of hate but my experience is that its strategy is the optimal solution for the problem it solves. And, I think a lot of things people complain about (lots of trivial packages, huge dependency trees, etc.) are an effect of solving the packaging problem well: if you make it easy to add dependencies, people will take advantage of that to add lots of dependencies.
      - justinsaccount 4y ago
        See my above comment, there's a reason why all of your examples work: Java package managers tend to install packages written in java Go installs packages written in go, and maybe C using cgo Cargo installs packages written in rust php package managers install packages written in PHP, extra extensions are rare etc People having trouble with python are NOT having trouble with python. They are having trouble because they are trying to use packages that are just python bindings to absolutely massively complex c++ and Fortran libraries. Often people using python don't even have a C compiler installed (let alone a fortran one for the scientific stuff), so pip blows up the first time they try to install a package that hasn't been pre-built for their system+python version.
- cpach 4y ago
  IMHO, Go’s packaging system is very pleasant to use.
- throwawaymaths 4y ago
  Elixir's packaging system is quite good. We went from empty project to working stable diffusion in 2h. 1.75 of those hours was installing CUDA.
- ChancyChance 4y ago
  Same here, I've been using yarn for years, and when I started using venv, I didn't understand why it had to be so complex. Even after reading this article, I still don't see why it is so complex! Yarn/npm has the right idea: dependencies go in the working folder and expect that hierarchy/protocol. Problem solved. The only problem I have with yarn/npm is the problem any package manager has and that is the attrition of dependencies and how to rank their security risk.
- hot_gril 4y ago
  I can't think of any package/dep systems I actually like other than npm. And they're even starting to screw that up with the `import` weirdness instead of the `require` that's been so simple and easy. Rust's system is probably the next best. ObjC/Swift packaging is a flaming disaster in practice, unless it's improved since I jumped that ship. Last time, I remember every single project having to rely on Cocoapods.
  - Nadya 4y ago
    Weirdness? `require` was Node weirdness because Javascript lacked imports/exports at the time. The ES6 syntax is remarkably better, allows imports to be async, and doesn't need to run the code to see what should be assigned to `module.exports` allowing it to be statically analyzed which allows tree-shaking. Node's CJS syntax will only work for Node requiring that you transpile and bundle it for browsers. The ES6 syntax will work for Node and the browser. I see anyone sticking with CJS syntax the same way I see Python devs who continue writing 2.7 code by choice in new projects and not because they are maintaining older projects.
    - hot_gril 4y ago
      Import is weird because there are several different ways to do the same thing, listed at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... and summarized under "there are four forms of import declarations." Four! And I always forget the different ways they work. Sure tree-shaking and browser support are nice, but they didn't have to make the syntax this complicated to achieve that. Not an issue with other languages.
      - Nadya 4y ago
        I'd argue three. Namespace import is really just giving a name to the idea that "you must provide an alias to be used as a namespace if trying to do a wildcard import to avoid naming conflicts" because executing JS in a browser would otherwise not be able to detect naming conflicts - you'd end up overriding values of the same name with the most recent import which is almost never desired behavior. Think about how you might resolve the issue of two different modules exporting a function with the same name otherwise. "Mandatory namespacing" fixes the problem. The "weird one" of the remaining three is side effect imports which isn't all that weird when you realize you're not assigning it to anything. Functionally this is the same as calling a function rather than assigning a function to a value. eg `myFunction = function myfunction() { //stuff }` vs `myFunction()` and when you think about it like that it becomes significantly less weird but also something you rarely want to do - it's mostly used for polyfills. Good to know it exists but you can probably ignore that it does. So now you're left with two: Default and Named. Use Default when you want the entire library - or almost the entire library. Used Named when you want specific pieces of the library. That's all there is too it really. If I want a specific function from a library - there's no reason to import the entire library. For a while you'd mix both Default and Named exports for React due to how transpiling worked - this React blog post explains it well: https://reactjs.org/blog/2020/09/22/introducing-the-new-jsx-transform.html https://reactjs.org/blog/2020/09/22/introducing-the-new-jsx-... but you don't really have a reason to mix the two in modern codebases. Named imports tend to be preferred because Default imports means giving a name to it which can result in inconsistencies across a codebase when many people are working on it. (eg: `import SumTwo from 'sumtwoNumbers.js'` vs `import AddTwo from 'sumTwoNumbers.js'`. A named import `import {SumTwoNumbers} from 'sumtwoNumbers.js'` solves this problem) There's still one final little "gotcha" - there can only be a single Default export. Generally it's an object that contains "everything" but it doesn't need to be and those times are the only edge cases you'll run into though I can't say I ever have encountered this so it is a "theoretical" reason to avoid Default imports but I can't say it's ever been an issue in practice. I guess I avoided a lot of this weirdness by basically only ever using the ES6 syntax and preferring Named imports (and not being stuck in the React ecosystem). The CommonJS got to avoid some of the "weirdness" because it could pretend the Browser doesn't exist (and leave handling it to bundling tools). So I guess I'll capitulate and say it's a little weird but you can basically ignore it and used Named imports as the "One True Style".
- lmm 4y ago
  The JVM (maven) has quietly had everything working really well for decades. You rarely hear much about it because it just works, and what you hear is mostly people hating on it because it wouldn't let them shoot themselves in the foot. Cargo works much the same way AIUI.
zelphirkalt 4y ago
My advice would be to not use Conda or other such "extra tooling". More trouble than benefit. Stick with venv and poetry.
- zzzeek 4y ago
  poetry is "extra tooling"
  - epgui 4y ago
    It's really nice though.
    - 331c8c71 4y ago
      Until it hangs at dependencies resolution step which happened to me recently on a fastapi/sqlachemy project. Had to add deps one by one not to overwhelm it (rollseyes). Also doesn't play nice with publishing to custom pypi destinations (e.g. self-hosted Gitlab) in my experience. I could track down the issue but the code around was clearly a mess so that I gave up on that one.
prds_lost 4y ago
Your colleagues should consider skipping conda and stick to using venv. It will make life much easier. Given pandas/numpy is huge in data science, moving away is not much of an option unless you are working on a personal project or already have a dedicated team comfortable with using a different stack. There is also the Docker option which is great but much more involved.
justinsaccount 4y ago
Things like pandas and numpy are not python packages. Yes, they are packages FOR python, but they are not python. https://hpc.guix.info/blog/2021/09/whats-in-a-package/ https://hpc.guix.info/blog/2021/09/whats-in-a-package/ does a good job of explaining why installing packages like that is a complete shitshow.

It sounds mean to say it, but it's 100% true. I moved away from using python wherever I can. I've had colleagues struggle for days to install well used packages like pandas and numpy in conda.

danielvaughn 4y ago

I just began writing Python a few months ago. For years prior, I'd been a JS dev, and while NPM can be frustrating at times, I never encountered so many issues as I have in Python. It's crazy. I'm now curious whether there are languages out there that do have a really nice packaging system.

dvlsg 4y ago
Cargo (Rust) is pretty solid. Most of my minor complaints (like being unable to add packages from the CLI) have been resolved with time, as well.
Nadya 4y ago
Personally, zero complaints about Cargo (Rust) and very minimal complaints about NuGet (C#/.NET). My issues around NuGet are probably self-created because I refuse to learn the CLI [0] for it and I've had occasional issues with Visual Studio's UI for managing things. https://learn.microsoft.com/en-us/nuget/reference/nuget-exe-cli-reference https://learn.microsoft.com/en-us/nuget/reference/nuget-exe-...
- evntdrvn 4y ago
  In a lot of ways, Paket is significantly better than NuGet, if you ever want to try something new :) It uses a lockfile approach like Cargo, has better dependency resolution, etc https://fsprojects.github.io/Paket/index.html https://fsprojects.github.io/Paket/index.html
cstrahan 4y ago
FWIW, I find Cargo to be one of the biggest reasons I like Rust so much — maybe even more than anything to do with Rust itself or safe code. I’ll often look for command line tools written in Rust, but not because of Rust fanboyism, but because I know I can just git clone the project and immediately start hacking on a new feature I need or a quick bug fix. In almost every other language I have to jump through one million hoops before I can build and run whatever it is, let alone have a nice developer experience (autocomplete, go to definition, etc).
- adgjlsfhk1 4y ago
  Yeah, one of Julia's best decisions was taking heavy inspiration from Rust for the package manager. Rust was 100% the first language to get dependency management right.
  - okeuro49 4y ago
    > Rust was 100% the first language to get dependency management right. In my experience, Java, Go, PHP, NodeJS have all got similar package management that works.
    - fiddlerwoaroof 4y ago
      Yeah, npm was the first good package manager. It gets a lot of hate but my experience is that its strategy is the optimal solution for the problem it solves. And, I think a lot of things people complain about (lots of trivial packages, huge dependency trees, etc.) are an effect of solving the packaging problem well: if you make it easy to add dependencies, people will take advantage of that to add lots of dependencies.
    - justinsaccount 4y ago
      See my above comment, there's a reason why all of your examples work: Java package managers tend to install packages written in java Go installs packages written in go, and maybe C using cgo Cargo installs packages written in rust php package managers install packages written in PHP, extra extensions are rare etc People having trouble with python are NOT having trouble with python. They are having trouble because they are trying to use packages that are just python bindings to absolutely massively complex c++ and Fortran libraries. Often people using python don't even have a C compiler installed (let alone a fortran one for the scientific stuff), so pip blows up the first time they try to install a package that hasn't been pre-built for their system+python version.
cpach 4y ago
IMHO, Go’s packaging system is very pleasant to use.
throwawaymaths 4y ago
Elixir's packaging system is quite good. We went from empty project to working stable diffusion in 2h. 1.75 of those hours was installing CUDA.
ChancyChance 4y ago
Same here, I've been using yarn for years, and when I started using venv, I didn't understand why it had to be so complex. Even after reading this article, I still don't see why it is so complex! Yarn/npm has the right idea: dependencies go in the working folder and expect that hierarchy/protocol. Problem solved. The only problem I have with yarn/npm is the problem any package manager has and that is the attrition of dependencies and how to rank their security risk.
hot_gril 4y ago
I can't think of any package/dep systems I actually like other than npm. And they're even starting to screw that up with the `import` weirdness instead of the `require` that's been so simple and easy. Rust's system is probably the next best. ObjC/Swift packaging is a flaming disaster in practice, unless it's improved since I jumped that ship. Last time, I remember every single project having to rely on Cocoapods.
- Nadya 4y ago
  Weirdness? `require` was Node weirdness because Javascript lacked imports/exports at the time. The ES6 syntax is remarkably better, allows imports to be async, and doesn't need to run the code to see what should be assigned to `module.exports` allowing it to be statically analyzed which allows tree-shaking. Node's CJS syntax will only work for Node requiring that you transpile and bundle it for browsers. The ES6 syntax will work for Node and the browser. I see anyone sticking with CJS syntax the same way I see Python devs who continue writing 2.7 code by choice in new projects and not because they are maintaining older projects.
  - hot_gril 4y ago
    Import is weird because there are several different ways to do the same thing, listed at https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... and summarized under "there are four forms of import declarations." Four! And I always forget the different ways they work. Sure tree-shaking and browser support are nice, but they didn't have to make the syntax this complicated to achieve that. Not an issue with other languages.
    - Nadya 4y ago
      I'd argue three. Namespace import is really just giving a name to the idea that "you must provide an alias to be used as a namespace if trying to do a wildcard import to avoid naming conflicts" because executing JS in a browser would otherwise not be able to detect naming conflicts - you'd end up overriding values of the same name with the most recent import which is almost never desired behavior. Think about how you might resolve the issue of two different modules exporting a function with the same name otherwise. "Mandatory namespacing" fixes the problem. The "weird one" of the remaining three is side effect imports which isn't all that weird when you realize you're not assigning it to anything. Functionally this is the same as calling a function rather than assigning a function to a value. eg `myFunction = function myfunction() { //stuff }` vs `myFunction()` and when you think about it like that it becomes significantly less weird but also something you rarely want to do - it's mostly used for polyfills. Good to know it exists but you can probably ignore that it does. So now you're left with two: Default and Named. Use Default when you want the entire library - or almost the entire library. Used Named when you want specific pieces of the library. That's all there is too it really. If I want a specific function from a library - there's no reason to import the entire library. For a while you'd mix both Default and Named exports for React due to how transpiling worked - this React blog post explains it well: https://reactjs.org/blog/2020/09/22/introducing-the-new-jsx-transform.html https://reactjs.org/blog/2020/09/22/introducing-the-new-jsx-... but you don't really have a reason to mix the two in modern codebases. Named imports tend to be preferred because Default imports means giving a name to it which can result in inconsistencies across a codebase when many people are working on it. (eg: `import SumTwo from 'sumtwoNumbers.js'` vs `import AddTwo from 'sumTwoNumbers.js'`. A named import `import {SumTwoNumbers} from 'sumtwoNumbers.js'` solves this problem) There's still one final little "gotcha" - there can only be a single Default export. Generally it's an object that contains "everything" but it doesn't need to be and those times are the only edge cases you'll run into though I can't say I ever have encountered this so it is a "theoretical" reason to avoid Default imports but I can't say it's ever been an issue in practice. I guess I avoided a lot of this weirdness by basically only ever using the ES6 syntax and preferring Named imports (and not being stuck in the React ecosystem). The CommonJS got to avoid some of the "weirdness" because it could pretend the Browser doesn't exist (and leave handling it to bundling tools). So I guess I'll capitulate and say it's a little weird but you can basically ignore it and used Named imports as the "One True Style".
lmm 4y ago
The JVM (maven) has quietly had everything working really well for decades. You rarely hear much about it because it just works, and what you hear is mostly people hating on it because it wouldn't let them shoot themselves in the foot. Cargo works much the same way AIUI.

zelphirkalt 4y ago

My advice would be to not use Conda or other such "extra tooling". More trouble than benefit. Stick with venv and poetry.

zzzeek 4y ago
poetry is "extra tooling"
- epgui 4y ago
  It's really nice though.
  - 331c8c71 4y ago
    Until it hangs at dependencies resolution step which happened to me recently on a fastapi/sqlachemy project. Had to add deps one by one not to overwhelm it (rollseyes). Also doesn't play nice with publishing to custom pypi destinations (e.g. self-hosted Gitlab) in my experience. I could track down the issue but the code around was clearly a mess so that I gave up on that one.

prds_lost 4y ago

Your colleagues should consider skipping conda and stick to using venv. It will make life much easier. Given pandas/numpy is huge in data science, moving away is not much of an option unless you are working on a personal project or already have a dedicated team comfortable with using a different stack. There is also the Docker option which is great but much more involved.

justinsaccount 4y ago

Things like pandas and numpy are not python packages. Yes, they are packages FOR python, but they are not python. https://hpc.guix.info/blog/2021/09/whats-in-a-package/ https://hpc.guix.info/blog/2021/09/whats-in-a-package/ does a good job of explaining why installing packages like that is a complete shitshow.