3 ms·

> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai

> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a bloody mess, but thankfully you can just run it as a docker container once it's all working...

ptero 2y ago
Not disagreeing, but IME much more often I have to run something that locks specific versions of their python dependencies because ... well ... why not -- kool kids run everything in a virtual environment anyway. Version locking is a powerful tool that should be used very sparingly. My 2c.
greatgib 2y ago
I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your case, you faced the current situation of lot of "data scientist" jumping the band wagon of AI are creating a lot of things like apprentice sorcerers without a real experience of software engineering.
- nubinetwork 2y ago
  > For other languages like Python, there is not a need and you should not do that. Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...
- rcxdude 2y ago
  No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime. It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything). Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.

ptero 2y ago

Not disagreeing, but IME much more often I have to run something that locks specific versions of their python dependencies because ... well ... why not -- kool kids run everything in a virtual environment anyway. Version locking is a powerful tool that should be used very sparingly. My 2c.

greatgib 2y ago

I think that 2 different things are mixed up. For Rust it is language specific, because the thing being statically built, doing differently is very hard. For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six). But in your case, you faced the current situation of lot of "data scientist" jumping the band wagon of AI are creating a lot of things like apprentice sorcerers without a real experience of software engineering.

nubinetwork 2y ago
> For other languages like Python, there is not a need and you should not do that. Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...
rcxdude 2y ago
No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime. It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything). Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.