9 ms·
That works but doesn’t solve the problem mentioned by the other comment. We also have to support a matrix of Python runtimes and dependencies in all of our mach
by aprdm 6y ago
That works but doesn’t solve the problem mentioned by the other comment. We also have to support a matrix of Python runtimes and dependencies in all of our machines. We can’t pre-bake the environments. It’s a very unique problem for HPC I guess. We don’t use pip, have a look in the Rez package manager if you haven’t yet for an idea of the problem.
For our webservices it is much less of a problem as we can isolate the runtime from software installed in machines or third party software dependencies that we integrate with (think a blender plugin, blender comes with its own versions of dependencies)
- rbanffy 6y ago> We don’t use pip What's the problem?
- aprdm 6y agoHave you read about the problems Rez solves ? Our environments are too dynamic and need to always be available. Our hpc cluster runs millions of different combinations of software runtimes at any given day...
- rbanffy 6y agoMy understanding is that it operates on a different level than Pip.
- aprdm 6y agoHm, not really. It has build scripts which you do whatever you want to build your package and put somewhere and then it adjusts PYTHONPATH dynamically at runtime. You don't need Pip if you're using rez. You also don't use a setup.py, you use a package.py instead.
- cozzyd 6y agoRez looks really great! Thanks for sharing.