4 ms·

Uv and Ray: Pain-Free Python Dependencies in Clusters

jessekv 1y ago
It would be a fun callback if the demo was a factorial server: https://joearms.github.io/#2013-11-21%20My%20favorite%20Erlang%20Program https://joearms.github.io/#2013-11-21%20My%20favorite%20Erla...
lz400 1y ago
Unfortunately uv is usually insufficient for certain ML deployments in Python. It's a real pain to install pytorch/CUDA with all the necessary drivers and C++ dependencies so people tend to fall back to conda. Any modern tips / life hacks for this situation?
- devjab 1y ago
  https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection https://docs.astral.sh/uv/guides/integration/pytorch/#automa... doesn't work?
  - lz400 1y ago
    the problem is that you still need to install all the low level stuff manually, conda does it automatically
    - gcarvalho 1y ago
      I was pleasantly surprised to try the guide out and see that it just worked: λ uv venv && uv pip install torch --torch-backend=auto λ uv run python -c 'import torch; print(torch.cuda.is_available())' True This is on Debian stable, and I don't remember doing any special setup other than installing the proprietary nvidia driver.
    - pcwelder 1y ago
      This script has been sufficient for me to configure gpu drivers on fresh ubuntu machines. It's just uv add torch after this. https://cloud.google.com/compute/docs/gpus/install-drivers-gpu#install-script https://cloud.google.com/compute/docs/gpus/install-drivers-g... (NOTE: not gcloud specific)
- Kydlaw 1y ago
  You should give a try to https://pixi.sh/latest/ https://pixi.sh/latest/ (I am not involve in the project). They are a little more focus on scientific computing than uv, which is more general. They might be a better option in your case.
- miohtama 1y ago
  Would it be possible to use Docker to manage native dependencies?
- rsfern 1y ago
  Are there particular libraries that make your setup difficult? I just manually set the index and source following the docs (didn’t know about the auto backend feature) and pin a specific version if I really have to with `uv add “torch==2.4”`. This works pretty well for me for projects that use dgl, which heavily uses C++ extensions and can be pretty finicky about working with particular versions This is in a conventional HPC environment, and I’ve found it way better than conda since the dependency solves are so much faster and I no longer experience PyTorch silently getting downgraded to cpu version of I install a new library. Maybe I’ve been using conda poorly though?
chatmasta 1y ago
A whole blog post and nine comments and nobody has made the pun about UV Rays?!

It would be a fun callback if the demo was a factorial server: https://joearms.github.io/#2013-11-21%20My%20favorite%20Erlang%20Program https://joearms.github.io/#2013-11-21%20My%20favorite%20Erla...

lz400 1y ago

Unfortunately uv is usually insufficient for certain ML deployments in Python. It's a real pain to install pytorch/CUDA with all the necessary drivers and C++ dependencies so people tend to fall back to conda. Any modern tips / life hacks for this situation?

devjab 1y ago
https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection https://docs.astral.sh/uv/guides/integration/pytorch/#automa... doesn't work?
- lz400 1y ago
  the problem is that you still need to install all the low level stuff manually, conda does it automatically
  - gcarvalho 1y ago
    I was pleasantly surprised to try the guide out and see that it just worked: λ uv venv && uv pip install torch --torch-backend=auto λ uv run python -c 'import torch; print(torch.cuda.is_available())' True This is on Debian stable, and I don't remember doing any special setup other than installing the proprietary nvidia driver.
  - pcwelder 1y ago
    This script has been sufficient for me to configure gpu drivers on fresh ubuntu machines. It's just uv add torch after this. https://cloud.google.com/compute/docs/gpus/install-drivers-gpu#install-script https://cloud.google.com/compute/docs/gpus/install-drivers-g... (NOTE: not gcloud specific)
Kydlaw 1y ago
You should give a try to https://pixi.sh/latest/ https://pixi.sh/latest/ (I am not involve in the project). They are a little more focus on scientific computing than uv, which is more general. They might be a better option in your case.
miohtama 1y ago
Would it be possible to use Docker to manage native dependencies?
rsfern 1y ago
Are there particular libraries that make your setup difficult? I just manually set the index and source following the docs (didn’t know about the auto backend feature) and pin a specific version if I really have to with `uv add “torch==2.4”`. This works pretty well for me for projects that use dgl, which heavily uses C++ extensions and can be pretty finicky about working with particular versions This is in a conventional HPC environment, and I’ve found it way better than conda since the dependency solves are so much faster and I no longer experience PyTorch silently getting downgraded to cpu version of I install a new library. Maybe I’ve been using conda poorly though?

chatmasta 1y ago

A whole blog post and nine comments and nobody has made the pun about UV Rays?!