3 ms·
Who is reinventing PyPi? Ollama isn't even written in Python. I think that's part of the point, PyPi and Python and Conda is an absolute shitshow of an ecosyst
by dheera 2y ago
Who is reinventing PyPi? Ollama isn't even written in Python.
I think that's part of the point, PyPi and Python and Conda is an absolute shitshow of an ecosystem right now. 3 packages depend on 3 different versions of Pytorch. System installs CUDA 12.3 with the NVIDIA driver but but Xformers wants CUDA 12.1, and the NVIDIA driver that would provide 12.1 doesn't compile on Ubuntu 24.04. Pytorch provides its own CUDA that's different from the system version, and that makes another package unhappy. One package installs opencv, another package overwrites it with opencv-headless, pip install apex doesn't work with gcc-12, ...
- wmedrano 2y agoMain reason I started using ollama is because it actually worked. I spent half an hour trying to get huggingface models to run on my GPU with no success. Meanwhile `sudo pacman -S ollama-rocm` seemed to just work out of the box.
- dheera 2y agoExactly, and as a bonus it runs an JSON API on localhost so you can call it from anything on your LAN, it's really, really amazing