4 ms·

Why is there so much investment in ecosystems outside of Hugging Face? And why do they keep reinventing PyPi?

cosmez 2y ago
I don't think diversity in a software ecosystem is a bad idea. Nothing is stopping Hugging Face from changing their terms and screwing us over
nickpsecurity 2y ago
Google is great. Their slogan is “Don’t Be Evil.” We don’t need to invest in anything else. (Time passes.) “This ad-heavy, search engine is horrible. We need more solid competition!” Supplier diversity prevents or reduces situations like the above.
ms-menardi 2y ago
Huggingface isn't local, while Ollama is.
- Carrok 2y ago
  This is more important than people seem to realize. Local means almost no data security issues. Not local means, well, all the data security issues.
- Der_Einzige 2y ago
  What? That's not true at all. Citation needed!
  - neals 2y ago
    "Huggingface isn't local, while Ollama is." - ms-menardi
    - Der_Einzige 2y ago
      It's just wrong and a lie. Huggingface is local.
      - jhbadger 2y ago
        I think you are arguing at cross-purposes. If you use HuggingFace to simply download models to run locally with ollama or llama.cpp, then you can say it is "local". But you can also use it as a service to run models (which is how they make money). Then they obviously aren't local.
        Der_Einzige 2y ago
        Huggingface let's you run locally with huggingface exclusive code! They started this way and the managed/hosted solution came later. You downvoters are the folks who jumped into this craze post ChatGPT.
        doctorpangloss 2y ago
        I have no idea where people got the idea that Hugging Face isn’t local. I mean they show you how to run everything locally, with all the quantization strategies you could want, with far fewer bugs.
dheera 2y ago
Who is reinventing PyPi? Ollama isn't even written in Python. I think that's part of the point, PyPi and Python and Conda is an absolute shitshow of an ecosystem right now. 3 packages depend on 3 different versions of Pytorch. System installs CUDA 12.3 with the NVIDIA driver but but Xformers wants CUDA 12.1, and the NVIDIA driver that would provide 12.1 doesn't compile on Ubuntu 24.04. Pytorch provides its own CUDA that's different from the system version, and that makes another package unhappy. One package installs opencv, another package overwrites it with opencv-headless, pip install apex doesn't work with gcc-12, ...
- wmedrano 2y ago
  Main reason I started using ollama is because it actually worked. I spent half an hour trying to get huggingface models to run on my GPU with no success. Meanwhile `sudo pacman -S ollama-rocm` seemed to just work out of the box.
  - dheera 2y ago
    Exactly, and as a bonus it runs an JSON API on localhost so you can call it from anything on your LAN, it's really, really amazing
lolinder 2y ago
Personally I love that ollama gives me a really simple API that I can code against in any language and on any platform that I care to. I can `docker run ollama` on my gaming rig and then hit it up from a Raspberry Pi running anything, Python or not.

I don't think diversity in a software ecosystem is a bad idea. Nothing is stopping Hugging Face from changing their terms and screwing us over

nickpsecurity 2y ago

Google is great. Their slogan is “Don’t Be Evil.” We don’t need to invest in anything else. (Time passes.) “This ad-heavy, search engine is horrible. We need more solid competition!” Supplier diversity prevents or reduces situations like the above.

ms-menardi 2y ago

Huggingface isn't local, while Ollama is.

Carrok 2y ago
This is more important than people seem to realize. Local means almost no data security issues. Not local means, well, all the data security issues.
Der_Einzige 2y ago
What? That's not true at all. Citation needed!
- neals 2y ago
  "Huggingface isn't local, while Ollama is." - ms-menardi
  - Der_Einzige 2y ago
    It's just wrong and a lie. Huggingface is local.
    - jhbadger 2y ago
      I think you are arguing at cross-purposes. If you use HuggingFace to simply download models to run locally with ollama or llama.cpp, then you can say it is "local". But you can also use it as a service to run models (which is how they make money). Then they obviously aren't local.
      - Der_Einzige 2y ago
        Huggingface let's you run locally with huggingface exclusive code! They started this way and the managed/hosted solution came later. You downvoters are the folks who jumped into this craze post ChatGPT.
        doctorpangloss 2y ago
        I have no idea where people got the idea that Hugging Face isn’t local. I mean they show you how to run everything locally, with all the quantization strategies you could want, with far fewer bugs.

dheera 2y ago

Who is reinventing PyPi? Ollama isn't even written in Python. I think that's part of the point, PyPi and Python and Conda is an absolute shitshow of an ecosystem right now. 3 packages depend on 3 different versions of Pytorch. System installs CUDA 12.3 with the NVIDIA driver but but Xformers wants CUDA 12.1, and the NVIDIA driver that would provide 12.1 doesn't compile on Ubuntu 24.04. Pytorch provides its own CUDA that's different from the system version, and that makes another package unhappy. One package installs opencv, another package overwrites it with opencv-headless, pip install apex doesn't work with gcc-12, ...

wmedrano 2y ago
Main reason I started using ollama is because it actually worked. I spent half an hour trying to get huggingface models to run on my GPU with no success. Meanwhile `sudo pacman -S ollama-rocm` seemed to just work out of the box.
- dheera 2y ago
  Exactly, and as a bonus it runs an JSON API on localhost so you can call it from anything on your LAN, it's really, really amazing

lolinder 2y ago

Personally I love that ollama gives me a really simple API that I can code against in any language and on any platform that I care to. I can `docker run ollama` on my gaming rig and then hit it up from a Raspberry Pi running anything, Python or not.