4 ms·
Hmm I don't think so. Most comments are pretty positive. I think the articles are just not really upvoted unless it's really big news, makes sense because HN i
by wkat4242 3y ago
Hmm I don't think so. Most comments are pretty positive.
I think the articles are just not really upvoted unless it's really big news, makes sense because HN is for more than just AI.
But I don't think it's anti-AI like most people here would be pretty anti-cryptocurrency (and for good reason IMO)
- p1esk 3y agoI didn’t upvote it because I don’t use Ollama. To experiment with LLMs I use Huggingface. Does Ollama provide something I cannot get with Huggingface?
- gertop 3y agoHugging face is a model repository. Ollama allows you to run those models. Different things.
- p1esk 3y agoI run models using HF just fine. I mean I’m using HF transformers repo, which gets models from HF hub. Or do you mean commercial deployment of models for inference?
- simonw 3y agoAre you talking about the Hugging Face Python libraries, the Hugging Face hosted inference APIs, the Hugging Face web interfaces, the Hugging Face iPhone app, Hugging Face Spaces (hosted Docker environments with GPU access) or something else?
- p1esk 3y agoI updated my comment above: I’m using HF transformers repo, which gets models from HF hub.
- simonw 3y agoDo you have an NVIDIA GPU? I have not had much luck with the transformers library on a Mac.
- p1esk 3y agoOf course. I thought Nvidia GPUs are pretty much a must have to play with DL models.
- objektif 3y agoWell being able to run these models on CPU was pretty much the revolutionary part of llama.cpp.
- p1esk 3y agoI can run them on CPU - HF uses plain Pytorch code - fully supported on CPU.
- tmostak 3y agoBut it's likely to be much slower than what you'd get with a backend like llama.cpp on CPU (particularly if you're running on a Mac, but I think on Linux as well), as well as not supporting features like CPU offloading.
- p1esk 3y agoAre there benchmarks? 2x speed up would not be enough for me to return to c++ hell, but 5x might be, in some circumstances.
- SushiHippie 3y agoI think the biggest selling point of ollama (llama.cpp) are quantizations, for a slight hit (with q8 or q4) in quality you can get a significant performance boost.
- lolinder 3y agoOllama provides a web server with API that just works out of the box, which is great when you want to integrate multiple applications (potentially distributed on smaller edge devices) with LLMs that run on a single beefy machine. In my home I have a large gaming rig that sometimes runs Ollama+Open WebUI, then I also have a bunch of other services running on a smaller server and a Raspberry Pi which reach out to Ollama for their LLM inference needs.
- p1esk 3y agoSure, maybe it’s better for niche use cases like yours. HF is the biggest provider of llms, and I guess I haven’t run into it’s limitations yet.
- jkh1 3y agoRunning locally is sometimes necessary, e.g. you don't want to send sensitive data to any random third party server.
- Zambyte 3y agoBoth Ollama and Huggingface distribute models. The latter sites have model hosting services too, but that isn't the only way to use models from there.