4 ms·
Ollama is really easy. brew install ollama brew services start ollama ollama pull mistral Ollama you can query via http. It provides a consistent interface
by aantix 3y ago
Ollama is really easy.
brew install ollama
brew services start ollama
ollama pull mistral
Ollama you can query via http. It provides a consistent interface for prompting, regardless of model.
https://github.com/ollama/ollama/blob/main/docs/api.md#request https://github.com/ollama/ollama/blob/main/docs/api.md#reque...
- crazypython 3y agoIt lacks batching support (n>1), unfortunately, which is necessary for Loom-like applications
- aantix 3y agoLoom - this application? https://medium.com/@tofujoy77/loom-ai-uncovering-creative-writing-possibilities-in-space-7cd351e0880f https://medium.com/@tofujoy77/loom-ai-uncovering-creative-wr...
- brushfoot 3y agoFor Windows users without brew, there's a Windows installer: https://ollama.com/download/windows https://ollama.com/download/windows WinGet and Scoop apparently also have it. Chocolatey doesn't seem to.
- ActorNightly 3y agoWorks very well in WSL2 as well. I prefer that as you have to start the server manually, so it doesn't sit in the background.
- brushfoot 3y agoHow does Ollama distribute these models? The downloads page on the Llama website, https://llama.meta.com/llama-downloads/ https://llama.meta.com/llama-downloads/, has a heading that reads: > Request access to Llama Which to me gives the impression that access is gated and by application only. But Ollama downloaded it without so much as a "y". Is that just Meta's website UI? Registration isn't actually required?
- cymor 3y agoIt doesn't distribute. The app pulls from hugging face.
- brushfoot 3y agoHugging Face's Llama 2 page says, "You need to share contact information with Meta to access this model": https://huggingface.co/meta-llama/Llama-2-7b https://huggingface.co/meta-llama/Llama-2-7b Ollama's installer didn't ask me for any contact info.
- michaelt 3y agoHuggingface has many, many fine tunes of Llama from different people, which don't ask that. The requirement has always been more of a fig leaf than anything else. It allows Facebook to say everyone who downloaded from them agreed to some legal/ethics nonsense. Like all click-through license agreements nobody reads it; it's there so Facebook can pretend to be shocked when misuse of the model comes to light.
- brushfoot 3y agoI mean Llama itself, not a fine tune. Ollama seemed to download and run Llama 2 with no agreement.
- whimsicalism 3y agobecause this field is a baby, noncompliance is rampant
- eatonphil 3y agoThis repo doesn't seem to say anything about what it does with anything you pass it. There's no privacy policy or anything? It being open source doesn't necessarily mean it isn't passing all my data somewhere. I didn't see anything about this in the repo that everything definitely stays local.
- okwhateverdude 3y agoI run the docker container locally. As far as I can tell, it doesn't call home or anything (from reading the source and from opensnitch). It is just a cgo wrapped llama.cpp that provides an HTTP API. It CAN fetch models from their library, but you can just as easily load in your own GGUF formatted llama models. They implement a docker-like layers mechanism for model configuration that is pretty useful.
- aantix 3y agoIt is free.. So why not just audit the source personally to put your mind at ease and build locally if that’s a major concern of yours?
- aantix 3y agoAnd since you did the audit yourself, you could help out the maintainers by adding a privacy policy.
- jhugo 3y agoA privacy policy isn't typical for code that you build and run yourself on your own machine; it's something that services publish to explain what they will do with the data you send them.
- joshspankit 3y agoThis. Some will do it anyway for PR/marketing, but if the creator does not interact with, have access to, or collect your data they have no obligation to have a privacy policy.
- m463 3y agoI associate brew with macos (where a 3090 would not venture) But it seems like there's a linux brew.