7 ms·

Are these libraries for connecting to an ollama service that the user has already installed or do they work without the user installing anything? Sorry for not

by rgbrgb 3y ago

Are these libraries for connecting to an ollama service that the user has already installed or do they work without the user installing anything? Sorry for not checking the code but maybe someone has the same question here. I looked at using ollama when I started making FreeChat [0] but couldn't figure out a way to make it work without asking the user to install it first (think I asked in your discord at the time). I wanted FreeChat to be 1-click install from the mac app store so I ended up bundling the llama.cpp server instead which it runs on localhost for inference. At some point I'd love to swap it out for ollama and take advantage of all the cool model pulling stuff you guys have done, I just need it to be embeddable. My ideal setup would be importing an ollama package in swift which would start the server if the user doesn't already have it running. I know this is just js and python to start but a dev can dream :) Either way, congrats on the release! [0]: https://github.com/psugihara/FreeChat https://github.com/psugihara/FreeChat

Abishek_Muthian 3y ago
I used Ollama docker image to integrate with Gait Analyzer[1], a self-hosted gait analysis tool; all I had to do was to set up the docker compose file, I was able to get the setup done with a single script for the end user and I used langchain to interact with Ollama. [1] https://github.com/abishekmuthian/gaitanalyzer https://github.com/abishekmuthian/gaitanalyzer
icyfox 3y ago
Just for connecting to an existing service: https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L109 https://github.com/ollama/ollama-python/blob/main/ollama/_cl...
- thrdbndndn 3y ago
  For the client API it's pretty clear: from ollama import Client client = Client(host='http://localhost:11434') But I don't quite get how the example in "Usage" can work: import ollama response = ollama.chat(model='llama2', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ]) print(response['message']['content']) Since there is no parameter for host and/or port.
  - jmorgan 3y ago
    Once you have a custom `client` you can use it in place of `ollama`. For example: client = Client(host='http://my.ollama.host:11434') response = client.chat(model='llama2', messages=[...])
    - thrdbndndn 3y ago
      Thanks. I don't have the service installed on my computer RN, but I assume the former works because it by default uses a host (localhost) and port number that is also the default for ollma service?
      - flakes 3y ago
        Exactly that. Client host options default, https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L610-L657 https://github.com/ollama/ollama-python/blob/main/ollama/_cl... Also overrideable with OLLAMA_HOST env var. The default imported functions are then based off of a no-arg constructed client https://github.com/ollama/ollama-python/blob/main/ollama/__init__.py https://github.com/ollama/ollama-python/blob/main/ollama/__i... # ollama-python/ollama/__init__.py _client = Client() generate = _client.generate chat = _client.chat embeddings = _client.embeddings ...
SnowLprd 3y ago
On the subject of installing Ollama, I found it to be a frustrating and user-hostile experience. I instead recommend the much more user-friendly LLM[0] by Simon Willison. Among the problems with Ollama include: * Ollama silently adds a login item with no way to opt out: <https://github.com/jmorganca/ollama/issues/162 https://github.com/jmorganca/ollama/issues/162> * Ollama spawns at least four processes, some persistently in the background: 1 x Ollama application, 1 x `ollama` server component, 2 x Ollama Helper * Ollama provides no information at install time about what directories will be created or where models will be downloaded. * Ollama prompts users to install the `ollama` CLI tool, with admin access required, with no way to cancel, and with no way to even quit the application at that point. Ollama provides no clarity that about what is actually happening during this step: all it is doing is symlinking `/Applications/Ollama.app/Contents/Resources/ollama` to `/usr/local/bin/` The worst part is that not only is none of this explained at install time, but the project README doesn’t tell you any of this information either. Potential users deserve to know what will happen on first launch, but when a PR arrived to at least provide that clarification in the README, Ollama maintainers summarily closed that PR and still have not rectified the aforementioned UX problems. As an open source maintainer myself, I understand and appreciate that Ollama developers volunteer their time and energy into the project, and they can run it as they see fit. So I intend no disrespect. But these problems, and a seeming unwillingness to prioritize their resolution, caused me to delete Ollama from my system entirely. As I said above, I think LLM[0] by Simon Willison is an excellent and user-friendly alternative. [0]: https://llm.datasette.io/ https://llm.datasette.io/
- siquick 3y ago
  "User hostile experience" is complete hyperbole and disrespectful to the efforts of the maintainers of this excellent library.
  - config_yml 3y ago
    Indeed, I thought the user experience was great. Simple way to download, install and start: everything just worked.
  - SnowLprd 3y ago

by rgbrgb 3y ago

Abishek_Muthian 3y ago

I used Ollama docker image to integrate with Gait Analyzer[1], a self-hosted gait analysis tool; all I had to do was to set up the docker compose file, I was able to get the setup done with a single script for the end user and I used langchain to interact with Ollama. [1] https://github.com/abishekmuthian/gaitanalyzer https://github.com/abishekmuthian/gaitanalyzer

icyfox 3y ago

Just for connecting to an existing service: https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L109 https://github.com/ollama/ollama-python/blob/main/ollama/_cl...

thrdbndndn 3y ago
For the client API it's pretty clear: from ollama import Client client = Client(host='http://localhost:11434') But I don't quite get how the example in "Usage" can work: import ollama response = ollama.chat(model='llama2', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ]) print(response['message']['content']) Since there is no parameter for host and/or port.
- jmorgan 3y ago
  Once you have a custom `client` you can use it in place of `ollama`. For example: client = Client(host='http://my.ollama.host:11434') response = client.chat(model='llama2', messages=[...])
  - thrdbndndn 3y ago
    Thanks. I don't have the service installed on my computer RN, but I assume the former works because it by default uses a host (localhost) and port number that is also the default for ollma service?
    - flakes 3y ago
      Exactly that. Client host options default, https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L610-L657 https://github.com/ollama/ollama-python/blob/main/ollama/_cl... Also overrideable with OLLAMA_HOST env var. The default imported functions are then based off of a no-arg constructed client https://github.com/ollama/ollama-python/blob/main/ollama/__init__.py https://github.com/ollama/ollama-python/blob/main/ollama/__i... # ollama-python/ollama/__init__.py _client = Client() generate = _client.generate chat = _client.chat embeddings = _client.embeddings ...

SnowLprd 3y ago

On the subject of installing Ollama, I found it to be a frustrating and user-hostile experience. I instead recommend the much more user-friendly LLM[0] by Simon Willison. Among the problems with Ollama include: * Ollama silently adds a login item with no way to opt out: <https://github.com/jmorganca/ollama/issues/162 https://github.com/jmorganca/ollama/issues/162> * Ollama spawns at least four processes, some persistently in the background: 1 x Ollama application, 1 x `ollama` server component, 2 x Ollama Helper * Ollama provides no information at install time about what directories will be created or where models will be downloaded. * Ollama prompts users to install the `ollama` CLI tool, with admin access required, with no way to cancel, and with no way to even quit the application at that point. Ollama provides no clarity that about what is actually happening during this step: all it is doing is symlinking `/Applications/Ollama.app/Contents/Resources/ollama` to `/usr/local/bin/` The worst part is that not only is none of this explained at install time, but the project README doesn’t tell you any of this information either. Potential users deserve to know what will happen on first launch, but when a PR arrived to at least provide that clarification in the README, Ollama maintainers summarily closed that PR and still have not rectified the aforementioned UX problems. As an open source maintainer myself, I understand and appreciate that Ollama developers volunteer their time and energy into the project, and they can run it as they see fit. So I intend no disrespect. But these problems, and a seeming unwillingness to prioritize their resolution, caused me to delete Ollama from my system entirely. As I said above, I think LLM[0] by Simon Willison is an excellent and user-friendly alternative. [0]: https://llm.datasette.io/ https://llm.datasette.io/

siquick 3y ago
"User hostile experience" is complete hyperbole and disrespectful to the efforts of the maintainers of this excellent library.
- config_yml 3y ago
  Indeed, I thought the user experience was great. Simple way to download, install and start: everything just worked.
- SnowLprd 3y ago