3 ms·

Just for connecting to an existing service: https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L109 https://github.com/ollama/ollama-python/blo

• thrdbndndn 3y ago
  For the client API it's pretty clear: from ollama import Client client = Client(host='http://localhost:11434') But I don't quite get how the example in "Usage" can work: import ollama response = ollama.chat(model='llama2', messages=[ { 'role': 'user', 'content': 'Why is the sky blue?', }, ]) print(response['message']['content']) Since there is no parameter for host and/or port.
  • jmorgan 3y ago
    Once you have a custom `client` you can use it in place of `ollama`. For example: client = Client(host='http://my.ollama.host:11434') response = client.chat(model='llama2', messages=[...])
    • thrdbndndn 3y ago
      Thanks. I don't have the service installed on my computer RN, but I assume the former works because it by default uses a host (localhost) and port number that is also the default for ollma service?
      • flakes 3y ago
        Exactly that. Client host options default, https://github.com/ollama/ollama-python/blob/main/ollama/_client.py#L610-L657 https://github.com/ollama/ollama-python/blob/main/ollama/_cl... Also overrideable with OLLAMA_HOST env var. The default imported functions are then based off of a no-arg constructed client https://github.com/ollama/ollama-python/blob/main/ollama/__init__.py https://github.com/ollama/ollama-python/blob/main/ollama/__i... # ollama-python/ollama/__init__.py _client = Client() generate = _client.generate chat = _client.chat embeddings = _client.embeddings ...