7 ms·

I am somewhat surprised that this app doesn't seem to offer any way to connect to a remote Ollama instance. The most powerful computer I own isn't necessarily t

I am somewhat surprised that this app doesn't seem to offer any way to connect to a remote Ollama instance. The most powerful computer I own isn't necessarily the one I'm running the GUI on.

jay_kyburz 1y ago
The app does have a function to expose Ollama to the network, so perhaps its coming?
- deleted 1y ago
  [deleted]
- hodgehog11 1y ago
  It's definitely coming, there is no way they would leave such an important feature on the table. My guess is they are waiting so they can announce connections to their own servers.
jart 1y ago
This. This. A thousand times this. I hate Windows / MacOS but love their desktops. I love Linux / BSD but hate their desktops. So my most expensive most powerful workstation is always a headless Linux machine that I ssh into from a Windows or MacOS toy computer. Unfortunately most developers do not understand this. Every time I run a command in the terminal and it tries to open a browser tab without printing the URL, it makes me want to scream and shout and retire from tech forever to be a plumber.
- tux1968 1y ago
  You can replace the xdg-open command (or whichever command is used on your linux system) with your own. Just program it to fire over the url to a waiting socket on your windows box, and have it automatically open there. The details are pretty easy to work out, and the result will be seamless.
  - mbreese 1y ago
    I usually do this with a port forward (ip or Unix socket) over SSH. This way my server just sends data to ~/.tunnel/socket, and my SSH connection handles getting it to my client. (It’s a bit more complicated with starting a listening server in my laptop, making sure the port forwarded file doesn’t exist, etc, but this is the basic idea.)
- hyperbolablabla 1y ago
  Hey Justine! Thank you for all your fantastic work
  - jart 1y ago
    Thanks!
- amelius 1y ago
  I can recommend to spend a day finding and configuring a window manager that suits your needs.
  - graemep 1y ago
    This was my thought. Given the huge range of desktop environments and window managers available there has to be one that suits you. Probably one that suits you pretty much out of the box.
  - Eisenstein 1y ago
    I doubt that she spent the time to create a cross platform C compiler library but didn't bother trying out a few Linux desktops.
    - amelius 1y ago
      I doubt that there is no wm that suits their needs.
  - VladCuciureanu 1y ago
    Or just display the URL in terminal. I spent 5 years of my life ricing my Linux machine to get it as I want it to be only to realise that, at least for my needs and likes, nothing matches MacOS’s DE, compositor and font rendering. Not a bash on Linux desktop users, just my experience.
  - deleted 1y ago
    [deleted]
- deleted 1y ago
  [deleted]
ethan_smith 1y ago
You can work around this by using SSH port forwarding (ssh -L 11434:localhost:11434 user@remote) to connect to a remote Ollama instance, though native support would definitely be better.
- stavros 1y ago
  Wait, it already connects over the network, it just doesn't let you specify the hostname? That's really surprising to me.
mchiang 1y ago
This is a feature we are looking into supporting. Thank you for reaffirming the need.
- stavros 1y ago
  But it seems like the GUI already connects over the network, no? In that case, why do you need to do user research for adding what is basically a command line option, at its simplest? It would probably take less time to add that than to write the comment.
  - silentguy 1y ago
    They will have to support auth if they are adding support for connecting with remote host. It's not difficult but it's not as trivial as you suggested.
    - stavros 1y ago
      Ollama server already supports authentication, and URLs already have a place for credentials.
ddlsmurf 1y ago
I use ollama via the llm plugin: https://github.com/taketwo/llm-ollama?tab=readme-ov-file#ollama-server-address https://github.com/taketwo/llm-ollama?tab=readme-ov-file#oll...

I am somewhat surprised that this app doesn't seem to offer any way to connect to a remote Ollama instance. The most powerful computer I own isn't necessarily the one I'm running the GUI on.

jay_kyburz 1y ago

The app does have a function to expose Ollama to the network, so perhaps its coming?

deleted 1y ago
[deleted]
hodgehog11 1y ago
It's definitely coming, there is no way they would leave such an important feature on the table. My guess is they are waiting so they can announce connections to their own servers.

jart 1y ago

This. This. A thousand times this. I hate Windows / MacOS but love their desktops. I love Linux / BSD but hate their desktops. So my most expensive most powerful workstation is always a headless Linux machine that I ssh into from a Windows or MacOS toy computer. Unfortunately most developers do not understand this. Every time I run a command in the terminal and it tries to open a browser tab without printing the URL, it makes me want to scream and shout and retire from tech forever to be a plumber.

tux1968 1y ago
You can replace the xdg-open command (or whichever command is used on your linux system) with your own. Just program it to fire over the url to a waiting socket on your windows box, and have it automatically open there. The details are pretty easy to work out, and the result will be seamless.
- mbreese 1y ago
  I usually do this with a port forward (ip or Unix socket) over SSH. This way my server just sends data to ~/.tunnel/socket, and my SSH connection handles getting it to my client. (It’s a bit more complicated with starting a listening server in my laptop, making sure the port forwarded file doesn’t exist, etc, but this is the basic idea.)
hyperbolablabla 1y ago
Hey Justine! Thank you for all your fantastic work
- jart 1y ago
  Thanks!
amelius 1y ago
I can recommend to spend a day finding and configuring a window manager that suits your needs.
- graemep 1y ago
  This was my thought. Given the huge range of desktop environments and window managers available there has to be one that suits you. Probably one that suits you pretty much out of the box.
- Eisenstein 1y ago
  I doubt that she spent the time to create a cross platform C compiler library but didn't bother trying out a few Linux desktops.
  - amelius 1y ago
    I doubt that there is no wm that suits their needs.
- VladCuciureanu 1y ago
  Or just display the URL in terminal. I spent 5 years of my life ricing my Linux machine to get it as I want it to be only to realise that, at least for my needs and likes, nothing matches MacOS’s DE, compositor and font rendering. Not a bash on Linux desktop users, just my experience.
- deleted 1y ago
  [deleted]
deleted 1y ago
[deleted]

ethan_smith 1y ago

You can work around this by using SSH port forwarding (ssh -L 11434:localhost:11434 user@remote) to connect to a remote Ollama instance, though native support would definitely be better.

stavros 1y ago
Wait, it already connects over the network, it just doesn't let you specify the hostname? That's really surprising to me.

mchiang 1y ago

This is a feature we are looking into supporting. Thank you for reaffirming the need.

stavros 1y ago
But it seems like the GUI already connects over the network, no? In that case, why do you need to do user research for adding what is basically a command line option, at its simplest? It would probably take less time to add that than to write the comment.
- silentguy 1y ago
  They will have to support auth if they are adding support for connecting with remote host. It's not difficult but it's not as trivial as you suggested.
  - stavros 1y ago
    Ollama server already supports authentication, and URLs already have a place for credentials.

ddlsmurf 1y ago

I use ollama via the llm plugin: https://github.com/taketwo/llm-ollama?tab=readme-ov-file#ollama-server-address https://github.com/taketwo/llm-ollama?tab=readme-ov-file#oll...