4 ms·

Are there ways to run such apps with a remote GPU over network? I want to run the UI on my laptop, but use my homeserver GPU from the local network. Anything b

Are there ways to run such apps with a remote GPU over network? I want to run the UI on my laptop, but use my homeserver GPU from the local network. Anything better than X forwarding?

freeone3000 3y ago
This project returns results over HTTP. You get what you want by running the server binary on the server, then access it with your web browser.
- captn3m0 3y ago
  Ooh, perfect. Now just have to wait for a Linux/docker release.
  - heyitsguay 3y ago
    It was easy for me to set up on Ubuntu with CUDA 11.7. I cloned the repo, created a new python venv, and installed the requirements in requirements_versions.txt, but looking at launch.py, that wasn't even necessary. Just create a new venv for the project, activate it, and run `python launch.py`.
- boneitis 3y ago
  To flesh out this response with a common way to do this, the free tier of ngrok will likely help someone accomplish this, by running it on the server to get a routable endpoint to your ssh port (wouldn't hurt to further lock it down with any firewalling, if that's available ngrok-side.. haven't used ngrok in a while). For example, to run the Stable Diffusion webui (which defaults to port 7860), it would be something not too far off from: $ ssh -p1234 -L7860:127.0.0.1:7860 assigned.dns.at.startup.ngrok.io (IIRC ngrok free-tier will allocate you a random port on the public side every time you start the service) Then, can just browse localhost:7860 from the remote field machine. I got bit by the AI bug a few days ago, and I already rent some lightsail instances, one of which has a static IP reserved and one of my domain names pointing to it. So, I set up something a little more convoluted and perhaps unnecessarily complex, turning that lightsail VM into a jumpbox/bastion host. Anywho: AutoSSH from server to my lightsail instance, with two remote port forwards (not local port forwards): SSH and SD webui. Then, connect from the field machine anywhere in the world to the jumpbox with matching local port forwards. (I set up both ports, so I can shell back into the original machine, but this isn't strictly required.) Then, fire up localhost:7860 in the web browser. Make sure this isn't being served on 0.0.0.0 or un-firewalled. (e: I re-read original question after posting and realized GP was merely asking for over the LAN, but hey, now they know how to do this over the 'net :)
  - microbass 3y ago
    Even easier is to use Tailscale. Run tailscale on your laptop and server, and then access the web UI by using <tailnet_server_ip>:<port>

Are there ways to run such apps with a remote GPU over network? I want to run the UI on my laptop, but use my homeserver GPU from the local network. Anything better than X forwarding?

freeone3000 3y ago

This project returns results over HTTP. You get what you want by running the server binary on the server, then access it with your web browser.

captn3m0 3y ago
Ooh, perfect. Now just have to wait for a Linux/docker release.
- heyitsguay 3y ago
  It was easy for me to set up on Ubuntu with CUDA 11.7. I cloned the repo, created a new python venv, and installed the requirements in requirements_versions.txt, but looking at launch.py, that wasn't even necessary. Just create a new venv for the project, activate it, and run `python launch.py`.
boneitis 3y ago
To flesh out this response with a common way to do this, the free tier of ngrok will likely help someone accomplish this, by running it on the server to get a routable endpoint to your ssh port (wouldn't hurt to further lock it down with any firewalling, if that's available ngrok-side.. haven't used ngrok in a while). For example, to run the Stable Diffusion webui (which defaults to port 7860), it would be something not too far off from: $ ssh -p1234 -L7860:127.0.0.1:7860 assigned.dns.at.startup.ngrok.io (IIRC ngrok free-tier will allocate you a random port on the public side every time you start the service) Then, can just browse localhost:7860 from the remote field machine. I got bit by the AI bug a few days ago, and I already rent some lightsail instances, one of which has a static IP reserved and one of my domain names pointing to it. So, I set up something a little more convoluted and perhaps unnecessarily complex, turning that lightsail VM into a jumpbox/bastion host. Anywho: AutoSSH from server to my lightsail instance, with two remote port forwards (not local port forwards): SSH and SD webui. Then, connect from the field machine anywhere in the world to the jumpbox with matching local port forwards. (I set up both ports, so I can shell back into the original machine, but this isn't strictly required.) Then, fire up localhost:7860 in the web browser. Make sure this isn't being served on 0.0.0.0 or un-firewalled. (e: I re-read original question after posting and realized GP was merely asking for over the LAN, but hey, now they know how to do this over the 'net :)
- microbass 3y ago
  Even easier is to use Tailscale. Run tailscale on your laptop and server, and then access the web UI by using <tailnet_server_ip>:<port>