4 ms·
This is a weakness of docker, a bit, I think. I was rigging this up, myself, and conciscious of the fact that basic docker is "all or none" for container port
by meltyness 8mo ago
This is a weakness of docker, a bit, I think.
I was rigging this up, myself, and conciscious of the fact that basic docker is "all or none" for container port forwarding because it's for presenting network services, had to dig around with iptables so it'd be similar to binding on localhost.
The use case https://github.com/meltyness/tax-pal https://github.com/meltyness/tax-pal
The ollama container is fairly easy to deploy, and supports GPU inference through container toolkit. I'd imagine many of these are docker containers.
e: i stand corrected, apparently -p of `docker run` can have a binding interface stipulated
e2: https://docs.docker.com/engine/containers/run/#exposed-ports https://docs.docker.com/engine/containers/run/#exposed-ports which is not in some docs
e3: but it's in the man page ofc
- threecheese 8mo agoOut of curiosity, why would you need to wrap the call to an Ollama modelfile in docker? Does the dockerized ollama client provide some benefit, when it’s shelling down to local Ollama instance anyway? (Wrt tax-pal)
- meltyness 8mo agoIt's more of a distribution thing for me really. I'm basically using docker as a package manager since they otherwise distribute through one of those ad-hoc shell scripts that I'd prefer to avoid accidentally breaking Debian with somehow. I've built ollama before too, but, I like that I can cleanly rip it out of my system or upgrade it without handing root off to some shell script somewhere I guess. If anyone's gonna bash up my system it oughta be me
- threecheese 8mo agoMakes sense, thank you!
- cyberax 8mo agoI'm using open-webui project to host a Web UI for Ollama, and Ollama itself in Docker containers. It's super-useful, because I don't have to worry about it blowing up stuff on my system with automatic installations.
- throw20251220 8mo agoYou work with Linux. On a mac, you run ollama on the host because the gpu is not available in the container.
- crimsonnoodle58 8mo agoYes the binding interface can be specified, but the default for -p 11434:11434 is 0.0.0.0. IMO the default should be 127.0.0.1 and the user should have to explicitly bind to all via -p 0.0.0.0:11434:11434.
- meltyness 8mo agoApparently been that way for a while haha https://github.com/moby/moby/commit/1cbdaebaa1c2326e57945333420d25d6f77011d5 https://github.com/moby/moby/commit/1cbdaebaa1c2326e57945333...
- xhcuvuvyc 8mo agoDocker has a lot of lazy hacks to make it work well on MacOS that had to have it running in a VM for any of the linux containers to work.