apitman

16 ms·

421.
by apitman 1y ago
Not exactly. OWUI is a server with a web app frontend. Jan is a desktop app you install. But it does have the ability to run a server for other apps like OWUI to talk to.
422.
by apitman 1y ago
Their new app is closed source right?
423.
by apitman 1y ago
I was under the impression that context takes up a lot more VRAM than this.
424.
by apitman 1y ago
Did you test multi-user cases?
425.
by apitman 1y ago
So I still need SNI for IPv4 requests.
426.
by apitman 1y ago
But you're still using NAT and SNI for IPv4 hosts, so there's little incentive to add the complexity of IPv6.
427.
by apitman 1y ago
I'm not really familiar with IPv6. If I have a million IPv4 servers, it's pretty simple to set up a million subdomains and route incoming TLS requests using SNI. If I have a million IPv6 servers, can I somehow accomplish the same
428.
by apitman 1y ago
If I have a bunch of machines on my network and want them all to be able to access the internet, I can use NAT and let them all share a single IP. No one on the internet needs to do anything. If I have a bunch of servers and want them to be
429.
by apitman 1y ago
Yep. And the reason they were successful is because you can solve the problem on your end without the other end needing to do anything. IPv6 requires both parties to do something. So now we're stuck with NAT and SNI.
430.
by apitman 1y ago
IPv4 is never going away barring massive adoption of p2p protocols to drive the switch. Sadly NAT and SNI solve most of the problems well enough for things to limp along indefinitely. The only orgs with the power to fix this from the top do
431.
by apitman 1y ago
You can open a file for performant access in all major browsers. It's the same API used for uploading files (<input type="file" />), but you can also just load them into memory and do stuff.
432.
by apitman 1y ago
Does anyone know why all of these WebGPU LLM demos have you download the models to browser storage rather than letting you open a gguf already on your local drive? I have several models downloaded already that I would be interested in tryin
433.
by apitman 1y ago
Doesn't Msty alreayd run models? What is LM Studio needed for in this setup?
434.
by apitman 1y ago
Good to know, thanks. What do people generally use to connect to it for chat?
435.
by apitman 1y ago
Is the Open WebUI license still OSI-compatible? I saw some drama about this on reddit but I'm not sure about the current state. https://docs.openwebui.com/license/
436.
by apitman 1y ago
Is there a way to get LM Studio to talk to remote OpenAI API servers and cloud providers?
437.
by apitman 1y ago
Kudos for making an effort to keep it small. Some of these Electron AppImages are 1GB+ which is pretty wild. How do you handle markdown rendering?
438.
by apitman 1y ago
I've been on something of a quest to find a really good chat interface for LLMs. Most import feature for me is that I want to be able to chat with local models, remote models on my other machines, and cloud models (OpenAI API compatibl
439.
by apitman 1y ago
Most humans don't know what a script is
440.
by apitman 1y ago
Like I said, an order of magnitude harder than it should be. It should be install app, do an oauth flow to open a tunnel, done.
441.
by apitman 1y ago
Backed up from what? People only use the cloud for backups, at which point the e2ee is compromised.
442.
by apitman 1y ago
If I only use 1 device for 5 years, then lose it, how do I recover?
443.
by apitman 1y ago
I've tried to adopt Matrix multiple times over years. IMO the UX is evidence that e2ee is not solved.
444.
by apitman 1y ago
Tailscale + Proxmox is still an order of magnitude more difficult than this is going to need to be.
445.
by apitman 1y ago
Signal is a promising example, but ephemeral messaging is a very different use case from things like photo management.
446.
by apitman 1y ago
I'm optimistic about self-hosting/self sovereignty (which both fall under the umbrella of what I call indie hosting) long term. But I think both of these articles gloss over the fact that end-to-end encryption has never been shown
447.
by apitman 1y ago
Those red Xs represent libraries that only work on Nvidia GPUs and would represent a massive amount of work to re-implement in a cross-platform way, and you may never achieve the same performance either because of abstraction or because you
448.
by apitman 1y ago
Do you have a link? I went to the rust GPU repo and didn't see anything. I have an academic pipeline that currently heavily tied to CUDA because we need nvCOMP. Eventually we hope us or someone else makes an open source gzip library fo
449.
by apitman 1y ago
Could something like this be done in WebGPU?
450.
by apitman 1y ago
I'm not generally an open source purest but I believe their license is no longer OSI-compatible.

More ›