I have a similar pc and I use text-generation-webui and mostly exllama quantized models. I also deploy text-generation-webui for clients on k8s with gpu for si

I have a similar pc and I use text-generation-webui and mostly exllama quantized models. I also deploy text-generation-webui for clients on k8s with gpu for similar reasons. Last I checked, llamafile / ollama are not as optimised for gpu use. For image generation I moved from automatic webui to comfyui a few months ago - they're different beasts, for some workflow automatic is easier to use but for most tasks you can create a better workflow with enough comfy extensions. Facefusion warrants a mention for faceswapping