One Docker command if you don't mind waiting minutes for CPU-bound replies: https://localai.io/ https://localai.io/ You can also use several GPU options, but

One Docker command if you don't mind waiting minutes for CPU-bound replies: https://localai.io/ https://localai.io/ You can also use several GPU options, but they are not as easy to get working.