I'm working on this problem in my spare time, it requires a GPU (or two) RTX 3090 or 4090 because of the need for vRAM memory but I could also forsee APIs in fr

I'm working on this problem in my spare time, it requires a GPU (or two) RTX 3090 or 4090 because of the need for vRAM memory but I could also forsee APIs in front that allow worker nodes to run the quantized models for cheaper. the problem is model switching is hard so you would have to pick which model to run upfront for inference openchat in particular ships with an inference server compatible with openai follow me here: https://www.youtube.com/@fxhp1 https://www.youtube.com/@fxhp1