It's one of those triangles with speed \ accuracy / cost. You can have a small model that's cost effective to serve, and gives fast responses, but will be wron

It's one of those triangles with speed \ accuracy / cost. You can have a small model that's cost effective to serve, and gives fast responses, but will be wrong half the time. Or you can have a large model that's slow to run on cheap hardware, but will give more accurate answers. This is usually only fast enough for personal use. And the third option with a large model that's fast and accurate, and you'll have to pay Nvidia/Groq/etc. a small fortune to be able to run it at speed and also probably build a solar powerplant to make it cost effective in power use.