First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly. If you want to use the model through

First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly. If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory. Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder. Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.