The best place to start is the localllama subreddit. The install guide describes text generation webui, a generic frontend for multiple ways of running models

The best place to start is the localllama subreddit. The install guide describes text generation webui, a generic frontend for multiple ways of running models locally, and llama.cpp, a way of running llama-derived models locally on CPU from the command-line. https://www.reddit.com/r/LocalLLaMA/wiki/guide/ https://www.reddit.com/r/LocalLLaMA/wiki/guide/ The bundled models that the oobabooga offers during install are probably not what you want. To find good models the localllama wiki has a handy models page. GPTQ models are only running on GPU, GGML models are for running on CPU or GPU through llama.cpp (directly or via textgen webui). https://www.reddit.com/r/LocalLLaMA/wiki/models/ https://www.reddit.com/r/LocalLLaMA/wiki/models/