I run LLMs locally on a 2080TI I bought used years ago for another deep learning project. It's not the fastest thing in the world, but adequate for running 8B m

I run LLMs locally on a 2080TI I bought used years ago for another deep learning project. It's not the fastest thing in the world, but adequate for running 8B models. 70B models technically work but are too slow to realistically use them.