4 ms·
Can you elaborate on the not cost effective part? That seems surprising unless the API providers are running at a loss.
by babyshake 2y ago
Can you elaborate on the not cost effective part? That seems surprising unless the API providers are running at a loss.
- amazingamazing 2y agoA 4090 is $1000 at least. That’s years of subscription for the latest models, which don’t run locally anyway.
- homarp 2y agounless you have one already for playing games
- wongarsu 2y agoI run LLMs locally on a 2080TI I bought used years ago for another deep learning project. It's not the fastest thing in the world, but adequate for running 8B models. 70B models technically work but are too slow to realistically use them.