4 ms·
What kind of hardware are you using to run the LLMs locally?
by atentaten 1y ago
What kind of hardware are you using to run the LLMs locally?
- alienbaby 1y ago5090 GPU, and on a system with 64GB ram. I'm comfortably running 8b param models
loading thread…