2 ms·
External GPU is actually a really good option here. Get a laptop with a beefy CPU (definitely Ryzen; Strix Halo chips are really good and would be decent at sta
by LorenDB 1y ago
External GPU is actually a really good option here. Get a laptop with a beefy CPU (definitely Ryzen; Strix Halo chips are really good and would be decent at standalone AI as well) and Thunderbolt/Oculink, then grab an eGPU dock and get the best GPU you can afford. For gaming, eGPUs suffer from bandwidth constraints, but for LLMs it shouldn't matter as much since the model only needs loaded once; after that it can sit in VRAM.
- zyruh 1y agoThis is great - thank you!