4 ms·
Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
by jarenmf 3y ago
Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
- thangngoc89 3y agoIt’s about VRAM, I would say the more the better, 4060 with 8GB should be the starting point
- nickthegreek 3y ago3060 with 12gb is cheaper and provides more vram.
- brucethemoose2 3y agoThis is not available in laptops, where the 3060 is stuck with 6GB.
- cosmojg 3y agoYou can always try sticking it into an eGPU enclosure.
- speedgoose 3y agoThe GPUs with the most VRAM you can justify spending money on.
- brucethemoose2 3y agoA 16GB 3080 is probably the cheapest and most ideal in a big laptop. But you can get some acceleration with anything ~6GB and up.
- thangngoc89 3y agoLaptop RTX has half the VRAM comparing to their PC counterparts. So 3080 laptop has 8GB
- brucethemoose2 3y agoAlso, what size and ballpark price are you looking for?
- tamimio 3y agoI had alienware with 3080 16 GB, while it was nice but the laptop is so buggy with all sorts of problems both hardware and software that I sold it at the end, still happy with my MSI Titan, bigger and heavier but overall better experience.
- jbellis 3y agoIf you insist on running models locally on a laptop then a Macbook with as much unified ram as you can afford is the only way to get decent amounts of vram. But you'll save a ton of money (and time from using more capable hardware) if you treat the laptop as a terminal and either buy a desktop or use cloud hardware to run the models.
- sundarurfriend 3y agoCloud hardware like? (Is Google Colab the best option, or even one of the best? Is Paperspace Gradient any good? Others?)