2 ms·
What’s the cheapest machine I could Buy to do this?
by bilsbie 3y ago
What’s the cheapest machine I could
Buy to do this?
- sosuke 3y agoShooting from the hip here I’d say $800 would be a good start for all new parts. You might get down to $500 with used parts. The most expensive piece in either build is a video card. You want to be able to load the LLM file into your video card RAM. I just got a 16gb new card for this. I can load up to 34b models on it but poorly. Anything 13b or less runs perfectly. A 12gb card would be able to run 7b models and with the right training I think 7b models can be awesome. New 4060ti Used 3090
- talldayo 3y agoMistral-7B will run on a Raspberry Pi at q04 and a little bit of patience. For good acceleration though, you'll want an Nvidia machine with enough VRAM to comfortable store the model.