4 ms·
70GB of RAM would cost around $150 these days depending on how you get there. 64GB (2x32GB) of DDR4 is around $140 then another 8GB stick would be around $15.
by gymbeaux 3y ago
70GB of RAM would cost around $150 these days depending on how you get there. 64GB (2x32GB) of DDR4 is around $140 then another 8GB stick would be around $15.
Used DDR3 ECC would be roughly half that.
- Semaphor 3y agoI think they are talking about VRAM
- gymbeaux 3y agooop you're right... I know once the model is trained they run on the CPU/regular RAM
- drdaeman 3y agoAnything that runs on GPU can be run on CPU, the only question is how much slower it's gonna be.
- Semaphor 3y agoI know, I have a weak GPU but 64 GB RAM. Using those models works, but it’s more "ask a question, then do something else for a while, while your fans spin up" ;)
- amstan 3y agoI've been running vicunda-13B on a workstation that can be aquired from ebay for about $500. It's slow compared to online services, but probably slightly faster than text to speech would recite its output, so plenty. Falcon 40B is probably too much for it, but apparently there's similar cheap hardware that could work.