3 ms·
I agree with this in the sense that once you have enough, you stop caring about the metric.
by frabjoused 2y ago
I agree with this in the sense that once you have enough, you stop caring about the metric.
- paradite 2y agoAnd how much RAM do you need to run Mixtral 8*22B? Probably not enough on a personal laptop.
- Lacerda69 2y agoI run it fine on my 64gb RAM beast.
- coder543 2y agoAt what quantization? 4-bit is 80GB. Less than 4-bit is rarely good enough at this point.
- apexalpha 2y agoIs that normal ram of GPU ram?
- samus 2y ago64GB is not GPU RAM, but system RAM. Consumer GPUs have 24GB at most, those with good value/price have way less. Current generation workstation GPUs are unaffordable; used can be found on ebay for a reasonable price, but they are quite slow. DDR5 RAM might be a better investment.
- user_7832 2y agoGenerally about ~1gb ram per billion parameters. I've run a 30b model (vicuna) on my 32gb laptop (but it was slow).