4 ms·
And how much RAM do you need to run Mixtral 8*22B? Probably not enough on a personal laptop.
by paradite 3y ago
And how much RAM do you need to run Mixtral 8*22B? Probably not enough on a personal laptop.
- Lacerda69 3y agoI run it fine on my 64gb RAM beast.
- coder543 3y agoAt what quantization? 4-bit is 80GB. Less than 4-bit is rarely good enough at this point.
- apexalpha 3y agoIs that normal ram of GPU ram?
- samus 3y ago64GB is not GPU RAM, but system RAM. Consumer GPUs have 24GB at most, those with good value/price have way less. Current generation workstation GPUs are unaffordable; used can be found on ebay for a reasonable price, but they are quite slow. DDR5 RAM might be a better investment.
- user_7832 3y agoGenerally about ~1gb ram per billion parameters. I've run a 30b model (vicuna) on my 32gb laptop (but it was slow).