4 ms·
Does anyone know why they haven't released any 30B-ish param models? I was expecting that to happen with this release and have been disappointed once more. They
by TheAceOfHearts 2y ago
Does anyone know why they haven't released any 30B-ish param models? I was expecting that to happen with this release and have been disappointed once more. They also skipped doing a 30B-ish param model for llama2 despite claiming to have trained one.
- nickpsecurity 2y agoMaybe they think more people will just use quantized versions of 70B.
- prvc 2y agoWhy should they?
- TheAceOfHearts 2y agoUnless I'm misremembering, they announced it at one point. It's just giving people more options.
- prvc 2y agoThat was for version 2, not 3 or 3.1, if I recall correctly.
- michaelt 2y agoI suspect 30B models are in a weird spot, too big for widespread home use, too small for cutting edge performance. For home users 7B models (which can fit on an 8GB GPU) and 13B models (which can fit on a 16GB GPU) are in far more demand. If you're a researcher, you want a 70B model to get the best performance, and so your benchmarks are comparable to everyone else.
- drdaeman 2y agoI thought home use is whatever fits in 24GB (a single 3090 GPU, which is pretty affordable), not 8 or 16. 30B models fit.
- michaelt 2y agoWhile some home users do indeed have 24GB of vram, the fact is a 4090 costs $1700 Such models will never top the number of downloads charts, or the community hype, as there’s just loads more people who can use the smaller models. And if you can afford one 4090 you can probably afford two.
- drdaeman 2y agoWhy 4090, though? I read (and agree) that 3090 is generally considered to be the best bang for the buck: 24GB, priced at $800-1000 range, and giving decent TPS for LLMs.