7B seems to be the limit of what people can comfortably fit in last-gen GPUs, having ~6GB of VRAM. It's also the lower acceptable boundary of coherence for gene

7B seems to be the limit of what people can comfortably fit in last-gen GPUs, having ~6GB of VRAM. It's also the lower acceptable boundary of coherence for generative text. There are some major releases at lesser parameter counts though. Databricks' Dolly had a 3B model, and Microsoft's Orca also had a recent 3B release. They're both abysmal at generating text, but I find them quick and useful for reductive tasks ("summarize this," "extract keywords from," etc.). (I like to treat parameter count as a measure of age/WIS/INT. For this question, do I need the wisdom of a 7-year old, a 13-year old, a 30-year old, etc. 3B is like polling preschoolers at daycare.)