4 ms·
7B parameter models are more than enough for this and run faster than talking pace on even a low end CPU. Even a finetuned 3B model would be excellent for gene
by MacsHeadroom 3y ago
7B parameter models are more than enough for this and run faster than talking pace on even a low end CPU.
Even a finetuned 3B model would be excellent for generative agents and only use about 2GB of RAM to at high speeds on even a single core CPU.
- holografix 3y agoCan you share some examples Of what models you’re referring to?
- T-A 3y agohttps://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp https://huggingface.co/mosaicml/mpt-7b https://huggingface.co/mosaicml/mpt-7b