3 ms·
Is there any trained model for text generation that you can run locally yet?
by yreg 4y ago
Is there any trained model for text generation that you can run locally yet?
- throwaway743 4y agoPlenty. Huggingface alone has a ton
- deqwer 4y agoThere’s LAION working on open source[1] version of chatGPT [1] https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant
- turmeric_root 4y agoThough their roadmap doc says they're looking into finetuning existing GPT-J/T5 models for this task. So you'll probably want a 3090 (24GB VRAM) and at least 16GB of CPU RAM to run inference if/when the project is complete.
- Metus 4y agoThis should be way higher up.
- wongarsu 4y agoGPT2 can be run locally (on a somewhat beefy consumer GPU)
- karmajuney 4y agoCan you add some info on what consumer GPU would be needed for this? Would a 3080 be able to handle this?
- wongarsu 4y agoAssuming you get the 12GB version of the 3080. A 2080TI is another option. Though you can reduce precision or use one of the smaller GPT2 versions to run on smaller cards as well.
- int_19h 4y agoLet me slightly rephrase the question: what is the best model that one can run on high-end consumer grade hardware? Let's say RTX 3090.
- minimaxir 4y agoThe original GPT-2 small (the 124M one) can run on a CPU, just slowly and not scalably.