4 ms·
GPT2 can be run locally (on a somewhat beefy consumer GPU)
by wongarsu 4y ago
GPT2 can be run locally (on a somewhat beefy consumer GPU)
- karmajuney 4y agoCan you add some info on what consumer GPU would be needed for this? Would a 3080 be able to handle this?
- wongarsu 4y agoAssuming you get the 12GB version of the 3080. A 2080TI is another option. Though you can reduce precision or use one of the smaller GPT2 versions to run on smaller cards as well.
- int_19h 4y agoLet me slightly rephrase the question: what is the best model that one can run on high-end consumer grade hardware? Let's say RTX 3090.
- minimaxir 4y agoThe original GPT-2 small (the 124M one) can run on a CPU, just slowly and not scalably.