2 ms·
That's for the max-size model (6B parameters). GPT-Neo has pruned models 125m and 1.3b that fit into 1.5gb and 6gb of memory respectively. These models are obv
by smoldesu 4y ago
That's for the max-size model (6B parameters). GPT-Neo has pruned models 125m and 1.3b that fit into 1.5gb and 6gb of memory respectively.
These models are obviously nerfed, but you can run them on budget ARM hardware and get legible, paragraph-length responses in less than 10 seconds.
- flangola7 4y agoThere's a 20B model now
- smoldesu 4y agoThe 20B model was the original I believe, and the other models were all pruned from it.
- atylerrice 4y agoThis is incorrect. Thought would be interesting. Pruning usually removes some parameters from the model (setting certain ones to 0 for instance). The smaller models all have less layers and therefore weren’t pruned but were smaller to begin with.