3 ms·
The size of the model you need to get good enough generation with something like GPT-2 is going to be pretty impractical on a raspberry pi. You might maybe be a
by Voloskaya 6y ago
The size of the model you need to get good enough generation with something like GPT-2 is going to be pretty impractical on a raspberry pi.
You might maybe be able to fit a 3-layer distilled GPT-2 in RAM (not quite sure what the latest RPI have in term of RAM, 4GB?), but the latency is going to be pretty horrible (multiple seconds).