33 ms·
Has anyone tried using this with GPT-2? I’m curious if it would run locally at usable speeds.
by FreakyT 6y ago
Has anyone tried using this with GPT-2? I’m curious if it would run locally at usable speeds.
- kevingadd 6y agoGPT-2 appears to require at least 24GB of memory, so if that's accurate absolutely not.
- minimaxir 6y agoYou can load the 1.5B GPT-2 into a 16GB VRAM GPU...with fp16 downcasting. Still not ideal for M1 though.
- minimaxir 6y agoFor the small 124M model, it’s already runable on the CPU. It’s going beyond that is tricky.
- FreakyT 6y agoI’ve run the 1.5B model on my CPU, and it works...just very very slowly.