2 ms·
It's quite unlikely that you would even be able to run it at all. The model is quite large and probably won't fit in memory. If you could get it to run, it woul
by verdverm 4y ago
It's quite unlikely that you would even be able to run it at all. The model is quite large and probably won't fit in memory. If you could get it to run, it would be extremely slow
Worth checking out: https://github.com/karpathy/nanoGPT https://github.com/karpathy/nanoGPT The associated videos will go into this more, iirc, at the end he said the real GPT-2 would take 5-20 seconds to generate a small amount of text using CPU only.