3 ms·
This runs fine in RAM constrained (<2gb) situations: https://huggingface.co/EleutherAI/gpt-neo-125M https://huggingface.co/EleutherAI/gpt-neo-125M It's bigger
by smoldesu 4y ago
This runs fine in RAM constrained (<2gb) situations: https://huggingface.co/EleutherAI/gpt-neo-125M https://huggingface.co/EleutherAI/gpt-neo-125M
It's bigger brother, 1.3b, uses ~5.5gb of memory but yields slightly more GPT-like answers. Both take ~5-20 seconds to generate a response though, so take that into account when building with it.
- bcarambio 4y agoHi from OctoML -- our team just put out some benchmarking results using GPT-J with a few model config/hw combinations that have relatively low latency: /octoml.ai/blog/evaluating-gpt-j-deployment-options/
- bcarambio 4y agoHi from OctoML -- our team just put out some benchmarking results using GPT-J with a few model config/hw combinations that have relatively low latency: https:/octoml.ai/blog/evaluating-gpt-j-deployment-options/