3 ms·
There is, called "GPT-NeoX", or its TPU-based predecessor GPT-Neo. However, even running inference on these models is much, much harder than Stable Diffusion --
by spijdar 4y ago
There is, called "GPT-NeoX", or its TPU-based predecessor GPT-Neo. However, even running inference on these models is much, much harder than Stable Diffusion -- the GPT-NeoX-20B weights for GPT-NeoX requires a minimum of two GPUs with 24 GB of VRAM each to simply run inference, never-mind training or fine-tuning.
I believe there are some tricks for cutting down the VRAM requirements a bit by dropping precision at different points, but the gist is that these big text models are actually quite a bit more resource intensive than the image models.
- Tepix 4y agoWhere can i find more information about running GPT-NeoX? Is it covered in the paper? Or is there a forum, HOWTO or Wiki somewhere?
- juliensalinas 4y agoYou could follow EleutherAI's official guide: https://github.com/EleutherAI/gpt-neox https://github.com/EleutherAI/gpt-neox You could also use a hosted service that proposes GPT-NeoX like https://nlpcloud.com https://nlpcloud.com or https://goose.ai https://goose.ai
- tinco 4y agoInteresting that they're saying they have a target of replicating GPT-3 with 175B parameters, but since the RETRO paper it should (in theory?) be possible to replicate GPT-3 with a tenth of the parameters. Are they not planning to use a retrieval system, is it difficult to adopt their system for retrieval, or is their readme out of date?
- deleted 4y ago[deleted]