4 ms·
Oh that's unfortunate, can't the models be exported to pytorch through e.g onnx?
by The_rationalist 6y ago
Oh that's unfortunate, can't the models be exported to pytorch through e.g onnx?
- machinevision 6y agoEven small models can be a headache to export if they have even they use anything custom. I can't even imagine something the size of GPT-3.
- Voloskaya 6y agoLarger models aren't really more complicated than smaller ones though. GPT-2 is already supported, I believe only difference with GPT-3 is sparse attention.
- minimaxir 6y agoThe GPT-2 models included with Transformers can export to ONNX fine w/ a helper included with the Python onnxruntime.
- minimaxir 6y agoTo run in PyTorch the model architecture must be ported. ONNX is slightly different; you could export the model to run in the onnxruntime but that has tradeoffs.
- stellaathena 6y agoThere's a PyTorch + DeepSpeed repository here: https://github.com/EleutherAI/gpt-neox https://github.com/EleutherAI/gpt-neox