3 ms·

There is a Colab that you can run to set up the model with TPUs: https://colab.research.google.com/github/kingoflolz/mesh-transformer-jax/blob/master/colab_demo

• Firerouge 5y ago
  It's interesting, it seems to get stuck on a sentence or word a lot, then half the output is repeating a slightly varied sentence, or rambling about that unrelated word.
  • minimaxir 5y ago
    That's the one thing very consistent with Transformer models, even GPT-3.