4 ms·
GPT-J specifically was written using Google Cloud's TPU stuff. https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b https://github.com/kingoflolz/mesh-
by mosseater 4y ago
GPT-J specifically was written using Google Cloud's TPU stuff.
https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b
They even have a google colab notebook thingie. Not much setup needed, just an account.
https://colab.research.google.com/github/kingoflolz/mesh-transformer-jax/blob/master/colab_demo.ipynb#scrollTo=lwNETD2Uk8nu https://colab.research.google.com/github/kingoflolz/mesh-tra...