The most promising are probably one of the smaller versions of bloom https://huggingface.co/docs/transformers/model_doc/bloom https://huggingface.co/docs/transf

The most promising are probably one of the smaller versions of bloom https://huggingface.co/docs/transformers/model_doc/bloom https://huggingface.co/docs/transformers/model_doc/bloom or gpt-j https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax