Mosaic wrote about their new model here. https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b It was trained on 65k inputs and has decent p

Mosaic wrote about their new model here. https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b It was trained on 65k inputs and has decent performance working with 80k+ tokens.