There are a bunch of really good ideas used to train this model - multi query attention, infilling, near deduplication and dataset cleaning. I do wish that the

There are a bunch of really good ideas used to train this model - multi query attention, infilling, near deduplication and dataset cleaning. I do wish that the demo was a little more interactive (not needing to click buttons to create a generation) since it makes it hard to see the full power of the model. One of the things we tried at Codeium for our playground on browser was to make it super clear how well the model performs by making the experience interactive - https://www.codeium.com/playground https://www.codeium.com/playground