Finetuning might not be the best word; sometimes it is a grey line. Token embeddings can be trained without changing the other parameters. There is a number of

Finetuning might not be the best word; sometimes it is a grey line. Token embeddings can be trained without changing the other parameters. There is a number of models which add tokens as a finetuning step. Here is recently StarCoder adding ChatML-equivalent tokens: https://huggingface.co/blog/starchat-alpha#a-standard-format-for-dialogues https://huggingface.co/blog/starchat-alpha#a-standard-format...