3 ms·
No, that's not how anything works. You can finetune an LLM to a new tokenizer by nudging just a few layers (even wildly different kinds of tokens), and there's
by Chu4eeno 1mo ago
No, that's not how anything works.
You can finetune an LLM to a new tokenizer by nudging just a few layers (even wildly different kinds of tokens), and there's nothing stopping a lab from using someone else's tokenizer.