3 ms·

New knowledge cutoff date means this is a new foundation model?

• jimmypk 6mo ago
  [flagged]
• lkbm 6mo ago
  Yes, but doesn't the token change mean that?
  • clickety_clack 6mo ago
    You can train a tokenizer on old data just like you can train a model on old data.
    • wongarsu 6mo ago
      But you can't use an old model with a new tokenizer. Changing the tokenizer implies you trained the model from scratch
      • dannyw 6mo ago
        A little bit of post-training will fix that. Folks on /r/LocalLLaMa have been making effective finetunes with diff. tokenizers for years.