4 ms·

Any idea on how it does on other languages? In particular, non-Latin languages like Arabic, Persian, Urdu, Hindi, etc.?

• brucethemoose2 3y ago
  There will be finetunes for other languages just like LLaMAv1
  • 1024core 3y ago
    How can you finetune for a new language? Aren't the tokens baked in by the time the model is done training?
    • brucethemoose2 3y ago
      Apparently not. shrug The backend does sometimes need a new tokenizer, depending on how its implemented.