3 ms·

Yes, and also waiting for the next iteration of Gemma. Muse or Qwen are optimized for coding, while IMO Gemma is still better for non-coding tasks. https://x.c

• dannyw 2mo ago
  You can partially tell by the tokeniser; which gives you some hint into the training corpus mix. </div> is four Gemma4 tokens, but one Qwen3.6 token.
  • venusenvy47 2mo ago
    Where do you find this information for each model?
    • ComputerGuru 2mo ago
      The tokenizers are included in the open s̶o̶u̶r̶c̶e̶ weights releases; you wouldn’t be able to use the weights without the corresponding encoder/decoder, in fact.
    • adrian_b 2mo ago
      When you look on HuggingFace.co at the files of a model, for each model you will see a file "tokenizer.json". In that file you can see all tokens and their corresponding numeric codes.
  • deleted 2mo ago
    [deleted]
  • stymaar 2mo ago
    Looks like we have a /r/localllama dweller here.
• malshe 2mo ago
  I am working on a project where we have to classify customer calls into more than 10 categories. As the client wants everything locally I tried a few local LLMs. Gemma turned out to be the best model for this task. The classification accuracy is impressive, and the client is happy that I am using an American model.
  • LeBit 2mo ago
    I guess the client is American.
    • IshKebab 2mo ago
      Not necessarily. Except for SpaceXAI's (haha) models, there's less risk of reality distortion in Western models. Probably doesn't affect his use case but you wouldn't really want to ask Kimi K3 for a history of China for example.