philomath868

4 ms·

1. 1.▲
   by philomath868 1y ago

   I hear you loud and clear... Thanks! What about deleting vision layers (e.g. the "multi_modal_projector" and the "vision_tower.vision_model" layers, assuming I go with Gemma 3), since I need just language generation? Wou
2. 2.▲
   by philomath868 1y ago

   Perhaps. But I don't think there is an existing (open weights) model that really knows YIVO Yiddish, either, so what should I base this fine-tuning on?
3. 3.▲
   by philomath868 1y ago

   Thank you! The language is Hasidic Yiddish (which is by now different enough from YIVO Yiddish to almost be considered a different language). The amount of (all kinds of) Yiddish included in pre training is probably very little, but not not
4. 4.▲
   by philomath868 1y ago

   Thank you! I have thought about initializing the new embeddings based on equivalent tokens in the old ones (e.g. by translating a token to English and finding the closest old token), but this is all getting convoluted. New tokenizer and emb
5. 5.▲
   Ask HN: Best foundation model for CLM fine-tuning?

   28 points by philomath868 1y ago|19 comments
6. 6.▲
   by philomath868 1y ago

   How does the "continuous training pipeline" work? You rebuild the model after every N corrections, with the corrections included in the data?