3 ms·
Yes, because transfer learning works. A specialized model for X will be subsumed by a general model for X/Y/Z as it becomes better at Y/Z. This is why models wh
by a_wild_dandan 2y ago
Yes, because transfer learning works. A specialized model for X will be subsumed by a general model for X/Y/Z as it becomes better at Y/Z. This is why models which learn other languages become better at English.
Custom models still have use cases, e.g. situations requiring cheaper or faster inference. But ultimately The Bitter Lesson holds -- your specialized thing will always be overtaken by throwing more compute at a general thing. We'll be following around foundation models for the foreseeable future, with distilled offshoots bubbling up/dying along the way.
- kingkongjaffa 2y ago> This is why models which learn other languages become better at English. Do you have a source for that, I'd love to learn more!
- a_wild_dandan 2y agoEvaluating cross-lingual transfer learning approaches in multilingual conversational agent models[1] Cross-lingual transfer learning for multilingual voice agents[2] Large Language Models Are Cross-Lingual Knowledge-Free Reasoners[3] An Empirical Study of Cross-Lingual Transfer Learning in Programming Languages[4] That should get you started on transfer learning re. languages, but you'll have more fun personally picking interesting papers over reading a random yahoo's choices. The fire hose of papers is nuts, so you'll never be left wanting. [1] https://www.amazon.science/publications/evaluating-cross-lingual-transfer-learning-approaches-in-multilingual-conversational-agent-models https://www.amazon.science/publications/evaluating-cross-lin... [2] https://www.amazon.science/blog/cross-lingual-transfer-learning-for-multilingual-voice-agents https://www.amazon.science/blog/cross-lingual-transfer-learn... [3] https://arxiv.org/pdf/2406.16655v1 https://arxiv.org/pdf/2406.16655v1 [4] https://arxiv.org/pdf/2310.16937v2 https://arxiv.org/pdf/2310.16937v2