3 ms·
ModernBERT may be a better base model if finetuning a model for a specific use case: https://huggingface.co/blog/modernbert https://huggingface.co/blog/modernbe
by minimaxir 1y ago
ModernBERT may be a better base model if finetuning a model for a specific use case: https://huggingface.co/blog/modernbert https://huggingface.co/blog/modernbert
- diwank 1y agoalso ettin is a new favorite and a solid alternative: https://huggingface.co/jhu-clsp/ettin-encoder-1b https://huggingface.co/jhu-clsp/ettin-encoder-1b I'd encourage you to give setfit a try, along with aggressively deduplicating your training set, finding top ~2500 clusters per label, and using setfit to train multilabel classifier on that. Either way- would love to know what worked for you! :)