3 ms·
Well, LLMs don't consistently beat purpose-trained BERT-type models just yet, for example in [1] the authors show RoBERTa (fine-tuned for each individual task s
by miven 3y ago
Well, LLMs don't consistently beat purpose-trained BERT-type models just yet, for example in [1] the authors show RoBERTa (fine-tuned for each individual task separately of course) going basically toe-to-toe with GPT-4 on quite a few somewhat conventional NLP tasks, while open-source LLaMA 2 models are getting severely outclassed.
[1] https://arxiv.org/abs/2308.10092 https://arxiv.org/abs/2308.10092
- minimaxir 3y agoYou'll get far more than 80% of the way with just running a pretrained text embeddings model. There's always room for optimization (e.g. finetuning on your own data) but it's an incredible baseline.