3 ms·
But is the 7B model any good and actually production worthy for things like RAG?
by binarymax 3y ago
But is the 7B model any good and actually production worthy for things like RAG?
- treprinum 3y agoNot as good as GPT-4 of course but nor far from 3.5 if you just need to reword whatever returned by the retrieval. It's like losing 20 IQ points which might be still better than most support interactions I had.
- huac 3y agoI'm writing a blog post with some more reasoning but my view is that it can be useful for certain simpler tasks (eg unstructured -> structured, basic summarization) and not more complex things (eg generation). The tricky thing is that finetuning makes a big difference, and while it should be possible to hotswap LoRA adapters (at some cost to performance), I haven't figured that out yet.