I've tested it and many fine tunes and while it seems to work ok for short chat it's really inattentive in summarisation and information extraction tasks. Conti

I've tested it and many fine tunes and while it seems to work ok for short chat it's really inattentive in summarisation and information extraction tasks. Continuation of long context stories/chat is also severely lacking, making up lot of contradictory details as it goes on. It might or might not be better in benchmarks, but I just don't see it in practice for many tasks where you'd pick a LLM and go There is one thing tho where I've seen it perform really well, which is single task fine tuning. It's small enough you can do full tuning and it's steerable enough you don't need more than few thousand samples. But it's definitely not a pick and go model like a Vicuna-13b would be.