4 ms·
SetFit is fairly good, and we do help train SetFit like models for the results you get, however, the issue with SetFit is that its latency and cost benefits com
by hellovai 3y ago
SetFit is fairly good, and we do help train SetFit like models for the results you get, however, the issue with SetFit is that its latency and cost benefits come at the price of flexibility.
If you want to add a new class, to update an existing one, it requires training a new model. Sometimes this is ok and sometimes it's not. This is why we generally prefer a hybrid approach where some classes are using traditional models (BERT based) while others are determined by the LLM.
- specproc 3y agoI guess use case is everything. There are numerous reasons, not least of which confidentiality, why chatgpt is a no go for me. What I'd like to see more of is systematic comparison between chatgpt and classic models. I was hoping to see a bit of this in this article and was disappointed.
- hellovai 3y agoI appreciate the feedback, and also agree that chatgpt is a no-go for many use cases. We're working on putting together a better comparison specifically along the lines of accuracy between the LLMs (chatgpt, bard, falcon) and also traditional models. Hope that one hits the spot for you! Are their specific metrics you think might be interesting? We were primarily looking at f1/accuracy for this task, but also attempting to see what types of classes they work well in using semantic similarity.