4 ms·
Neat idea. Regarding the performance, I think you could get a lot better performance by training a small classifier model, essentially an embedding model, and u
by kevmo314 2y ago
Neat idea. Regarding the performance, I think you could get a lot better performance by training a small classifier model, essentially an embedding model, and using the LLM as the distillation source. This would both be much smaller, addressing your desire for it to run in browser, while also being much more performant, addressing your quantization need. Using the full LLM is a bit overkill and you can extract the core of what you're looking for out of it with something a little custom.
- dheera 2y agoI just wish the stupid chatbots on customer service websites would show "Get human" as an option sooner. 99% of time there is a mistake and the bots don't have authority to correct it.
- indexerror 2y agoThats why I go for the nuclear option (the one that requires higest attention, like account cancellation) when presented with a list of options by a bot. More likely to get a person that way.