3 ms·
ACL 2023 Tutorial: Retrieval-Based Language Models and Applications
- jerpint 3y agoIf anyone has any good pointers/references/benchmarks on how to evaluate such systems that would be really helpful
- TalktoCrystal 3y agoyes, the market is asking for this.
- cschmidt 3y agoWhile on the subject of ACL 2023 tutorials, here are two I went to: Complex Reasoning in Natural Language https://wenting-zhao.github.io/complex-reasoning-tutorial/ https://wenting-zhao.github.io/complex-reasoning-tutorial/ Generating Text from Language Models https://rycolab.io/classes/acl-2023-tutorial/ https://rycolab.io/classes/acl-2023-tutorial/ the second one in particular was very well taught. I didn't catch the Retrieval based talk, since it conflicted with the text generation one.
- TalktoCrystal 3y agoThis is useful
- monkeydust 3y agoWere using retrieval-based system for Q&A for a complex product. Feeding in user guides, release notes even transcribed some tutorial videos. Its quick to create a 'wow' style demo but putting into production is challenge with hallucinations and testing/benchmarks hard to do given the stochastic nature of the output each time your run with same input (temp>0). Curious on pointers if anyone is doing something similar.
- sharemywin 3y agoI'm curious what LLM your using. I would think some of the open source ones could be trained to use the facts given with much more accuracy.
- monkeydust 3y agoOpenai Gpt4. retrieval works best given the data source although debating if we should explore tuning as a next step