3 ms·
Thanks for the q. The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here: https://tally.so/r/w2XVQA https://tally.so/r/
by coffeephoenix 2y ago
Thanks for the q.
The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here:
https://tally.so/r/w2XVQA https://tally.so/r/w2XVQA
In the free trial of Log10.io you can use the ICL (self-improving) version of AutoFeedback for hallucination detection and RAG evals. In generalize, you can customize evals for a wide range of use cases using AutoFeedback:
https://docs.log10.io/feedback/auto_feedback https://docs.log10.io/feedback/auto_feedback
https://github.com/log10-io/log10-cookbook/tree/main https://github.com/log10-io/log10-cookbook/tree/main
https://arjunbansal.substack.com/p/hybrid-evaluation-scaling-human-feedback https://arjunbansal.substack.com/p/hybrid-evaluation-scaling...
As for models, we support evaluating a wide range of models including Llama - in addition we support OpenAI, Anthropic, Gemini, Mistral, MosaicML, Together, and self-hosted models. We are also compatible with frameworks such as Langchain, Magentic and LiteLLM.
For how to integrate different models please see here:
https://github.com/log10-io/log10 https://github.com/log10-io/log10
- ruby314 2y agoAlso wanted to address the confusion on the role of llama-3-8b and llama-3.1-8b. In the blog post we use these models as an example of an evaluator LLM. We select what is best for your custom eval under the hood. LSR is just one example of the research powering our custom evals