3 ms·
Show HN: I implemented evals metrics for LLMs that runs locally on your machine
- ChrisPzilla 3y agoOh cool - we just did a writeup on Ragas so will check this one out!
- srbhr 3y agoThe idea around a testing framework for LLMs is nice. Can you provide some examples where this can be used?
- 3d27 3y agoThe package I built is like a provider for 10+ different evaluation metrics that run both locally on your machine using models from hugging-face but also on the cloud IF you want more functionality. If you want to evaluate a fine-tuned model, we have integrations with LM Harness and Stanford HELM coming out. If you want to evaluate a RAG application, we have 7+ metrics available for that. You can also create your custom metrics using our interface!
- mertbozkirr 3y ago[dead]