8 ms·
Launch HN: Traceloop (YC W23) – Detecting LLM Hallucinations with OpenTelemetry
Hey everyone, we are Nir and Gal from Traceloop (https://www.traceloop.com https://www.traceloop.com). We help teams understand when their LLM apps are failing or hallucinating at scale. See a demo: https://www.traceloop.com/video https://www.traceloop.com/video or try it yourself at https://www.traceloop.com/docs/demo https://www.traceloop.com/docs/demo.
When moving your LLM app to production, significant scale makes it harder for engineers and data scientists alike to understand when their LLM is hallucinating or returning malformed responses. When you get to millions of calls to OpenAI a month, methods like “LLM as a judge” can’t work at a reasonable cost or latency. So, what most people we talked to usually do is sample some generations by hand, maybe for some specific important customers, and manually look for errors or hallucinations.
Traceloop is a monitoring platform that detects when your LLM app fails. Under the hood, we built real-time versions of known metrics like faithfulness, relevancy, redundancy, and many others. These are loosely based on some well-known NLP metrics that work well for LLM-generated texts. We correlate them with changes we detect in your system - like updates to prompts or to the model you’re using - to detect regressions automatically.
Here are some cool examples we’ve seen with our customers -
1. Applying our QA relevancy metric to an entity extraction task, we managed to discover issues where the model was not extracting the right entities (like an address instead of a person’s name); or returning random answers like “I’m here! What can I help you with today?”.
2. Our soft-faithfulness metric was able to detect cases in summarization tasks where a model was completely making up stuff that never appeared in the original text.
One of the challenges we faced was figuring out how to collect the data that we need from our customers' LLM apps. That’s where OpenTelemetry came in handy. We built OpenLLMetry (https://github.com/traceloop/openllmetry https://github.com/traceloop/openllmetry), and announced it here almost a year ago. It standardized the use of OpenTelemetry to observe LLM apps. We realized that the concepts of traces, spans, metrics, and logs that were standardized with OpenTelemetry can easily extend to gen AI. We partnered with 20+ observability platforms to make sure that OpenLLMetry becomes the standard for GenAI observability and that the data that we collect can be sent to other platforms as well.
We plan to extend the metrics we provide to support agents that use tools, vision models, and other amazing developments in our fast-paced industry.
We invite you to give Traceloop a spin and are eager for your feedback! How do you track and debug hallucinations? How much has that been an issue for you? What types of hallucinations have you encountered?
- Oras 2y agoCongratulations on launch. This is a crowded market, and there are many tools doing the same thing. How are you differentiating yourself from other tools like: Langfuse Portkey Keywords ai Promptfoo
- nirga 2y agoThanks! We differentiate in 2 ways: 1. We focus on real-time monitoring. This is where we see the biggest pain with our customers, so we spent a lot of time researching and building the right metrics that can run at scale, fast and at low cost (and you can try them all in our platform). 2. OpenTelemetry - we think this is the best way to observe LLM app. It gives you a better understanding of how other parts of the system are interacting with your LLM. Say you're calling a vector DB, or making an HTTP call - you get them all on the same trace. It's also better for the customers - they're not vendor locked to us and can easily switch to another platform (or even use them in parallel).
- swyx 2y agonot to mention langsmith? braintrust? humanloop? does that count? not sure what else - lets crowdsource a list here so that people can find them in future
- nirga 2y agoI have it internally, I can share it if you want! But to the point of comparison between these and tools like Traceloop - it's interesting to see this space and how each platform takes it's own path and finds its own use cases. LangSmith works well within the LangChain ecosystem together with LangGraph, LangServe. But if you're using LlamaIndex, or even just vanilla OpenAI you'll be spending hours to set up your observability systems. Braintrust and Humanloop (and to some extend other tools I saw in this area) take the path of "full development platform for LLMs". We try to look at it as developers look at tools like Sentry. Continue working in your own IDE with your own tools (wanna manage your prompts in a DB or in git? Wanna use LLMs your own way with no frameworks? no problem). We install in your app, with one line and we work around your existing code base and make monitoring, evaluation and tracing work.