3 ms·
Cool! Two questions: 1. Where do you see this observability for LLM thing going? What's the end game? Is it like in traditional observability where all formats
by tomgs 3y ago
Cool! Two questions:
1. Where do you see this observability for LLM thing going? What's the end game? Is it like in traditional observability where all formats eventually will converge to one format (which OpenTelemetry is trying to be)? I feel it might be a little bit early to tell, tho
2. I noticed you do auto-detection of the framework used, like LLamaIndex et al. Except for annotations, is there a deeper connection to the LLM framework used? This is auto-instrumentation, so I assume you do most of the heavy lifting, but should users of this framework expect some cool hidden eggs when they look at their telemetry?
- tomerf2 3y agoThanks! 1. Huh, good question. Hopefully there will be convergence. We started discussing with other companies in this domain to support or even switch to OpenTelemetry. 2. Nothing specific, except for - as you mentioned - being able to see trace of a RAG pipeline automatically.
- tomgs 3y agoA quick one :) While we're on the topic - how does traceloop factor into all of this? What's the connection between the two? I assume the former is the LLM observability platform (Datadog for LLM?) and the latter is your own auto-instrumentation thingie to supplement it?