3 ms·
+1 on the OTel mention. Having telemetry in place in a system, one would typically implement a single behavioral tracking SDK on top. Adding yet another SDK for
by hrpnk 3y ago
+1 on the OTel mention. Having telemetry in place in a system, one would typically implement a single behavioral tracking SDK on top. Adding yet another SDK for LLMs is a hard ask given how specific the implementation will be. Backing back on a standard you offer value-added insights on top.
On the other hand, if you target just the applications that implement an API behind an LLM, you will have customers expecting value-added services on top of telemetry, like prompt optimization, classification, result caching, etc.
Your choice which direction and target group you will focus on first.