3 ms·
A full observability stack is just a docker compose away: Otel + Phoenix + Clickhouse and off to the races. No excuse not to do it.
by CuriouslyC 1y ago
A full observability stack is just a docker compose away: Otel + Phoenix + Clickhouse and off to the races. No excuse not to do it.
- perfmode 1y agoPhoenix as in Elixir?
- mindcrime 1y agoI imagine they meant: https://github.com/Arize-ai/phoenix https://github.com/Arize-ai/phoenix
- pranay01 1y agoone of the cases we have observed is that Phoenix doesn't completely stick to OTel conventions. More specifically, one issue I observed is how it handles span kinds. If you send via OTel, the span Kinds are classified as unknown e.g. The Phoneix screenshot here - https://signoz.io/blog/llm-observability-opentelemetry/#the-standards-problem https://signoz.io/blog/llm-observability-opentelemetry/#the-...
- CuriouslyC 1y agoIf it doesn't work for your use case that's cool, but in terms of interface for doing this kind of work it is the best. Tradeoffs.
- 7thpower 1y agoI’ve found phoenix to be a clunky experience and have been far happier with tools like langfuse. I don’t know how you can confidently say one is “the best”.
- a_khan 1y agoCurious what you prefer from langfuse over Phoenix!
- 7thpower 1y agoSorry for the delayed response! The main thing was wrestling with the instrumentation vs the out of the box langfuse python decorator that works pretty well for basic use cases. It’s been a while but I also recall that prompt management and other features in Phoenix weren’t really built out (probably not a goal for them, but I like having that functionality under the same umbrella).
- ijk 1y agoSpans labeled as 'unknown' when I definitely labeled them in the code is probably the most annoying part of Phoenix right now.
- pranay01 1y agoYes, it is happening because OpenInference assumes these span kind values https://github.com/Arize-ai/openinference/blob/b827f3dd659fc41d0002fc4d5d5f5fff2f8f5857/python/openinference-semantic-conventions/src/openinference/semconv/trace/__init__.py#L446 https://github.com/Arize-ai/openinference/blob/b827f3dd659fc... Anything which doesn't fall in other span kinds is classified as `unknown` For reference, these are span kinds which opentelemetry emits - https://github.com/open-telemetry/opentelemetry-python/blob/6f97ed8482610f70ad6c295d1c14a21af7bc3417/opentelemetry-api/src/opentelemetry/trace/__init__.py#L162 https://github.com/open-telemetry/opentelemetry-python/blob/...
- cephalization 1y agoPhoenix ingests any opentelemetry compliant spans into the platform, but the UI is geared towards displaying spans whose attributes adhere to “openinference” naming conventions. There are numerous open community standards for where to put llm information within otel spans but openinference predates most of em.
- dcreater 1y agoIs phoenix really the no-brainer go to? There are so many choices - langfuse, w&b etc.
- CuriouslyC 1y agoI suppose it depends on the way you approach your work. It's designed with an experimental mindset so it makes it very easy to keep stuff organized, separate, and integrate with the rest of my experimental stack. If you come from an ops background, other tools like SigNoz or LangFuse might feel more natural, I guess it's just a matter of perspective.
- jkisiel 1y agoWorking at a small startup, I evaluated numerous solutions for our LLM observability stack. That was early this year (IIRC Langfuse was not open source then) and Phoenix was the only solution that worked out of the box and seemed to have the right 'mindset', i.e. using Otel and integrating with Python and JS/Langchain. Wasted lots of time with others, some solutions did not even boot.
- dcreater 1y agoThis is exactly what I was looking for! An actual practitioners experience from trials! Thanks. Is it fair to assume you are happy with it?