3 ms·
Show HN: Llm.report – Open-source logging and analytics for OpenAI
Hi hacker news,
My name is Dillion and I'm the creator of llm.report.
A few months ago, I was frustrated by the lack of observability into the OpenAI API.
All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product.
So I ended up just building it myself.
There are three parts to the platform:
1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API itself
2. Logging - Log all your requests / responses to OpenAI to analyze cost, latency, token usage and more
3. User Metrics - Calculate your cost per user to figure out who your power users are, and how much they cost you per day
Would appreciate any feedback you have! Would you use something like this? Why or why not?
- iamjackg 3y agoVery cool project! I'm mainly drawn to this because it seems easier to set up for self-hosting than Helicone[0], which is doing the exact same thing. [0]: https://github.com/Helicone/helicone https://github.com/Helicone/helicone EDIT: I take it back :) The self-hosted instructions don't really work: they assume you have node, yarn, and I guess prisma already installed?
- dillionverma 3y agothanks! Ah good catch! I'll update the docs for that :). The project is still really early so thanks for bearing with me. I'll probably add a single docker compose file to get up and running easily.
- Ccasados 3y agoAwesome stuff Dillion! I can attest that llm.report is super performant and quality built. Thanks for making/sharing this!