3 ms·
Re: monitoring: I am wondering what a monitoring platform (from an ops perspective) would look like for an infrastructure like this, and what are the challenges
by uaas 3y ago
Re: monitoring: I am wondering what a monitoring platform (from an ops perspective) would look like for an infrastructure like this, and what are the challenges?
- retrovrv 3y agohelping build https://portkey.ai/ https://portkey.ai/ where we try to take care of storing all prompts, responses, erros, costs, latencies etc and make it available to debug.
- uaas 3y agoThanks for the link! I skimmed the articles on the blog, but couldn't find one describing the monitoring architecture behind all. Do you have any public references for that?
- retrovrv 3y agoNot yet, but it's on the roadmap to publish. Broadly, we use clickhouse, streams, mongo, cloudflare workers as the mainstay of our monitoring infra. Some more info here: https://docs.portkey.ai/overview/introduction https://docs.portkey.ai/overview/introduction