mvijaykarthik

5 ms·

1.
by mvijaykarthik 3mo ago
We store these files on S3 and directly read them in the query path using lambda functions. Parquet would work as a storage format for logs and traces, but not for metrics. At oodle we built the metrics engine first, so we designed a hybrid
2.
by mvijaykarthik 3mo ago
We price per GB of ingested trace - $0.3/GB, goes lower as you scale. Traditional APM spans are ~2KB - this would come to ~$0.6/million spans. How we arrived at $10 is that we assumed each agent span is ~35KB. Which means 1M spans
3.
by mvijaykarthik 3mo ago
Hi HN, Vijay here Adding support for agent traces turned out to be a great learning experience - it pushed us to rethink and replace parts of our storage engine https://blog.oodle.ai/how-we-achieved-10-million-agent-spans...
4.
by mvijaykarthik 10mo ago
For more info and examples - https://blog.oodle.ai/high-performance-in-golang/#memory-all...
5.
Show HN: SafePool – Type-safe object pooling for Go(github.com)
6 points by mvijaykarthik 10mo ago|1 comments
6.
by mvijaykarthik 11mo ago
1. We store high cardinality metrics in object storage and only keep most recent data in memory. We use serverless functions (lambdas) for querying these objects which helps us scale. 2. We do not have a global index - each hour of metrics
7.
by mvijaykarthik 2y ago
Thanks for your feedback, and great questions. 1. We create serverless functions to process each file and then combine the results, optimized for columnar file formats. 2. This is one of our core innovations :) We created custom representat
8.
by mvijaykarthik 2y ago
We are reasonably close to the latest version of Grafana. We periodically pull in new changes.
9.
by mvijaykarthik 2y ago
We are still early in our journey, and are currently working on leveraging LLMs for incidents and query / dashboard generation. We do use pre-LLM-era AI and statistical analysis to provide insights and auto create dashboards for alerts
10.
by mvijaykarthik 2y ago
Oodle can be utilized solely as a datasource, but we also wanted to provide a solution for customers who don’t have a visualization platform in place. Here is the branch we use: https://github.com/oodle-ai/grafana/
11.
by mvijaykarthik 2y ago
Technically you are correct, the scraper will still exist. However, the hard part is scaling the query and storage layers which we replace.
12.
by mvijaykarthik 2y ago
We use a local disk to temporarily stage data before putting it on S3. We have smaller WAL (write ahead log) objects, and a periodic compaction process which creates read-optimized files on S3.
13.
by mvijaykarthik 2y ago
Great question! Vijay here, I'm one of the co-founders of Oodle. Compared to thanos 1. We use object store (S3) for all queries - even recent time ranges. Object store is not just an archival solution 2. Customized indexing to minimize
14.
by mvijaykarthik 2y ago
Good question! Scraping and ingestion continues to use dedicated compute. Serverless helps with query performance. Queries can be parallelized with serverless functions, this especially helps with long range queries. As an added advantage,
15.
by mvijaykarthik 2y ago
About Oodle: Oodle is a fully managed, serverless metrics observability platform. It is a drop-in replacement for Prometheus that delivers fast performance at high scale. Oodle is 3x cheaper than Managed Prometheus and 10x cheaper than othe
16.
Serverless Prometheus Playground: 13M+ time-series/hr and 13B+ samples/day(play.oodle.ai)
11 points by mvijaykarthik 2y ago|4 comments
17.
by mvijaykarthik 8y ago
I've added a section in the README for this. It works in conjunction with NTPD and the time provided by this library has some extra guarantees like monotonicity, immunity to large clock jumps etc (more info in the README).
18.
by mvijaykarthik 8y ago
We recently open-sourced https://github.com/rubrikinc/kronos for the exact same problem. Coincidentally I shared that on Show HN just today: https://news.ycombinator.com/item?id=18037609
19.
by mvijaykarthik 8y ago
Hi HN, we at Rubrik have created this library which we use with CockroachDB to avoid issues with clock skews impacting service availability. Let us know if you have any questions!
20.
Show HN: A clock synchronization library in golang for distributed systems(github.com)
26 points by mvijaykarthik 8y ago|1 comments