3 ms·
Pinot is built to answer OLAP queries at high throughput while maintaining low latency. It powers many customer-facing analytics apps such as LinkedIn's who vie
by kishoreg 6y ago
Pinot is built to answer OLAP queries at high throughput while maintaining low latency. It powers many customer-facing analytics apps such as LinkedIn's who viewed my profile, Publisher Analytics, etc (50+).
At LinkedIn, it serves 100k+ queries per sec with 10-1000 ms latency while ingesting millions of events/sec from Kafka.
This is achieved by various indexing techniques - sorted index, bitmap index, range index, star-tree index, bloom filter, partitioning, etc and a flexible query execution planner that can dynamically pick the right plan based on the query and data profile.
https://www.youtube.com/watch?v=luMLCDANxiU https://www.youtube.com/watch?v=luMLCDANxiU should give you more info on why we built Pinot at LinkedIn.
Disclaimer: pinot committer
- igrekel 6y agoThanks for the link, I'll go through the presentation. We need to upgrade the way we compute indicators and the backend for our analytics and I was considering solutions like Druid and ElasticSearch and Pinot seems like another good option. Getting better latency is really interesting and I'm curious on how much we need to compromise on space usage etc. Another big subject is how it handles time-based data, similar to time series.