We faced a very similar problem when we started Kentik two years ago, except in our case the "transactions" are network traffic telemetry that we collect from o

We faced a very similar problem when we started Kentik two years ago, except in our case the "transactions" are network traffic telemetry that we collect from our customers' physical network infrastructure, and providing super-fast ad hoc queries over that data is our core service offering. We looked at just about every open source and commercial platform that we might use as a backend, and decided that none were appropriate, for scale, maturity, or fairness / scheduling. So we ended up building, from scratch, something that looks a bit like Google's Dremel / BigQuery, but runs on our own bare metal infrastructure. And then we put postgres on top of that using Foreign Data Wrappers so we could write standard SQL queries against it. Some blog posts about the nuts and bolts you might find interesting: https://www.kentik.com/postgresql-foreign-data-wrappers/ https://www.kentik.com/postgresql-foreign-data-wrappers/ https://www.kentik.com/metrics-for-microservices/ https://www.kentik.com/metrics-for-microservices/ If we were starting today, we might consider Apache Drill, although I haven't looked at the maturity and stability of that project recently.