2 ms·
We offer Redshift and raw SQL access as an add-on to the analytics, it's not used to power the dashboards at all. We've actually developed an in house backend
by sskates 11y ago
We offer Redshift and raw SQL access as an add-on to the analytics, it's not used to power the dashboards at all.
We've actually developed an in house backend from scratch that's gone through ~10 or so iterations. There are 3 main data stores:
1) We have a real-time in memory data store (similar to Redis) for recent data that aggregates data in various ways to display on the dashboard 2) We have a batch service backed by Amazon S3 that for data older than 24 hours that also aggregates data 3) We have a column store for more complex queries that can't be represented by the aggregated data
On top of all that we have a distributed query engine that access all the data stores and queries the appropriate ones in parallel at query time on the dashboards.
We'll have a blog post that has more details about our stack in the next week or so.