3 ms·
We also developed a product that collects the user event data via SDKs (https://github.com/rakam-io/rakam https://github.com/rakam-io/rakam), depending on the d
by buremba 7y ago
We also developed a product that collects the user event data via SDKs (https://github.com/rakam-io/rakam https://github.com/rakam-io/rakam), depending on the data volume the users pick one of the two deployment types. The first one is based on Postgresql 11 and it takes advantage of partitioned tables (one table for each event time for the configured time period such as month, week, etc.), BRIN indexes, and materialized views.
It's capable of handling 250M events per month without an issue but beyond that we suggest the users to use data-warehouse that supports horizontal scaling and columnar storage engine so we use Snowflake with Snowpipe, Kinesis or Kafka (pluggable) and S3.
I have also tried using Clickhouse which is much cheaper but it lacks tooling for automatic scaling.