3 ms·
I'm in no way a DBA, but if your main usecase is analytics (OLAP) and updates are infrequent it's common to use column-oriented DBMS. Postgres has cstore_fdw, b
by st1ck 6y ago
I'm in no way a DBA, but if your main usecase is analytics (OLAP) and updates are infrequent it's common to use column-oriented DBMS. Postgres has cstore_fdw, but you can also use others: DuckDB, MonetDB, ClickHouse among FOSS, and quite a few well-known proprietary options.
That said, 2M rows is not really a lot of data (unless the rows are huge). In case you don't need to worry about updates, you can just load everything into memory, e.g. in Pandas dataframe (large overhead, slow, many features) or more efficient implementation, like `datatable` (lower overhead, faster, less features).
Also I recently discovered BI tools (more like realized that despite the name it doesn't have to apply to business data). E.g. Metabase provides nice UI for non-complicated analytical (like SELECT avg(...) GROUP BY ...). So if it fits 90% of your queries, then maybe you got the frontend for free, and only need to work on backend (and the rest 10% of queries).