3 ms·
What is the cost of scaling up number of CPUs for parallel processing? That was always the culprit for me compared to AWS/Athena and BigQuery. They are dirt ch
by fifilura 2y ago
What is the cost of scaling up number of CPUs for parallel processing?
That was always the culprit for me compared to AWS/Athena and BigQuery. They are dirt cheap on analytics workloads when you can parallelize the calculations to 100 CPUs without really paying any extra.
With postgres you are stuck with linear cost for scaling up number of CPUs, so everything is slow anyway.
- zhousun 2y agozhou from mooncake labs here. Good point! Normally for postgres extension it won't be solvable, but for mooncake it is actually not the case! The core idea of mooncake is to built upon open columnar format + substitutable vectorized engine, while natively integrate with Postgres. So right now it is using duckdb within postgres to run the query, but we can and we will support ad-hoc using other 'stateless engines' like Athena, StarRocks or even spark to run a big query.