3 ms·
I've personally used Postgres with a couple of gigabytes of data without problems and without tuning. Plenty of people seem to be using Postgres for deployments
by wongarsu 7y ago
I've personally used Postgres with a couple of gigabytes of data without problems and without tuning. Plenty of people seem to be using Postgres for deployments in the terabyte range, and apparently some people use it for petabytes of data.
The default settings of postgres are more tuned for OLTP though, while you wanted to do OLAP. At least increasing the per-session memory limit would be advisiable to get anything done in any amount of time (the default is something tiny like 64MB). In general configuring Postgres to use more RAM is important to make things fast.
- riku_iki 7y agoFor me the biggest issue with older versions is that selects run in single thread, and don't scale well on multiple cores, and it makes processing tables with billions rows very slow. Hopefully it will be fixed with new parallelism improvements.
- anarazel 7y agoThat should already have been improved a lot over the last few versions. Which version did you observe this with last, and if it was PG 10 or PG11, what type of query was this?
- riku_iki 7y agoI run PG10 (default on Ubuntu LTS), usually join table with billion rows with table with millions rows with table with thousand rows, and constantly observe situation when just one core is loaded by 100%, and others are idle.
- anarazel 7y agoHm, that should have gotten better with 11 (shared parallel hashjoin), but even before, there's plenty plans where parallelism is possible (by e.g. building the hashtable for a hashjoin over the smaller table(s) in each worker). Any chance parallelism is disabled?
- riku_iki 7y agoThank you for suggestion, I will check config options listed in this post when will have a chance: https://dba.stackexchange.com/questions/226654/how-can-i-disable-parallel-queries-in-postgresql https://dba.stackexchange.com/questions/226654/how-can-i-dis...