8 ms·

I really don't like Oracle from a DBA perspective but it's still often far ahead of PostgreSQL when it comes to query performance. In Postgres, the query struct

by dx034 6y ago

I really don't like Oracle from a DBA perspective but it's still often far ahead of PostgreSQL when it comes to query performance. In Postgres, the query structure can make huge differences in terms of performance and it can take a lot of tuning to find the right query to optimize performance (especially when subqueries are involved). Oracle (and SQLServer) are usually pretty good at optimizing the query exactly the right way, reducing development time by quite a bit.

rpedela 6y ago
PG community has put a lot effort into performance the last few years, including JIT compilation in PG12. Is that criticism still true today?
- asah 6y ago
  I'd really like to know this as well !!
- ibejoeb 6y ago
  One of the big changes in 12 allows the optimizer to work properly with CTEs, which was a major barrier to using the more expressive language features. There a good writeup here: https://paquier.xyz/postgresql-2/postgres-12-with-materialize/ https://paquier.xyz/postgresql-2/postgres-12-with-materializ...
  - doublesCs 6y ago
    Thanks for that link. > Historically we've always materialized the full output of a CTE query Do you know if this means that CTEs are written to disk? I didn't think this was the case. (In the case of materialized views that qualifier means the view is written to disk).
    - dtheodor 6y ago
      It's in-memory materialization, which will spill to disk if it doesn't fit in memory. Search for "Materialize node" in https://www.postgresql.org/docs/12/using-explain.html https://www.postgresql.org/docs/12/using-explain.html
    - deleted 6y ago
      [deleted]
    - da_chicken 6y ago
      Here "materialization" just means that the query planner treats it as an optimization fence. That means the system doesn't do any optimization between the CTE and the query referencing it. It would prepare the output of the CTE as a completely separate entity essentially as if you had dumped it to a temp table. It may not be written to disk if there was sufficient memory, but either way you're sacrificing any optimization. I believe that it would even materialize the CTE multiple times if it was referenced multiple times, but don't quote me on that. On the other hand, if the same query were written with subqueries instead of with CTEs, then the query optimizer would not treat it as an optimization fence. If it could utilize indexes or rewrite the query to be relationally equivalent, it would do so. Note that sometimes that optimization fence is beneficial. There are situations where it's better to create temp tables and run smaller simpler queries instead of running an extremely complex monolithic query because the query planner isn't perfect even with hints. You can still enable that optimization fence functionality in PostgreSQL if you need to, but it's generally pretty rare that it happens like this. Still, you'll see stored procedures for reports still using temp tables and even cursors sometimes because they can be made to perform better in certain situations.
      - ibejoeb 6y ago
        A simple example would be something like predicate pushdown: with mycte as ( select a, count(b) from foo group by a ) select * from mycte where a = 10; There is enough information in the statement to plan it as with mycte as ( select a, count(b) from foo where a = 10; group by a ) select * from mycte; Before, Postgres would have computed the aggregate for all values of `a`.
  - mrits 6y ago
    I left postgres v9 and have been working on Vertica. It's pretty surprising that Vertica came out with that WITH/WITHOUT MATERIALIZATION keywords 2 years ago.
- __s 6y ago
  & more's coming. PG13 has improved hash aggregation[0] & List (which is used throughout) has been updated from being a linked list to being an array 0: https://www.postgresql.org/message-id/507ac540ec7c20136364b5272acbcd4574aa76ef.camel%40j-davis.com https://www.postgresql.org/message-id/507ac540ec7c20136364b5...
- dtheodor 6y ago
  It's very true today. The problem discussed here is performance on complex queries (e.g. subqueries), and the query planner plays a huge role in that. The Postgres query planner has various issues. Here are two recent posts talking about planner issues: https://medium.com/@rbranson/10-things-i-hate-about-postgresql-20dbab8c2791 https://medium.com/@rbranson/10-things-i-hate-about-postgres... https://www.cybertec-postgresql.com/en/things-could-be-improved-postgresql/ https://www.cybertec-postgresql.com/en/things-could-be-impro... Also JIT compilation, while very nice and a step in the right direction, is very barebones at the moment, hardly achieving its potential. Here's a long todo list of what and how to make efficient use of JIT in postgres, by the main author of the feature. https://twitter.com/AndresFreundTec/status/1002589969616199680 https://twitter.com/AndresFreundTec/status/10025899696161996... Postgres release 12 did not add any JIT related improvements, and as far as I know no work has been done on it on 13 either.
  - ksec 6y ago
    For someone not following closely, What are the reasons behind the lack of work on JIT since then?
    - tibiahurried 6y ago
      > The problem discussed here is performance on complex queries This is the problem. In my experience, these complex queries exist because of either unnecessary super normalized schema or one schema to fit them all. Nowadays storage is so cheap that you can safely de-normalize the schema and have tables that fit your query model. If you have very complex queries chances are that you must take a closer look at your application design.
    - ris 6y ago
      The traditional view that the bottleneck for query performance is overwhelmingly disk throughput, which has become slightly less true with the advent of SSDs.
  - thda 6y ago
    > as far as I know no work has been done on it on 13 either. You should check the latest commitfest. Improvements are coming, at a steady pace.
- unclebucknasty 6y ago
  I would say performance is still an issue in certain cases. We had some long-running queries with views and subqueries running on a down level version of PG. We recently attempted an upgrade to 12 to address this and take advantage of the latest RDS generations. The ideal architectural solution would be to sunset the views (involving unions) in favor of a better normalized database, but you do what you can when you inherit legacy. And we're not talking about millions of rows here. It's a relatively small installment. Anyway, to our surprise, not only did it provide little boost, but other queries that were simpler and previously caused no issues became inexplicably and intolerably slow. We did not have time to go down the rabbit hole of query planning and explains, and being a highly dynamically queriable app, we didn't know what else was in store for other queries based on some combination of user inputs down the road. So we rolled back. PG has otherwise been great for us. It really is the performance issue in certain situations that has caused pain. One more note is with regard to what I would call extreme performance variability based on row counts. What I mean is there seems to be some threshold that, once crossed, causes some queries to go from perfectly fine to near-interminable. You expect some degradation as row counts go up, but here the performance behavior suddenly degrades in a nonlinear fashion. That kind of issue is difficult to tune.
jcadam 6y ago
Specifically for geospatial queries we found Oracle Spatial to be inferior to PostGIS (which is much more widely used). Regular relational queries seemed to be no worse (admittedly, our needs in a database were somewhat modest in that regard). Though, most of our team's expertise as far as databases went was in Postgres (including our DBA).
Pxtl 6y ago
I've generally found SqlServer's query optimizer to be a hellish nightmare of broken dreams, so if Postgres' is even worse I'm giving up and going back to flat files.
- carlosf 6y ago
  You might even call it a Data Lake and get away with it.
- da_chicken 6y ago
  I've found that SQL Server's is generally pretty good, with the big pitfall being the system's timeout for the query planner/compiler. Query compilation timeouts can be really frustrating to work on because, often, the query's complexity is a requirement. The only other problem is the parameter sniffing problem for stored procedures, although OPTIMIZE FOR UNKNOWN or specified values seem to work fairly well in my experience, though obviously not always. The real failing is that a common solution to a view query hitting the compiler timeout is to replace it with a stored procedure of some kind. However, if you're not careful you'll run into the parameter sniffing problem with stored procedures! So you run into one caveat and your attempted solution runs into the other one.
- sjwright 6y ago
  I found SQL Server's query optimiser to be magical, but it relies on its table statistics being somewhat correct. Every now and again they liked to suddenly become wrong enough that queries go from magical to catastrophic mush. (Most recent version I've used was SQL Server 2008.)
  - ants_a 6y ago
    All cost based optimisers rely on statistics being correct. And inevitably there will be a case where they aren't. The problem with PostgreSQLs optimiser, and I assume with others too, is that it's too risk happy. It's optimizing for the average case based on the statistics, when people actually tend to care about the worst case. As an example, say you have a database of all cars ever produced with indexes on model and production date. If you are looking for the latest Ford F-150 then the best plan is to just start looking backwards by date and you will find one soon enough. Much faster than looking up all F-150s and picking the latest one. On the other hand, if you are looking for latest Ford Model-T, that plan is going to be catastrophically terrible, going through 93 years of car production before finding the correct one.
ashtonkem 6y ago
This is actually my experience with Oracle; I had a DBA to take the management pain from us, and we had the budget for the correct licenses and hardware. The query performance was phenomenal considering the absolutely crazy amount of data we threw at that thing. That being said, I would not recommend Oracle without all of the above factors already in place. For most use cases, Oracle is more pain than it’s worth.
- AmericanChopper 6y ago
  As somebody who does a fair amount of DBA work, Oracle is my favorite RDBMS to operate. Low level administration is easy to manage, the optimizer is very performant, and the plan management features make tuning easy, and statistics management much less risky (and it can be very risky). I wouldn't use it on any of my own projects though, but only because I wouldn't want to pay for it, and because I have enough faith in myself to be able to manage the trickier bits of Postgres.
  - ashtonkem 6y ago
    To be fair, I have no idea how hard it was to manage Oracle vs. how much my DBA just liked to gripe. But I largely agree with you, except I would lean towards a managed service because I’m no DBA.
zip1234 6y ago
Does PostgreSQL have bitmap indexes. Very much a great feature that Oracle has for query performance
- fanf2 6y ago
  Looks like PostgreSQL only has ephemeral bitmap indexes used when making a query that combines multiple indexes - https://leopard.in.ua/2015/04/13/postgresql-indexes https://leopard.in.ua/2015/04/13/postgresql-indexes - https://www.postgresql.org/docs/current/indexes-bitmap-scans.html https://www.postgresql.org/docs/current/indexes-bitmap-scans...
- atombender 6y ago
  No. Postgres has in-memory bitmap indexes, which are built on the fly while scanning the index and used to more efficiently combine AND/OR clauses, but that's not quite the same thing. There have been several attempts at adding on-disk bitmap index support to Postgres, but they've all been abandoned.
  - anarazel 6y ago
    BRIN indexes are kind of bitmap like...
lukeschlather 6y ago
It's very convenient for Oracle that anyone making these sorts of claims is contractually barred from sharing any evidence.

by dx034 6y ago

rpedela 6y ago

PG community has put a lot effort into performance the last few years, including JIT compilation in PG12. Is that criticism still true today?

asah 6y ago
I'd really like to know this as well !!
ibejoeb 6y ago
One of the big changes in 12 allows the optimizer to work properly with CTEs, which was a major barrier to using the more expressive language features. There a good writeup here: https://paquier.xyz/postgresql-2/postgres-12-with-materialize/ https://paquier.xyz/postgresql-2/postgres-12-with-materializ...
- doublesCs 6y ago
  Thanks for that link. > Historically we've always materialized the full output of a CTE query Do you know if this means that CTEs are written to disk? I didn't think this was the case. (In the case of materialized views that qualifier means the view is written to disk).
  - dtheodor 6y ago
    It's in-memory materialization, which will spill to disk if it doesn't fit in memory. Search for "Materialize node" in https://www.postgresql.org/docs/12/using-explain.html https://www.postgresql.org/docs/12/using-explain.html
  - deleted 6y ago
    [deleted]
  - da_chicken 6y ago
    Here "materialization" just means that the query planner treats it as an optimization fence. That means the system doesn't do any optimization between the CTE and the query referencing it. It would prepare the output of the CTE as a completely separate entity essentially as if you had dumped it to a temp table. It may not be written to disk if there was sufficient memory, but either way you're sacrificing any optimization. I believe that it would even materialize the CTE multiple times if it was referenced multiple times, but don't quote me on that. On the other hand, if the same query were written with subqueries instead of with CTEs, then the query optimizer would not treat it as an optimization fence. If it could utilize indexes or rewrite the query to be relationally equivalent, it would do so. Note that sometimes that optimization fence is beneficial. There are situations where it's better to create temp tables and run smaller simpler queries instead of running an extremely complex monolithic query because the query planner isn't perfect even with hints. You can still enable that optimization fence functionality in PostgreSQL if you need to, but it's generally pretty rare that it happens like this. Still, you'll see stored procedures for reports still using temp tables and even cursors sometimes because they can be made to perform better in certain situations.
    - ibejoeb 6y ago
      A simple example would be something like predicate pushdown: with mycte as ( select a, count(b) from foo group by a ) select * from mycte where a = 10; There is enough information in the statement to plan it as with mycte as ( select a, count(b) from foo where a = 10; group by a ) select * from mycte; Before, Postgres would have computed the aggregate for all values of `a`.
- mrits 6y ago
  I left postgres v9 and have been working on Vertica. It's pretty surprising that Vertica came out with that WITH/WITHOUT MATERIALIZATION keywords 2 years ago.
__s 6y ago
& more's coming. PG13 has improved hash aggregation[0] & List (which is used throughout) has been updated from being a linked list to being an array 0: https://www.postgresql.org/message-id/507ac540ec7c20136364b5272acbcd4574aa76ef.camel%40j-davis.com https://www.postgresql.org/message-id/507ac540ec7c20136364b5...
dtheodor 6y ago
It's very true today. The problem discussed here is performance on complex queries (e.g. subqueries), and the query planner plays a huge role in that. The Postgres query planner has various issues. Here are two recent posts talking about planner issues: https://medium.com/@rbranson/10-things-i-hate-about-postgresql-20dbab8c2791 https://medium.com/@rbranson/10-things-i-hate-about-postgres... https://www.cybertec-postgresql.com/en/things-could-be-improved-postgresql/ https://www.cybertec-postgresql.com/en/things-could-be-impro... Also JIT compilation, while very nice and a step in the right direction, is very barebones at the moment, hardly achieving its potential. Here's a long todo list of what and how to make efficient use of JIT in postgres, by the main author of the feature. https://twitter.com/AndresFreundTec/status/1002589969616199680 https://twitter.com/AndresFreundTec/status/10025899696161996... Postgres release 12 did not add any JIT related improvements, and as far as I know no work has been done on it on 13 either.
- ksec 6y ago
  For someone not following closely, What are the reasons behind the lack of work on JIT since then?
  - tibiahurried 6y ago
    > The problem discussed here is performance on complex queries This is the problem. In my experience, these complex queries exist because of either unnecessary super normalized schema or one schema to fit them all. Nowadays storage is so cheap that you can safely de-normalize the schema and have tables that fit your query model. If you have very complex queries chances are that you must take a closer look at your application design.
  - ris 6y ago
    The traditional view that the bottleneck for query performance is overwhelmingly disk throughput, which has become slightly less true with the advent of SSDs.
- thda 6y ago
  > as far as I know no work has been done on it on 13 either. You should check the latest commitfest. Improvements are coming, at a steady pace.
unclebucknasty 6y ago
I would say performance is still an issue in certain cases. We had some long-running queries with views and subqueries running on a down level version of PG. We recently attempted an upgrade to 12 to address this and take advantage of the latest RDS generations. The ideal architectural solution would be to sunset the views (involving unions) in favor of a better normalized database, but you do what you can when you inherit legacy. And we're not talking about millions of rows here. It's a relatively small installment. Anyway, to our surprise, not only did it provide little boost, but other queries that were simpler and previously caused no issues became inexplicably and intolerably slow. We did not have time to go down the rabbit hole of query planning and explains, and being a highly dynamically queriable app, we didn't know what else was in store for other queries based on some combination of user inputs down the road. So we rolled back. PG has otherwise been great for us. It really is the performance issue in certain situations that has caused pain. One more note is with regard to what I would call extreme performance variability based on row counts. What I mean is there seems to be some threshold that, once crossed, causes some queries to go from perfectly fine to near-interminable. You expect some degradation as row counts go up, but here the performance behavior suddenly degrades in a nonlinear fashion. That kind of issue is difficult to tune.

jcadam 6y ago

Specifically for geospatial queries we found Oracle Spatial to be inferior to PostGIS (which is much more widely used). Regular relational queries seemed to be no worse (admittedly, our needs in a database were somewhat modest in that regard). Though, most of our team's expertise as far as databases went was in Postgres (including our DBA).

Pxtl 6y ago

I've generally found SqlServer's query optimizer to be a hellish nightmare of broken dreams, so if Postgres' is even worse I'm giving up and going back to flat files.

carlosf 6y ago
You might even call it a Data Lake and get away with it.
da_chicken 6y ago
I've found that SQL Server's is generally pretty good, with the big pitfall being the system's timeout for the query planner/compiler. Query compilation timeouts can be really frustrating to work on because, often, the query's complexity is a requirement. The only other problem is the parameter sniffing problem for stored procedures, although OPTIMIZE FOR UNKNOWN or specified values seem to work fairly well in my experience, though obviously not always. The real failing is that a common solution to a view query hitting the compiler timeout is to replace it with a stored procedure of some kind. However, if you're not careful you'll run into the parameter sniffing problem with stored procedures! So you run into one caveat and your attempted solution runs into the other one.
sjwright 6y ago
I found SQL Server's query optimiser to be magical, but it relies on its table statistics being somewhat correct. Every now and again they liked to suddenly become wrong enough that queries go from magical to catastrophic mush. (Most recent version I've used was SQL Server 2008.)
- ants_a 6y ago
  All cost based optimisers rely on statistics being correct. And inevitably there will be a case where they aren't. The problem with PostgreSQLs optimiser, and I assume with others too, is that it's too risk happy. It's optimizing for the average case based on the statistics, when people actually tend to care about the worst case. As an example, say you have a database of all cars ever produced with indexes on model and production date. If you are looking for the latest Ford F-150 then the best plan is to just start looking backwards by date and you will find one soon enough. Much faster than looking up all F-150s and picking the latest one. On the other hand, if you are looking for latest Ford Model-T, that plan is going to be catastrophically terrible, going through 93 years of car production before finding the correct one.

ashtonkem 6y ago

This is actually my experience with Oracle; I had a DBA to take the management pain from us, and we had the budget for the correct licenses and hardware. The query performance was phenomenal considering the absolutely crazy amount of data we threw at that thing. That being said, I would not recommend Oracle without all of the above factors already in place. For most use cases, Oracle is more pain than it’s worth.

AmericanChopper 6y ago
As somebody who does a fair amount of DBA work, Oracle is my favorite RDBMS to operate. Low level administration is easy to manage, the optimizer is very performant, and the plan management features make tuning easy, and statistics management much less risky (and it can be very risky). I wouldn't use it on any of my own projects though, but only because I wouldn't want to pay for it, and because I have enough faith in myself to be able to manage the trickier bits of Postgres.
- ashtonkem 6y ago
  To be fair, I have no idea how hard it was to manage Oracle vs. how much my DBA just liked to gripe. But I largely agree with you, except I would lean towards a managed service because I’m no DBA.

zip1234 6y ago

Does PostgreSQL have bitmap indexes. Very much a great feature that Oracle has for query performance

fanf2 6y ago
Looks like PostgreSQL only has ephemeral bitmap indexes used when making a query that combines multiple indexes - https://leopard.in.ua/2015/04/13/postgresql-indexes https://leopard.in.ua/2015/04/13/postgresql-indexes - https://www.postgresql.org/docs/current/indexes-bitmap-scans.html https://www.postgresql.org/docs/current/indexes-bitmap-scans...
atombender 6y ago
No. Postgres has in-memory bitmap indexes, which are built on the fly while scanning the index and used to more efficiently combine AND/OR clauses, but that's not quite the same thing. There have been several attempts at adding on-disk bitmap index support to Postgres, but they've all been abandoned.
- anarazel 6y ago
  BRIN indexes are kind of bitmap like...

lukeschlather 6y ago

It's very convenient for Oracle that anyone making these sorts of claims is contractually barred from sharing any evidence.