3 ms·

All of those tips and no mention of MATERIALIZED VIEWS[1] it's a pretty wonderful tool to use for data caching (potentially highly safe data if you use on actio

by munk-a 6y ago

All of those tips and no mention of MATERIALIZED VIEWS[1] it's a pretty wonderful tool to use for data caching (potentially highly safe data if you use on action trigger based refreshing) that can allow you to have a nice normalized data layout to write into and a beautiful de-normalized cross-table index supporting place to read from. If you've never heard of mat views please do look them up[2] and play around a bit. 1. As they're called in postgres at least 2. https://www.postgresql.org/docs/current/sql-creatematerializedview.html https://www.postgresql.org/docs/current/sql-creatematerializ...

gerhardi 6y ago
Materialized views are also nice for many modern DW applications where you don't live in legacy "once in a hour/day/week" ELT/ETL environment, but more in a real timeish or micro batch world. You can have data marts / reporting tables composed with these materialized views over many kinds of underlying DW models. Just dropping this to promote the Materialized View functionality :)
- chishaku 6y ago
  dbt (data build tool) is awesome for managing this.
  - aargh_aargh 6y ago
    https://www.getdbt.com/ https://www.getdbt.com/
djrobstep 6y ago
I've found them to be pretty limited as a feature - really they are just some syntactical sugar over "create table as select ... ". If you start to need any kind of flexibility over refreshing the table, you'll need to move to using a plain old table anyway.
- asah 6y ago
  This was my impression too - without incremental update, I don't see the benefit ?
  - gavinray 6y ago
    I'm someone with limited DB wizardry, but a large admiration for Postgres so take this for what it's worth: It's my understanding that there is a proposed feature, with a proof-of-concept floating around, called "Incremental View Maintenance", which is auto-update of dependent Materialized Views when their dependent base-tables change. https://wiki.postgresql.org/wiki/Incremental_View_Maintenance https://wiki.postgresql.org/wiki/Incremental_View_Maintenanc... Materialized view with IVM option created by CRATE INCREMENTAL MATERIALIZED VIEW command. Noted this syntax is just tentative, so it may be changed. When a materialized view is created, AFTER triggers are internally created on its all base tables. When the base tables is modified (INSERT, DELETE, UPDATE), this view is updated incrementally in the trigger function. I think there are some performance/technical things that are getting sorted out with this, but it would be a killer feature. It's the one thing I wish Postgres had that it doesn't. You have to use triggers to do this currently.
    - Normal_gaussian 6y ago
      Yes, this is a killer feature. There are many instances of what are essentially application side caches that try (and often fail) to do this; its non-trivial to do right, and I am greatly looking forwards to it. In particular, an event-sourced world becomes much easier to implement and maintain with the DB handling all the incremental magic.
  - Mouse47 6y ago
    For what it's worth SQL server supports incremental updates for materialized views. I think they're called indexed views, though
    - lazulicurio 6y ago

by munk-a 6y ago

gerhardi 6y ago

Materialized views are also nice for many modern DW applications where you don't live in legacy "once in a hour/day/week" ELT/ETL environment, but more in a real timeish or micro batch world. You can have data marts / reporting tables composed with these materialized views over many kinds of underlying DW models. Just dropping this to promote the Materialized View functionality :)

chishaku 6y ago
dbt (data build tool) is awesome for managing this.
- aargh_aargh 6y ago
  https://www.getdbt.com/ https://www.getdbt.com/

djrobstep 6y ago

I've found them to be pretty limited as a feature - really they are just some syntactical sugar over "create table as select ... ". If you start to need any kind of flexibility over refreshing the table, you'll need to move to using a plain old table anyway.

asah 6y ago
This was my impression too - without incremental update, I don't see the benefit ?
- gavinray 6y ago
  I'm someone with limited DB wizardry, but a large admiration for Postgres so take this for what it's worth: It's my understanding that there is a proposed feature, with a proof-of-concept floating around, called "Incremental View Maintenance", which is auto-update of dependent Materialized Views when their dependent base-tables change. https://wiki.postgresql.org/wiki/Incremental_View_Maintenance https://wiki.postgresql.org/wiki/Incremental_View_Maintenanc... Materialized view with IVM option created by CRATE INCREMENTAL MATERIALIZED VIEW command. Noted this syntax is just tentative, so it may be changed. When a materialized view is created, AFTER triggers are internally created on its all base tables. When the base tables is modified (INSERT, DELETE, UPDATE), this view is updated incrementally in the trigger function. I think there are some performance/technical things that are getting sorted out with this, but it would be a killer feature. It's the one thing I wish Postgres had that it doesn't. You have to use triggers to do this currently.
  - Normal_gaussian 6y ago
    Yes, this is a killer feature. There are many instances of what are essentially application side caches that try (and often fail) to do this; its non-trivial to do right, and I am greatly looking forwards to it. In particular, an event-sourced world becomes much easier to implement and maintain with the DB handling all the incremental magic.
- Mouse47 6y ago
  For what it's worth SQL server supports incremental updates for materialized views. I think they're called indexed views, though
  - lazulicurio 6y ago