5 ms·

How would abstraction magically materialize a data-layout optimizer into being? The interface of Postgres is in fact abstract enough to do such an optimization

How would abstraction magically materialize a data-layout optimizer into being? The interface of Postgres is in fact abstract enough to do such an optimization (perhaps there would be a problem with "SELECT * FROM table" queries? not sure if in that case the order of columns is specified by the standard), but they just haven't. Optimizations don't happen simply because they are allowed by an interface.

londons_explore 4y ago
A data-layout-optimizer could be as simple as 25 lines of C++. It's fairly trivial to just insert any other field where there would otherwise be padding. It hasn't been written because it would be a breaking change for the whole ecosystem of postgres addons, things that consume postgres WAL logs, etc.
- dymk 4y ago
  That's fine for new tables, but the abstraction now leaks when adding new columns. Now your abstraction has broken again, as existing tables with new columns added are performing poorly compared to new tables, even though from the abstraction level of the user, they _should_ be the same thing.
  - deleted 4y ago
    [deleted]
  - ghusbands 4y ago
    The abstraction is already heavily leaking. It's trivial to patch this claimed abstraction leakage and it won't be worse than the way PostgreSQL already often exclusive-locks whole tables when you add a column and can only statically know the position of columns appearing before a variable-sized one.
    - valenterry 4y ago
      Either every abstraction is leaky (so we can just replace "leaky abstraction" with "abstraction") - or only some abstraction is leaky, then please explain where the current abstraction gives any guarantees of performance that are violated.
      - ghusbands 4y ago
        I didn't introduce the claim of leaky abstractions. The context is that the 'new' abstraction is claimed to be leaky by not providing particular performance guarantees. I pointed out that the 'old' one is just as (or more) leaky by that (or most any) definition. If you want me to cite some guarantee for the old one, you should first cite the guarantee for the new one. What people have issue with is the effect column ordering has on the performance of fetching a fixed-size column from a loaded row and on the size of a row. Those are not necessary features and a better design at the start would have avoided it.
        valenterry 4y ago
        > I didn't introduce the claim of leaky abstractions But you used the term as well - I disagree with both you and the OP on if this case should be called "leaky abstraction". > The context is that the 'new' abstraction is claimed to be leaky by not providing particular performance guarantees That's not a useful definition of "leaky abstraction" by any means. Who decides if "particular performance guarantees" have been provided or not?
        ghusbands 4y ago
        I agree that it's not a good term. Also, you're confusing your posters - "abstraction now leaks" was introduced by someone opposing the optimization (dymk), not the OP (londons_explore, who used "insufficient abstraction"). And when I used it I intentionally paraquoted as "claimed abstraction leakage", due to the leaky terminology.
        valenterry 4y ago

londons_explore 4y ago

A data-layout-optimizer could be as simple as 25 lines of C++. It's fairly trivial to just insert any other field where there would otherwise be padding. It hasn't been written because it would be a breaking change for the whole ecosystem of postgres addons, things that consume postgres WAL logs, etc.

dymk 4y ago
That's fine for new tables, but the abstraction now leaks when adding new columns. Now your abstraction has broken again, as existing tables with new columns added are performing poorly compared to new tables, even though from the abstraction level of the user, they _should_ be the same thing.
- deleted 4y ago
  [deleted]
- ghusbands 4y ago
  The abstraction is already heavily leaking. It's trivial to patch this claimed abstraction leakage and it won't be worse than the way PostgreSQL already often exclusive-locks whole tables when you add a column and can only statically know the position of columns appearing before a variable-sized one.
  - valenterry 4y ago
    Either every abstraction is leaky (so we can just replace "leaky abstraction" with "abstraction") - or only some abstraction is leaky, then please explain where the current abstraction gives any guarantees of performance that are violated.
    - ghusbands 4y ago
      I didn't introduce the claim of leaky abstractions. The context is that the 'new' abstraction is claimed to be leaky by not providing particular performance guarantees. I pointed out that the 'old' one is just as (or more) leaky by that (or most any) definition. If you want me to cite some guarantee for the old one, you should first cite the guarantee for the new one. What people have issue with is the effect column ordering has on the performance of fetching a fixed-size column from a loaded row and on the size of a row. Those are not necessary features and a better design at the start would have avoided it.
      - valenterry 4y ago
        > I didn't introduce the claim of leaky abstractions But you used the term as well - I disagree with both you and the OP on if this case should be called "leaky abstraction". > The context is that the 'new' abstraction is claimed to be leaky by not providing particular performance guarantees That's not a useful definition of "leaky abstraction" by any means. Who decides if "particular performance guarantees" have been provided or not?
        ghusbands 4y ago
        I agree that it's not a good term. Also, you're confusing your posters - "abstraction now leaks" was introduced by someone opposing the optimization (dymk), not the OP (londons_explore, who used "insufficient abstraction"). And when I used it I intentionally paraquoted as "claimed abstraction leakage", due to the leaky terminology.
        valenterry 4y ago