15 ms·
> And yet, Postgres foreign data wrappers exist providing encapsulation to the declarative SQL interface. I'm not sure what this means. Do you have a reference
by naasking 8y ago
> And yet, Postgres foreign data wrappers exist providing encapsulation to the declarative SQL interface.
I'm not sure what this means. Do you have a reference?
It seems pretty clear that building an efficient query plan depends on having access to the internals of any entities referenced by the query. Encapsulation hides the internal of entities, so the claim seems pretty solid to me.
- numbsafari 8y agoInteresting. Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? That is, the API allows the encapsulated subsystem to describe its contents in a way that supports query planning (e.g. shape of the contained data, supported operators, and general statistics), without having to reveal its exact implementation (columnar v row store, distributed v centralized storage, exact file format used, etc)? Perhaps the evolving shape of that API as the query planner becomes more capable (and therefore requires more knowledge) is what was meant by “are in tension”. Tension isn’t a bad thing. My favorite art is all about tension. Life is at its most immediate and palpable in the presence of tension.
- naasking 8y ago> Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? Perhaps, but this is still a clear weakening of encapsulation, and it's merely one example where it's most obvious. I think most uses of encapsulation are probably overkill anyway, necessitated only because the language is weak in other areas. For instance, encapsulation is often used to control state changes due to aliasing concerns, but if you're working with an immutable language, state change and aliasing don't conflict. Or you can go the other way, like Rust, and forbid aliasing via ownership and then state change is also easy.
- skybrian 8y agoAbstraction allows us to substitute one thing for another without breaking compatibility. This is at the heart of both API-level compatibility and also most optimizations. If two implementations weren't considered equivalent, we wouldn't be able to do the substitution without breaking things. On the other hand, if the change didn't have some important, user-visible difference (such as performance), there would be no point in doing the substitution. Now consider that most optimizations are enabled by inlining, which means that changing the internal details of a function could break optimizations in calling code and this could cause performance regression. A stricter API could prevent regressions (some bad changes are disallowed) but it also prevents good changes (they break compatibility or are disallowed optimizations). Language designers get to decide what "equivalent code" means using various forms of abstraction, but there is not going to be a perfect answer.
- fnord123 8y ago> I'm not sure what this means. Do you have a reference? https://www.postgresql.org/docs/9.3/postgres-fdw.html https://www.postgresql.org/docs/9.3/postgres-fdw.html https://www.percona.com/blog/2018/08/21/foreign-data-wrappers-postgresql-postgres_fdw/ https://www.percona.com/blog/2018/08/21/foreign-data-wrapper... https://pgxn.org/tag/foreign%20data%20wrapper/ https://pgxn.org/tag/foreign%20data%20wrapper/ >It seems pretty clear that building an efficient query plan depends on having access to the internals of any entities referenced by the query. Most query planners are based on table statistics so what you need to know are the size of the table, and the correlation of the actual data, but they don't need to know whether it's a BTree, LSM, epsilon tree, etc. This can be measured without knowing the underlying implementation. Postgres query planning statistics: https://www.postgresql.org/docs/current/view-pg-stats.html https://www.postgresql.org/docs/current/view-pg-stats.html
- naasking 8y agoFirstly, try writing a query planner when some of the columns being joined are hidden or inaccessible (which is what I described in my post). For instance, suppose it's a dynamically computed property whose computation you don't have access to. That's specifically what the tension between encapsulation and declarative programming means. Secondly, it seems clear that "table statistics" are a weakening of encapsulation properties. Perhaps they are the minimal weakening needed to provide efficient query planning, but it still justifies the claim that encapsulation and declarativity are in tension.
- fnord123 8y ago>it seems clear that "table statistics" are a weakening of encapsulation properties. I don't have the original source of the Ullman comments so I don't know the definitions he's using to make his assertion. So unfortunately I'm speaking a bit out of ignorance about precisely what he meant. I can't really say whether or not table statistics is a weakening of what he intended.