3 ms·

From the transcript: > I knew what I wanted my language to talk about, but I had no idea what it looked like, or how you could use it, or what its semantics we

From the transcript: > I knew what I wanted my language to talk about, but I had no idea what it looked like, or how you could use it, or what its semantics were. I had six or seven years to screw around with this stuff, so I did. A lot of time passed. I spent my time on Google docs thinking about what the right state primitive should be. How can we talk about data changing? Whatever language I come up with there should be a way to efficiently evaluate it, that's pretty important if you're a systems person. I screwed around more with syntax. I got really upset about this thing that Jeff Ullman said in one of his books, in which he argued that encapsulation, this thing that you really want, you hide an implementation so that you can abstract above it, and declarativity, allowing a programmer to give a precise description of what should happen in a computation but not necessarily how to do it, appear to be very much at odds with each other. > So, declarativity tends to work really well for very small programs, but when we want to build big programs out of small programs, stuff gets fussy. I'm not going to talk much more about this in this talk, but, this is a problem. So anybody in the audience who is interested in declarative programming, I'd love to hear if you have any ideas about how we can do a better job of hiding encapsulation and reuse in query languages. And then, of course, the number one thing that occupied all my attention and kept me up at night, which was how do we, in a reasonable way, talk about uncertainty? How does a language talk about what it can't talk about? That's weird. I don't know which book this refers to or how Ullma supports his claim. And yet, Postgres foreign data wrappers exist providing encapsulation to the declarative SQL interface. shrug. Maybe Ullman is just wrong - or his comments were from a long time ago and the world has moved on since then.

naasking 8y ago
> And yet, Postgres foreign data wrappers exist providing encapsulation to the declarative SQL interface. I'm not sure what this means. Do you have a reference? It seems pretty clear that building an efficient query plan depends on having access to the internals of any entities referenced by the query. Encapsulation hides the internal of entities, so the claim seems pretty solid to me.
- numbsafari 8y ago
  Interesting. Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? That is, the API allows the encapsulated subsystem to describe its contents in a way that supports query planning (e.g. shape of the contained data, supported operators, and general statistics), without having to reveal its exact implementation (columnar v row store, distributed v centralized storage, exact file format used, etc)? Perhaps the evolving shape of that API as the query planner becomes more capable (and therefore requires more knowledge) is what was meant by “are in tension”. Tension isn’t a bad thing. My favorite art is all about tension. Life is at its most immediate and palpable in the presence of tension.
  - naasking 8y ago
    > Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? Perhaps, but this is still a clear weakening of encapsulation, and it's merely one example where it's most obvious. I think most uses of encapsulation are probably overkill anyway, necessitated only because the language is weak in other areas. For instance, encapsulation is often used to control state changes due to aliasing concerns, but if you're working with an immutable language, state change and aliasing don't conflict. Or you can go the other way, like Rust, and forbid aliasing via ownership and then state change is also easy.
  - skybrian 8y ago
    Abstraction allows us to substitute one thing for another without breaking compatibility. This is at the heart of both API-level compatibility and also most optimizations. If two implementations weren't considered equivalent, we wouldn't be able to do the substitution without breaking things. On the other hand, if the change didn't have some important, user-visible difference (such as performance), there would be no point in doing the substitution. Now consider that most optimizations are enabled by inlining, which means that changing the internal details of a function could break optimizations in calling code and this could cause performance regression. A stricter API could prevent regressions (some bad changes are disallowed) but it also prevents good changes (they break compatibility or are disallowed optimizations). Language designers get to decide what "equivalent code" means using various forms of abstraction, but there is not going to be a perfect answer.
bunderbunder 8y ago
I don't know that he's wrong, but there's a major premise in there that's easy to miss if you aren't watching for it: "Whatever language I come up with there should be a way to efficiently evaluate it, that's pretty important if you're a systems person." It makes me think that he mis-stated his case slightly. In an ideal case, SQL's encapsulation supports its declarativity. In practice, it's riddled with holes. Some of them, like micro-managing the indexing scheme, aren't part of the basic idea behind relational databases (Codd's paper doesn't even mention them, IIRC), but have become so deeply engrained in our expectations of how a relational database should work that we don't even recognize them as a compromise on the idea behind the relational calculus anymore. But they're generally either deeply non-declarative, or lay bare some pretty deep implementation details of the system, or, in the case of index optimization features, both. So, what I'm guessing is really going on here is an incomplete observation that there's a "pick two" situation here: You can only have two of performance, declarativity, encapsulation.

naasking 8y ago

> And yet, Postgres foreign data wrappers exist providing encapsulation to the declarative SQL interface. I'm not sure what this means. Do you have a reference? It seems pretty clear that building an efficient query plan depends on having access to the internals of any entities referenced by the query. Encapsulation hides the internal of entities, so the claim seems pretty solid to me.

numbsafari 8y ago
Interesting. Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? That is, the API allows the encapsulated subsystem to describe its contents in a way that supports query planning (e.g. shape of the contained data, supported operators, and general statistics), without having to reveal its exact implementation (columnar v row store, distributed v centralized storage, exact file format used, etc)? Perhaps the evolving shape of that API as the query planner becomes more capable (and therefore requires more knowledge) is what was meant by “are in tension”. Tension isn’t a bad thing. My favorite art is all about tension. Life is at its most immediate and palpable in the presence of tension.
- naasking 8y ago
  > Can this be addressed by ensuring the API of the encapsulated subsystem is sufficiently rich to provide what is needed by the declarative query planner? Perhaps, but this is still a clear weakening of encapsulation, and it's merely one example where it's most obvious. I think most uses of encapsulation are probably overkill anyway, necessitated only because the language is weak in other areas. For instance, encapsulation is often used to control state changes due to aliasing concerns, but if you're working with an immutable language, state change and aliasing don't conflict. Or you can go the other way, like Rust, and forbid aliasing via ownership and then state change is also easy.
- skybrian 8y ago
  Abstraction allows us to substitute one thing for another without breaking compatibility. This is at the heart of both API-level compatibility and also most optimizations. If two implementations weren't considered equivalent, we wouldn't be able to do the substitution without breaking things. On the other hand, if the change didn't have some important, user-visible difference (such as performance), there would be no point in doing the substitution. Now consider that most optimizations are enabled by inlining, which means that changing the internal details of a function could break optimizations in calling code and this could cause performance regression. A stricter API could prevent regressions (some bad changes are disallowed) but it also prevents good changes (they break compatibility or are disallowed optimizations). Language designers get to decide what "equivalent code" means using various forms of abstraction, but there is not going to be a perfect answer.

bunderbunder 8y ago

I don't know that he's wrong, but there's a major premise in there that's easy to miss if you aren't watching for it: "Whatever language I come up with there should be a way to efficiently evaluate it, that's pretty important if you're a systems person." It makes me think that he mis-stated his case slightly. In an ideal case, SQL's encapsulation supports its declarativity. In practice, it's riddled with holes. Some of them, like micro-managing the indexing scheme, aren't part of the basic idea behind relational databases (Codd's paper doesn't even mention them, IIRC), but have become so deeply engrained in our expectations of how a relational database should work that we don't even recognize them as a compromise on the idea behind the relational calculus anymore. But they're generally either deeply non-declarative, or lay bare some pretty deep implementation details of the system, or, in the case of index optimization features, both. So, what I'm guessing is really going on here is an incomplete observation that there's a "pick two" situation here: You can only have two of performance, declarativity, encapsulation.