5 ms·

I've always said SQL is a bad abstraction. An artifact from the past that we have to deal with like css and JavaScript. This article solidifies my point. Serio

I've always said SQL is a bad abstraction. An artifact from the past that we have to deal with like css and JavaScript. This article solidifies my point. Seriously, ors are bad so union two queries as a cheaper alternative? Theres no rhyme or reason to it at the API level. You have to look under the hood to see why this hack works. Hence why SQL is such a bad abstraction. We program and optimize sql not with logic but with memorization of hacks.

ben509 7y ago
> Seriously, ors are bad so union two queries as a cheaper alternative? They should be equivalent, letting you choose the notation that expresses the problem cleanly. If relations were sets of tuples, boolean expressions could be rewritten safely as straightforward set arithmetic and vice versa. But SQL relations (tables) are bags of tuples (rows), which breaks that equivalence.
- crimsonalucard 7y ago
  That's why the abstraction is bad. The abstraction is not zero cost. Instead a high level API should be exposed that mirrors the underlying implementation with a bijection. Databases are the bottleneck of most web development and we need to optimize "bags of tuples" not "sets" in order to gain the performance we need. The problem is, the API (SQL) exposes things as if the underlying implementation was sets. Hence the need to memorize ugly hacks like this in order to optimize SQL. SQL is a language designed for business users, not a language designed to optimize data retrieval speed. We need an API that can do the latter because web devs are doing all these optimizations in the business language. I'm not saying there's a better solution out there. I'm exposing a problem. The problem is, SQL is not a zero cost abstraction. We need a "Rust" for databases. A high level query language where the optimization paths are isomorphic to the optimizations paths in the underlying implementation. Maybe at first glance you don't agree with me. But think of it this way. If an optimization path in Postgresql involves rewriting a Query with an OR clause as a UNION of two queries and all SQL implementations are loaded with strange hacks like this... is there a problem? and what is the solution?
cabaalis 7y ago
I think the main issue is that to replace SQL, you'll need to build a database which you then need some method of interfacing with. You could expose the underlying structures and operations with some API of course, but so much work has gone into optimizing the high-level CRUD/join operations that you'll force your users to reinvent the wheel.
- crimsonalucard 7y ago
  You're not wrong. Just note that in almost every other area of the stack language and technology transitions occur all the time. SQL is one of the few parts of the stack that has remained steadfast. Many argue for good reason, I would agree, but I would also agree that there is much room for improvement. What's strange to me is why developers welcome change anywhere else on the stack but vehemently dislike suggestions for improvement in a relational database api.
EpicEng 7y ago
>Theres no rhyme or reason to it at the API level. You have to look under the hood... Every abstraction is leaky. I have found that the best engineers are those who understand how things work under the hood. They can solve hard problems when they come up. People who rely upon a mountain of libraries are stumped and begin trying random things until something works. Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way.
- crimsonalucard 7y ago
  >Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way. There is actually a huge problem with this. Human intelligence is limited and therefore much of the art and science of computer programming is targeted towards managing complexity. The fundamental concept that humans use to reduce complexity in everything from math, physics, language, chemistry and everything out there is abstraction. When look at a wall you are looking at an abstraction. To truly understand the nature of the wall you must understand that it is composed of plant cells, (wood) which is composed of carbon based molecules which at is center is all serialized as a single hereditary molecule called DNA which is itself composed of atoms that are governed by complex differential equations and on and on and on... You don't need to know any of this to understand the wall because of abstraction. You encapsulate all that complexity into a concept called a wall. Programming is largely the same thing, with abstraction made explicit. To move forward, to produce things of greater complexity we have to move in the direction of good, non-leaky abstractions. Limitations in our intelligence will inevitably force us to become ignorant of the underlying implementations of many technologies so the abstractions around these things must be excellent. It is the only way forward and it is also arguably the reason why humans are different from other animals. We have the ability to abstract. This concept is everywhere in computer science. The web programmer doesn't need to understand assembly language even though it's the underlying implementation of all his tools. But you are right in the sense that it doesn't doesn't mean a programmer can't learn assembly. But this still illustrates my point. HUmans have a limited ability in handling complexity and therefore we must use abstraction to handle it. In fact, there is an area of computer science where this has become startlingly obvious. It involves machine learning. Take a visualization algorithm that can recognize objects. Currently the de facto way of doing this training a black box of neural networks. The complexity is so high that we have no idea how to program such algorithms by hand and the only way forward is to rely on these black box abstractions where we "train" the system. Even if the programmer pries open the neural network and tries to understand what each neuron is doing in the context of other neurons is impossible. In short. To build systems of greater complexity developers must use tools to abstract complexity. SQL has a different if not tangential problem. It's not leaking enough complexity. It's hiding too much, and producing an interface that is expensive in terms of performance. I'm proposing an abstraction that is one to one. Where if I learn the language and I understand how to optimize the language then a bijection allows me to optimize the underlying implementation as well without understanding it. To borrow the term from Rust and C++: Zero Cost abstractions.

ben509 7y ago

> Seriously, ors are bad so union two queries as a cheaper alternative? They should be equivalent, letting you choose the notation that expresses the problem cleanly. If relations were sets of tuples, boolean expressions could be rewritten safely as straightforward set arithmetic and vice versa. But SQL relations (tables) are bags of tuples (rows), which breaks that equivalence.

crimsonalucard 7y ago
That's why the abstraction is bad. The abstraction is not zero cost. Instead a high level API should be exposed that mirrors the underlying implementation with a bijection. Databases are the bottleneck of most web development and we need to optimize "bags of tuples" not "sets" in order to gain the performance we need. The problem is, the API (SQL) exposes things as if the underlying implementation was sets. Hence the need to memorize ugly hacks like this in order to optimize SQL. SQL is a language designed for business users, not a language designed to optimize data retrieval speed. We need an API that can do the latter because web devs are doing all these optimizations in the business language. I'm not saying there's a better solution out there. I'm exposing a problem. The problem is, SQL is not a zero cost abstraction. We need a "Rust" for databases. A high level query language where the optimization paths are isomorphic to the optimizations paths in the underlying implementation. Maybe at first glance you don't agree with me. But think of it this way. If an optimization path in Postgresql involves rewriting a Query with an OR clause as a UNION of two queries and all SQL implementations are loaded with strange hacks like this... is there a problem? and what is the solution?

cabaalis 7y ago

I think the main issue is that to replace SQL, you'll need to build a database which you then need some method of interfacing with. You could expose the underlying structures and operations with some API of course, but so much work has gone into optimizing the high-level CRUD/join operations that you'll force your users to reinvent the wheel.

crimsonalucard 7y ago
You're not wrong. Just note that in almost every other area of the stack language and technology transitions occur all the time. SQL is one of the few parts of the stack that has remained steadfast. Many argue for good reason, I would agree, but I would also agree that there is much room for improvement. What's strange to me is why developers welcome change anywhere else on the stack but vehemently dislike suggestions for improvement in a relational database api.

EpicEng 7y ago

>Theres no rhyme or reason to it at the API level. You have to look under the hood... Every abstraction is leaky. I have found that the best engineers are those who understand how things work under the hood. They can solve hard problems when they come up. People who rely upon a mountain of libraries are stumped and begin trying random things until something works. Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way.

crimsonalucard 7y ago
>Honestly, what is the problem with understanding how your tools work? I don't know of any other engineering profession that thinks this way. There is actually a huge problem with this. Human intelligence is limited and therefore much of the art and science of computer programming is targeted towards managing complexity. The fundamental concept that humans use to reduce complexity in everything from math, physics, language, chemistry and everything out there is abstraction. When look at a wall you are looking at an abstraction. To truly understand the nature of the wall you must understand that it is composed of plant cells, (wood) which is composed of carbon based molecules which at is center is all serialized as a single hereditary molecule called DNA which is itself composed of atoms that are governed by complex differential equations and on and on and on... You don't need to know any of this to understand the wall because of abstraction. You encapsulate all that complexity into a concept called a wall. Programming is largely the same thing, with abstraction made explicit. To move forward, to produce things of greater complexity we have to move in the direction of good, non-leaky abstractions. Limitations in our intelligence will inevitably force us to become ignorant of the underlying implementations of many technologies so the abstractions around these things must be excellent. It is the only way forward and it is also arguably the reason why humans are different from other animals. We have the ability to abstract. This concept is everywhere in computer science. The web programmer doesn't need to understand assembly language even though it's the underlying implementation of all his tools. But you are right in the sense that it doesn't doesn't mean a programmer can't learn assembly. But this still illustrates my point. HUmans have a limited ability in handling complexity and therefore we must use abstraction to handle it. In fact, there is an area of computer science where this has become startlingly obvious. It involves machine learning. Take a visualization algorithm that can recognize objects. Currently the de facto way of doing this training a black box of neural networks. The complexity is so high that we have no idea how to program such algorithms by hand and the only way forward is to rely on these black box abstractions where we "train" the system. Even if the programmer pries open the neural network and tries to understand what each neuron is doing in the context of other neurons is impossible. In short. To build systems of greater complexity developers must use tools to abstract complexity. SQL has a different if not tangential problem. It's not leaking enough complexity. It's hiding too much, and producing an interface that is expensive in terms of performance. I'm proposing an abstraction that is one to one. Where if I learn the language and I understand how to optimize the language then a bijection allows me to optimize the underlying implementation as well without understanding it. To borrow the term from Rust and C++: Zero Cost abstractions.