15 ms·

I’ve been writing some pretty ambitious Hive queries at work lately. As I learn more about SQL and pull off more complex queries, my respect for it deepens. To

I’ve been writing some pretty ambitious Hive queries at work lately. As I learn more about SQL and pull off more complex queries, my respect for it deepens. To have such power and support so many use cases with so few constructs is really an engineering feat. It’s timeless for a reason. Some of my relatively common access patterns are awkward to express, but they can still be expressed in a few lines + a CTE or two, which is really impressive for a language so small. This is not to say we can’t do better. But SQL has achieved a deep resonance with its problem space that most tools don’t even come close to. The brightest minds and most effective tooling shops in our field would be lucky merely to do as well.

osrec 8y ago
It supports so many use cases primarily because of the small number of fundamental "constructs". I mean, if I gave you protons, electrons and neutrons, you could build the universe out of them!
- OJFord 8y ago
  That doesn't make sense, if I give you more things (I'm avoiding saying 'atom', but you know what I mean) then it's not the case that you can suddenly do less. Expressiveness isn't inversely correlated to number of constructs.
  - osrec 8y ago
    Let's say you gave me an atom. In this situation, you can't create other atoms. When you give me protons, electrons and neutrons, I can create any atom I like, in addition to anything else in the universe. Thus, when you give me an atom, you actually reduce the possibilities somewhat. In reality, it's not just about the small number of constructs, but also about how fundamental they are. In one sense, atoms are less fundamental than protons/neutrons/electrons, and therefore reduce the number of possible creations.
    - vageli 8y ago
      > Let's say you gave me an atom. In this situation, you can't create other atoms. When you give me protons, electrons and neutrons, I can create any atom I like, in addition to anything else in the universe. Thus, when you give me an atom, you actually reduce the possibilities somewhat. > In reality, it's not just about the small number of constructs, but also about how fundamental they are. In one sense, atoms are less fundamental than protons/neutrons/electrons, and therefore reduce the number of possible creations. I think what OJFord is getting at is, what if you had more subatomic particles to build with?
      - osrec 8y ago
        The more fundamental you go, the lesser number of things you need to form a "complete set". Thus, whatever the complete set is for subatomic particles, it will most certainly have lesser members than for the complete set of atoms. Getting back to my original point, SQL operates at a fundamental enough level that you do not need a great number of "constructs" to create a query that gives you whatever it is that you need. I'm not saying it's perfect, but was just supplementing the OPs observation that SQL has surprisingly few "constructs".
    - xelxebar 8y ago
      I think your premise might be flawed. Are we just assuming that protons and neutrons automatically pull in the quarks, neutrinos, other leptons and force carriers as well? Does the electron have a photon dependency? What about dark matter? If not, then the our three particles are internally inconsistent and woefully incapable of building a universe. Otherwise, our set of dependencies is essentially the whole darn universe anyway. Same goes for atoms. Give me enough and I can build a star or particle accelerator to create whatever elements or fundamental particles I want.
      - osrec 8y ago
        Wow, it was a simplistic passing metaphor. I was not expecting to get into the complexities of physics, but simply that if you operate at a more fundamental level, you can generally address a greater range of problems.
- closeparen 8y ago
  In your analogy, SQL is the periodic table.
- maxerickson 8y ago
  Depends on whether photons exist or not. Radiation makes the universe a lot more interesting.
asimpletune 8y ago
I think you mean the relational model is timeless. SQL is simply just not a great language, for an otherwise great idea.
- geezerjay 8y ago
  Blindly asserting that SQL is bad without providing any argument or proof does not contradict OP's opinion on SQL.
  - setr 8y ago
    The article itself gives a pretty decent description of how the language falls down.. and the ggp’s expression of his impression pretty trivially just maps to the relational model/algebra.... so all necessary arguments have been made :-)
    - marcus_holmes 8y ago
      really? the article has a handful of strange edge cases that you'll almost never smack into in practice, and that's cause to say the language is crap? Given the vast usage of SQL and its overwhelming popularity, if there were real problems with the language then there would be a lot more noise about them. But most people seem to be happy with it. If it's not broken, let's not fix it, eh?
      - setr 8y ago
        The only edge cases I could find in the article were the discussion of nulls... which aren’t so much an edge case as they are just an ever-present problem that no one really notices until it hurts (its not at all difficult to write a broken-on-null query). Is there something else you’re referring to? Afaict, everything else was just trivially observable aspects of the language >Given the vast usage of SQL and its overwhelming popularity, if there were real problems with the language then there would be a lot more noise about them. But most people seem to be happy with it. In what universe does popularity imply quality? Certainly not the one I’m in — Avengers endgame is apparently #2 gross worldwide ever :-)
daveFNbuck 8y ago
Hive queries are written in HiveQL, not SQL. I used to write a lot of Hive and Impala queries, and going back to plain SQL is disappointing.
- orange8 8y ago
  While based on SQL, HiveQL does not strictly follow the full SQL-92 standard, just like all the other SQL dialects out there. hiveQL is SQL.
  - daveFNbuck 8y ago
    That's a fair argument, but from personal experience HiveQL has some pretty major features added on that make it "have such power and support so many use cases" in a way that other common variants like the one used by MySQL doesn't. A lot of HiveQL's power comes from extending the language, not from the use of a small set of timeless features.
    - orange8 8y ago
      What is your favourite HiveQl extension?
avierax 8y ago
This shows that humans can learn languages and get motivated by mastering them. This does not tell anything about the consistency, composability or orthogonality if SQL. Those qualities affecting the newcomers effort to learn it.
- deleted 8y ago
  [deleted]

osrec 8y ago

It supports so many use cases primarily because of the small number of fundamental "constructs". I mean, if I gave you protons, electrons and neutrons, you could build the universe out of them!

OJFord 8y ago
That doesn't make sense, if I give you more things (I'm avoiding saying 'atom', but you know what I mean) then it's not the case that you can suddenly do less. Expressiveness isn't inversely correlated to number of constructs.
- osrec 8y ago
  Let's say you gave me an atom. In this situation, you can't create other atoms. When you give me protons, electrons and neutrons, I can create any atom I like, in addition to anything else in the universe. Thus, when you give me an atom, you actually reduce the possibilities somewhat. In reality, it's not just about the small number of constructs, but also about how fundamental they are. In one sense, atoms are less fundamental than protons/neutrons/electrons, and therefore reduce the number of possible creations.
  - vageli 8y ago
    > Let's say you gave me an atom. In this situation, you can't create other atoms. When you give me protons, electrons and neutrons, I can create any atom I like, in addition to anything else in the universe. Thus, when you give me an atom, you actually reduce the possibilities somewhat. > In reality, it's not just about the small number of constructs, but also about how fundamental they are. In one sense, atoms are less fundamental than protons/neutrons/electrons, and therefore reduce the number of possible creations. I think what OJFord is getting at is, what if you had more subatomic particles to build with?
    - osrec 8y ago
      The more fundamental you go, the lesser number of things you need to form a "complete set". Thus, whatever the complete set is for subatomic particles, it will most certainly have lesser members than for the complete set of atoms. Getting back to my original point, SQL operates at a fundamental enough level that you do not need a great number of "constructs" to create a query that gives you whatever it is that you need. I'm not saying it's perfect, but was just supplementing the OPs observation that SQL has surprisingly few "constructs".
  - xelxebar 8y ago
    I think your premise might be flawed. Are we just assuming that protons and neutrons automatically pull in the quarks, neutrinos, other leptons and force carriers as well? Does the electron have a photon dependency? What about dark matter? If not, then the our three particles are internally inconsistent and woefully incapable of building a universe. Otherwise, our set of dependencies is essentially the whole darn universe anyway. Same goes for atoms. Give me enough and I can build a star or particle accelerator to create whatever elements or fundamental particles I want.
    - osrec 8y ago
      Wow, it was a simplistic passing metaphor. I was not expecting to get into the complexities of physics, but simply that if you operate at a more fundamental level, you can generally address a greater range of problems.
closeparen 8y ago
In your analogy, SQL is the periodic table.
maxerickson 8y ago
Depends on whether photons exist or not. Radiation makes the universe a lot more interesting.

asimpletune 8y ago

I think you mean the relational model is timeless. SQL is simply just not a great language, for an otherwise great idea.

geezerjay 8y ago
Blindly asserting that SQL is bad without providing any argument or proof does not contradict OP's opinion on SQL.
- setr 8y ago
  The article itself gives a pretty decent description of how the language falls down.. and the ggp’s expression of his impression pretty trivially just maps to the relational model/algebra.... so all necessary arguments have been made :-)
  - marcus_holmes 8y ago
    really? the article has a handful of strange edge cases that you'll almost never smack into in practice, and that's cause to say the language is crap? Given the vast usage of SQL and its overwhelming popularity, if there were real problems with the language then there would be a lot more noise about them. But most people seem to be happy with it. If it's not broken, let's not fix it, eh?
    - setr 8y ago
      The only edge cases I could find in the article were the discussion of nulls... which aren’t so much an edge case as they are just an ever-present problem that no one really notices until it hurts (its not at all difficult to write a broken-on-null query). Is there something else you’re referring to? Afaict, everything else was just trivially observable aspects of the language >Given the vast usage of SQL and its overwhelming popularity, if there were real problems with the language then there would be a lot more noise about them. But most people seem to be happy with it. In what universe does popularity imply quality? Certainly not the one I’m in — Avengers endgame is apparently #2 gross worldwide ever :-)

daveFNbuck 8y ago

Hive queries are written in HiveQL, not SQL. I used to write a lot of Hive and Impala queries, and going back to plain SQL is disappointing.

orange8 8y ago
While based on SQL, HiveQL does not strictly follow the full SQL-92 standard, just like all the other SQL dialects out there. hiveQL is SQL.
- daveFNbuck 8y ago
  That's a fair argument, but from personal experience HiveQL has some pretty major features added on that make it "have such power and support so many use cases" in a way that other common variants like the one used by MySQL doesn't. A lot of HiveQL's power comes from extending the language, not from the use of a small set of timeless features.
  - orange8 8y ago
    What is your favourite HiveQl extension?

avierax 8y ago

This shows that humans can learn languages and get motivated by mastering them. This does not tell anything about the consistency, composability or orthogonality if SQL. Those qualities affecting the newcomers effort to learn it.

deleted 8y ago
[deleted]