5 ms·

>SQL is an old language and not designed for large scale programming, it is not a language that developers will love. Even worse, they will not be very producti

>SQL is an old language and not designed for large scale programming, it is not a language that developers will love. Even worse, they will not be very productive with it. What the hell. I stopped reading at this point.

hardware2win 4y ago
Why? I'd say its true
- ansgri 4y ago
  Case in point, the generally-loved C# feature, LINQ, is modeled after SQL. Except for a nonexistent (or at least not standardized) type system, SQL's just fine for data modeling and often analysis.
  - balfirevic 4y ago
    As a data point from a LINQ fan - I love it precisely because it improves on SQL, in terms of composability, regularity and allowing some amount of meta-programming.
tkot 4y ago
It's true though. Relational algebra is a very nice abstraction but SQL as a language is subpar to say the least. SQL is really unergonomic, it's difficult to reuse SQL code which is why there are no SQL libraries worth talking about and it's the reason why people go out of their way to create tools to replace SQL like ORMs or LINQ. SQL is basically a DSL for data processing just like XSLT for example. Would you call XSLT "designed for large scale programming"? The "they will not be very productive with it" comment is a bit of a stretch - it can be very productive in the domain it is designed for - but you can often replace SQL at its job (like for example use Dataframe API in Spark instead of SQL API) while there are many tasks that SQL is a very poor fit for.
- qsort 4y ago
  As much as SQL could be better, it's a very strong local optimum. There is no credible replacement whatsoever for what it does. It's true people try to replace it, but most solutions are of rather questionable quality. I personally argue against ORMs in pretty much any situation. > SQL is basically a DSL for data processing Yeah and what's wrong with that? SQL databases power everything, including data sources with billions of records that serve millions of customers, with hundreds of developers/DBAs working on them. Does that not count as programming in the large?
  - tkot 4y ago
    >As much as SQL could be better, it's a very strong local optimum I mostly agree, though I think that SQL's popularity comes mostly from network effects. > Yeah and what's wrong with that? Nothing is wrong wrong with that. I just think that it's reasonable to interpret the words "programming in the large" as "being usable as general purpose programming language" (which I admit is not a very strict definition) whereas SQL (or XSLT or awk or bash...) seem well suited for certain niches (important niches nonetheless).
  - samatman 4y ago
    > There is no credible replacement whatsoever for what it does. I'm compelled to agree with this under protest. There was a language called Dataphor based on D and the Third Manifesto, it's unfortunately hard to find even sample code any longer, but it would be a strict improvement on SQL. Did I mention you can't even find a corpus? Good luck running any implementation on a modern system. I'm reasonably content writing SQL, but I know a syntax with the same power but lacking several disadvantages is possible, and I'd rather use a mature implementation of that instead, if I were able.
    - qsort 4y ago
      The worst part of SQL is indeed how hard it is to compose. Something like D, where you would keep the relational model but add user-defined types would indeed be a strict improvement. The reason why I'm skeptical of most proposed alternatives is that I'm fundamentally in your exact same position: I know SQL isn't perfect but I'm reasonably content with it; and they invariably all end up throwing away the "good parts" of SQL (relational model, declarative, easy stuff is easy). The first step of an hypothetical solution that replaces SQL isn't "SQL sucks", it's "SQL is extremely good at what it does but has problems that are only fixable with a new language".
ezconnect 4y ago
That's also my experience on current programmers, they don't know how powerful and fast the database system and what it can do and implement the routine in the application level.
xmprt 4y ago
If you've ever tried debugging a broken 100 line SQL statement created by some business analyst for some niche use then you might agree.
- Closi 4y ago
  Have you ever tried debugging a broken 100 line python script created by a business analyst? It's not any better.
  - masklinn 4y ago
    It’s better by orders of magnitude, the python script you can literally trace instruction by instruction, and process every which way you want.
    - actually_a_dog 4y ago
      But that's not a function of the language itself. If there were SQL debuggers you could step through a big query with, debugging SQL would be a lot easier. As it is, you often have to take the whole damn thing apart from the inside to see what's going on, then put it all back together again. That's... suboptimal, to say the least.
      - masklinn 4y ago
        > But that's not a function of the language itself. It is tho, SQL is a declarative language, so the execution model is largely opaque, by language design. But even if it were solely an implementation detail, Closi would still be wrong: they literally stated that “it’s not any better” to try debugging a 100 lines python script than a 100 lines query. > As it is, you often have to take the whole damn thing apart from the inside to see what's going on, then put it all back together again. That's... suboptimal, to say the least. Which is the point? Sounds like you agree that it is significantly easier to debug a 100 lines python script than a 100 lines SQL query.
        4y ago
mieko 4y ago
The SQL language design is more contemporary to COBOL than anything we'd choose to use today. That's not implying that large, successful projects cannot be built upon SQL. We see them in the real world all the time. We have awesome implementations of the language, but any productivity is in spite of a quirky language designed in the early 1970s, not because of it.
- qsort 4y ago
  It's a spurious comparison. I wouldn't pick COBOL as part of my stack for a greenfield project. I would definitely 100% pick SQL as part of my stack for a greenfield project.
  - mieko 4y ago
    Because COBOL has been replaced with better languages for most purposes. SQL is entrenched. I'd also pick SQL for a greenfield project, because... what else? This says nothing about the quality of SQL as a language. What I think is more interesting is: if you were to greenfield a language for a relational database, how much would it look like SQL?
    - qsort 4y ago
      Considering (a) how big and important its domain is; (b) its utter lack of any credible competition whatsoever; (c) the wealth of attempts to displace it, yes, I'd say it says something about the quality of SQL as a language. As a language, i.e. as a vehicle to concisely, effectively, efficiently expressing ideas, it's unquestionably successful. > if you were to greenfield a language for a relational database, how much would it look like SQL? Save for secondary concerns like syntax, it would be exactly the same in all respects except easier composability.
      - wruza 4y ago
        utter lack of any credible competition whatsoever There is no low-level API to SQL servers to create competition, so this is like saying CSS has no competition in browsers whatsoever. Yes, it does not, because it was never allowed. Let's go make a browser competition as part of our weekend, or at least RDBMS, sure /s. ORMs with their "relations" etc are basically better DSLs over SQL databases which try to hide rough edges from an innocent user. concisely, effectively, efficiently expressing ideas, it's unquestionably successful Yeah, "group by case when cond1 then v1 when cond2 then v3 else v3 end" instead of "group by alias1". Because, you know, it's declarative, but not really: https://stackoverflow.com/a/3841804/3125367 https://stackoverflow.com/a/3841804/3125367 Or fetching thousands of rows out of explosion of "select t.a, t.b, p.propname, p.propvalue from items t left join props p on t.id = p.item_id". Or inability to join two different subtables like 'props' and 'per_city_prices' simultaneously without exploding into infinity. Of course you can always make 3 queries and join at your place, which is neither effective nor efficient nor concise. A business wants simple [{a, b, props:[{name, value}, ...], per_city_prices:[{city, price}, ...], ...]. Can SQL do that? It can not. Or getting corresponding non-aggregate values along with min/max results, which requires very funny self-joins if it's already e.g. a windowed aggregation. SQL is cool for these relational... matrices(?), but sucks as a programming language in both syntactic and semantic parts. Some servers fixed few dumb parts of it, but not generally.
jjice 4y ago
I like to think that a few weeks with your head in the trenches of SQL and you come out a competent SQL developer on the other side. I don't mind writing a query, or even a complex query at all. Sometimes it can be draining if you need to nest more queries than you'd like rather than have some joins, but I personally think SQL is a fine language. Could just be Stockholm Syndrome.
dominotw 4y ago
haha my job makes me rewrite sql that Data analysts come up with into spark jobs in scala. Their main argument is that sql isn't unit testable like scala. This grunt work of sql =>spark translation is easily the most boring part of my job. Other than that, I do kind of agree that sql isn't really good at large scale programming. There is no ide support while you are working, hard to extract common logic , hard to reuse , hard to compose, hard to unit test. All these things have been mainstays of conventional programming.

hardware2win 4y ago

Why? I'd say its true

ansgri 4y ago
Case in point, the generally-loved C# feature, LINQ, is modeled after SQL. Except for a nonexistent (or at least not standardized) type system, SQL's just fine for data modeling and often analysis.
- balfirevic 4y ago
  As a data point from a LINQ fan - I love it precisely because it improves on SQL, in terms of composability, regularity and allowing some amount of meta-programming.

tkot 4y ago

It's true though. Relational algebra is a very nice abstraction but SQL as a language is subpar to say the least. SQL is really unergonomic, it's difficult to reuse SQL code which is why there are no SQL libraries worth talking about and it's the reason why people go out of their way to create tools to replace SQL like ORMs or LINQ. SQL is basically a DSL for data processing just like XSLT for example. Would you call XSLT "designed for large scale programming"? The "they will not be very productive with it" comment is a bit of a stretch - it can be very productive in the domain it is designed for - but you can often replace SQL at its job (like for example use Dataframe API in Spark instead of SQL API) while there are many tasks that SQL is a very poor fit for.

qsort 4y ago
As much as SQL could be better, it's a very strong local optimum. There is no credible replacement whatsoever for what it does. It's true people try to replace it, but most solutions are of rather questionable quality. I personally argue against ORMs in pretty much any situation. > SQL is basically a DSL for data processing Yeah and what's wrong with that? SQL databases power everything, including data sources with billions of records that serve millions of customers, with hundreds of developers/DBAs working on them. Does that not count as programming in the large?
- tkot 4y ago
  >As much as SQL could be better, it's a very strong local optimum I mostly agree, though I think that SQL's popularity comes mostly from network effects. > Yeah and what's wrong with that? Nothing is wrong wrong with that. I just think that it's reasonable to interpret the words "programming in the large" as "being usable as general purpose programming language" (which I admit is not a very strict definition) whereas SQL (or XSLT or awk or bash...) seem well suited for certain niches (important niches nonetheless).
- samatman 4y ago
  > There is no credible replacement whatsoever for what it does. I'm compelled to agree with this under protest. There was a language called Dataphor based on D and the Third Manifesto, it's unfortunately hard to find even sample code any longer, but it would be a strict improvement on SQL. Did I mention you can't even find a corpus? Good luck running any implementation on a modern system. I'm reasonably content writing SQL, but I know a syntax with the same power but lacking several disadvantages is possible, and I'd rather use a mature implementation of that instead, if I were able.
  - qsort 4y ago
    The worst part of SQL is indeed how hard it is to compose. Something like D, where you would keep the relational model but add user-defined types would indeed be a strict improvement. The reason why I'm skeptical of most proposed alternatives is that I'm fundamentally in your exact same position: I know SQL isn't perfect but I'm reasonably content with it; and they invariably all end up throwing away the "good parts" of SQL (relational model, declarative, easy stuff is easy). The first step of an hypothetical solution that replaces SQL isn't "SQL sucks", it's "SQL is extremely good at what it does but has problems that are only fixable with a new language".

ezconnect 4y ago

That's also my experience on current programmers, they don't know how powerful and fast the database system and what it can do and implement the routine in the application level.

xmprt 4y ago

If you've ever tried debugging a broken 100 line SQL statement created by some business analyst for some niche use then you might agree.

Closi 4y ago
Have you ever tried debugging a broken 100 line python script created by a business analyst? It's not any better.
- masklinn 4y ago
  It’s better by orders of magnitude, the python script you can literally trace instruction by instruction, and process every which way you want.
  - actually_a_dog 4y ago
    But that's not a function of the language itself. If there were SQL debuggers you could step through a big query with, debugging SQL would be a lot easier. As it is, you often have to take the whole damn thing apart from the inside to see what's going on, then put it all back together again. That's... suboptimal, to say the least.
    - masklinn 4y ago
      > But that's not a function of the language itself. It is tho, SQL is a declarative language, so the execution model is largely opaque, by language design. But even if it were solely an implementation detail, Closi would still be wrong: they literally stated that “it’s not any better” to try debugging a 100 lines python script than a 100 lines query. > As it is, you often have to take the whole damn thing apart from the inside to see what's going on, then put it all back together again. That's... suboptimal, to say the least. Which is the point? Sounds like you agree that it is significantly easier to debug a 100 lines python script than a 100 lines SQL query.
      - 4y ago

mieko 4y ago

The SQL language design is more contemporary to COBOL than anything we'd choose to use today. That's not implying that large, successful projects cannot be built upon SQL. We see them in the real world all the time. We have awesome implementations of the language, but any productivity is in spite of a quirky language designed in the early 1970s, not because of it.

qsort 4y ago
It's a spurious comparison. I wouldn't pick COBOL as part of my stack for a greenfield project. I would definitely 100% pick SQL as part of my stack for a greenfield project.
- mieko 4y ago
  Because COBOL has been replaced with better languages for most purposes. SQL is entrenched. I'd also pick SQL for a greenfield project, because... what else? This says nothing about the quality of SQL as a language. What I think is more interesting is: if you were to greenfield a language for a relational database, how much would it look like SQL?
  - qsort 4y ago
    Considering (a) how big and important its domain is; (b) its utter lack of any credible competition whatsoever; (c) the wealth of attempts to displace it, yes, I'd say it says something about the quality of SQL as a language. As a language, i.e. as a vehicle to concisely, effectively, efficiently expressing ideas, it's unquestionably successful. > if you were to greenfield a language for a relational database, how much would it look like SQL? Save for secondary concerns like syntax, it would be exactly the same in all respects except easier composability.
    - wruza 4y ago
      utter lack of any credible competition whatsoever There is no low-level API to SQL servers to create competition, so this is like saying CSS has no competition in browsers whatsoever. Yes, it does not, because it was never allowed. Let's go make a browser competition as part of our weekend, or at least RDBMS, sure /s. ORMs with their "relations" etc are basically better DSLs over SQL databases which try to hide rough edges from an innocent user. concisely, effectively, efficiently expressing ideas, it's unquestionably successful Yeah, "group by case when cond1 then v1 when cond2 then v3 else v3 end" instead of "group by alias1". Because, you know, it's declarative, but not really: https://stackoverflow.com/a/3841804/3125367 https://stackoverflow.com/a/3841804/3125367 Or fetching thousands of rows out of explosion of "select t.a, t.b, p.propname, p.propvalue from items t left join props p on t.id = p.item_id". Or inability to join two different subtables like 'props' and 'per_city_prices' simultaneously without exploding into infinity. Of course you can always make 3 queries and join at your place, which is neither effective nor efficient nor concise. A business wants simple [{a, b, props:[{name, value}, ...], per_city_prices:[{city, price}, ...], ...]. Can SQL do that? It can not. Or getting corresponding non-aggregate values along with min/max results, which requires very funny self-joins if it's already e.g. a windowed aggregation. SQL is cool for these relational... matrices(?), but sucks as a programming language in both syntactic and semantic parts. Some servers fixed few dumb parts of it, but not generally.

jjice 4y ago

I like to think that a few weeks with your head in the trenches of SQL and you come out a competent SQL developer on the other side. I don't mind writing a query, or even a complex query at all. Sometimes it can be draining if you need to nest more queries than you'd like rather than have some joins, but I personally think SQL is a fine language. Could just be Stockholm Syndrome.

dominotw 4y ago

haha my job makes me rewrite sql that Data analysts come up with into spark jobs in scala. Their main argument is that sql isn't unit testable like scala. This grunt work of sql =>spark translation is easily the most boring part of my job. Other than that, I do kind of agree that sql isn't really good at large scale programming. There is no ide support while you are working, hard to extract common logic , hard to reuse , hard to compose, hard to unit test. All these things have been mainstays of conventional programming.