3 ms·

Agreed that SQL lends itself to spaghetti. However after working with it for 25 years both as a developer but also collaborating with analysts and data scienti

Agreed that SQL lends itself to spaghetti. However after working with it for 25 years both as a developer but also collaborating with analysts and data scientists, I have to say I appreciate its basic declarative nature and remarkable portability as it has expanded into distributed databases. For me the value of being able to walk up to an unfamiliar database of widely varying tech and start getting value out of it immediately is the killer app. Macros or other DB-specific extensions would be useful at times, but I’m not sure how much they would enable solving the messiness problem in a generic way that reshapes the paradigm more broadly. My instinct is the messiness is a consequence of the direct utility and trying to make it nicer from a programmerly aesthetics point of view might be hard to sell. It’s not like SQL doesn’t have affordances for aesthetics already (eg. WITH clauses).

fny 4y ago
SQL is masquerades as declarative. It's relational algebra in disguise. This results in... - Queries that are difficult to interpret/verify. - Non trivial amount of effort to go from question to query: I often feel like I have to jump through hoops to obtain something trivial. I have to think, and I don't want to think.
- dasil003 4y ago
  I agree that SQL is needlessly baroque. Not sure whether I agree that there are too many hoops for trivial things though. Leaving SQL ergonomics aside for the moment, isn't relational algebra something which does require one to think carefully to avoid hastily producing plausible but incorrect queries?
- alliao 4y ago
  PowerBI is definitely down the "don't want to think" alley, lots of misleading graphs are born that way lol
  - spaghettiToy 4y ago
    Disagree. Microsoft sells these as "unskilled", but at the end of the day, you still need skill but also perseverance. At least SQL is old enough to get plenty of relevant search results. You can be a copy-paste DJ if you really can't figure it out.
    - alliao 4y ago
      I agree! the bar's so low that I've seen a lot of people just click around till they get a pretty graph without knowing what/why they did!
- nextos 4y ago
  It's relational algebra, plus relational calculi. > Fundamentally, SQL lacks meaningful abstraction My RDBMS course at college was really hardcore and SQL code answers for exam questions were enormous. One neat trick they didn't teach us, but still perfectly legal SQL:1999, is to use WITH recursive constructs. This way you can decompose very complex queries into simple ones, that can tested independently.
  - grvdrm 4y ago
    The recursive stuff is pretty cool. In general, I like CTEs a way to break queries into pieces to improve readability and understanding (among other improvements). Also, integrating aggregate functions along with row_number() and similar functions to perform more complex work with less code.
- funcDropShadow 4y ago
  > SQL is masquerades as declarative. It's relational algebra in disguise. IMHO declarative and relational algebra are orthogonal concepts. And why do you say SQL is relational algebra in disguise (paraphrased) ? Most introductions to SQL I've read, but certainly the SQL page on Wikipedia [1], starts with a remark about Codd's relational model. Granted SQL does not implement it perfectly, and it is not perfectly declarative, although that is a matter of taste. > Non trivial amount of effort to go from question to query: I often feel like I have to jump through hoops to obtain something trivial. I've had colleagues which expressed similar sentiments, but I have the opposite impression. I think I can express many queries on a much higher abstraction level than with most other languages. > I have to think, and I don't want to think. Not wanting to think while writing code is an understandable wish, but also a big red flag about developers for me. Even the best AI with perfect mind reading capabilities will not be able to execute the query you want, if you don't know what you want yourself. Even the AI would guess sometimes right, you would have no way of knowing whether it was right or wrong. [1]: https://en.wikipedia.org/wiki/SQL https://en.wikipedia.org/wiki/SQL
  - fny 4y ago
    > Not wanting to think while writing code is an understandable wish, but also a big red flag about developers for me. Dude... these are simple queries. I have *far* more important things to ponder than the specific incantations needed to get one of the values that correspond to a maximum.
    - trimethylpurine 4y ago
      max() is an iso standard function. But you need to think relationally to express exactly which maximum you're looking for, that's true. I can see wanting the language to do that for you, but that would be extremely limiting, since every problem seeking the "maximum" isn't remotely similar. Imagine a request for fewer tools in a toolbox. It might take time to learn where they are located, but they are all in the box for a reason. There's a certain point in learning SQL where you stop thinking about your work in rows and columns and you start thinking in tables. When that happens, you'll easily write good code with very little thinking. Give yourself time. Also look into dynamic SQL if you're concerned with repetition. There's no reason SQL should be overly so.
- usehackernews 4y ago
  I want to train a language model (or use existing ones) to translate question to query for trivial questions. I find 80% of queries that people want across a company are trivial. I’d have saved myself many many hours over the years if people could just ask a question in natural language rather than asking me to write a query.
systemvoltage 4y ago
I think if your SQL becomes spaghetti, it is time to fetch the data out of the database and then process it. SQL databases aren't a data-science studio that lends itself for extreme processing capabilities just because it is fast. Furthermore, it is the last thing that scales typically. Relational processing is what its good at. Doing things like bayesian/statistical analysis on the data in SQL is asking for trouble.
- setr 4y ago
  Relational databases are perfect for data analysis scenarios, especially when it fits in a single server. Why are you pumping data over the network, and serializing/deserializing it from a table structure usually back into a table structure, when you could just process it , often using only the data in the datastore, and pipe the final computed results over? Hell they even have a name for databases specialized for this usecase — OLAP The problem is that databases, and SQL, are such a pain to extend and define sufficiently general operations that you don’t just have scipy in the database, and thus your complicated analytics have to be fully specified with the few rudimentary analytic functions that are actually provided to you.
  - ttfkam 4y ago
    https://madlib.apache.org/ https://madlib.apache.org/ https://docs.timescale.com/timescaledb/latest/how-to-guides/hyperfunctions/function-pipelines/ https://docs.timescale.com/timescaledb/latest/how-to-guides/... https://duckdb.org/2022/05/04/friendlier-sql.html https://duckdb.org/2022/05/04/friendlier-sql.html It's a shame that so many people regard SQL using the reference SQL92 spec when in fact that it is continually evolving and improving like any other language still in use.

fny 4y ago

SQL is masquerades as declarative. It's relational algebra in disguise. This results in... - Queries that are difficult to interpret/verify. - Non trivial amount of effort to go from question to query: I often feel like I have to jump through hoops to obtain something trivial. I have to think, and I don't want to think.

dasil003 4y ago
I agree that SQL is needlessly baroque. Not sure whether I agree that there are too many hoops for trivial things though. Leaving SQL ergonomics aside for the moment, isn't relational algebra something which does require one to think carefully to avoid hastily producing plausible but incorrect queries?
alliao 4y ago
PowerBI is definitely down the "don't want to think" alley, lots of misleading graphs are born that way lol
- spaghettiToy 4y ago
  Disagree. Microsoft sells these as "unskilled", but at the end of the day, you still need skill but also perseverance. At least SQL is old enough to get plenty of relevant search results. You can be a copy-paste DJ if you really can't figure it out.
  - alliao 4y ago
    I agree! the bar's so low that I've seen a lot of people just click around till they get a pretty graph without knowing what/why they did!
nextos 4y ago
It's relational algebra, plus relational calculi. > Fundamentally, SQL lacks meaningful abstraction My RDBMS course at college was really hardcore and SQL code answers for exam questions were enormous. One neat trick they didn't teach us, but still perfectly legal SQL:1999, is to use WITH recursive constructs. This way you can decompose very complex queries into simple ones, that can tested independently.
- grvdrm 4y ago
  The recursive stuff is pretty cool. In general, I like CTEs a way to break queries into pieces to improve readability and understanding (among other improvements). Also, integrating aggregate functions along with row_number() and similar functions to perform more complex work with less code.
funcDropShadow 4y ago
> SQL is masquerades as declarative. It's relational algebra in disguise. IMHO declarative and relational algebra are orthogonal concepts. And why do you say SQL is relational algebra in disguise (paraphrased) ? Most introductions to SQL I've read, but certainly the SQL page on Wikipedia [1], starts with a remark about Codd's relational model. Granted SQL does not implement it perfectly, and it is not perfectly declarative, although that is a matter of taste. > Non trivial amount of effort to go from question to query: I often feel like I have to jump through hoops to obtain something trivial. I've had colleagues which expressed similar sentiments, but I have the opposite impression. I think I can express many queries on a much higher abstraction level than with most other languages. > I have to think, and I don't want to think. Not wanting to think while writing code is an understandable wish, but also a big red flag about developers for me. Even the best AI with perfect mind reading capabilities will not be able to execute the query you want, if you don't know what you want yourself. Even the AI would guess sometimes right, you would have no way of knowing whether it was right or wrong. [1]: https://en.wikipedia.org/wiki/SQL https://en.wikipedia.org/wiki/SQL
- fny 4y ago
  > Not wanting to think while writing code is an understandable wish, but also a big red flag about developers for me. Dude... these are simple queries. I have *far* more important things to ponder than the specific incantations needed to get one of the values that correspond to a maximum.
  - trimethylpurine 4y ago
    max() is an iso standard function. But you need to think relationally to express exactly which maximum you're looking for, that's true. I can see wanting the language to do that for you, but that would be extremely limiting, since every problem seeking the "maximum" isn't remotely similar. Imagine a request for fewer tools in a toolbox. It might take time to learn where they are located, but they are all in the box for a reason. There's a certain point in learning SQL where you stop thinking about your work in rows and columns and you start thinking in tables. When that happens, you'll easily write good code with very little thinking. Give yourself time. Also look into dynamic SQL if you're concerned with repetition. There's no reason SQL should be overly so.
usehackernews 4y ago
I want to train a language model (or use existing ones) to translate question to query for trivial questions. I find 80% of queries that people want across a company are trivial. I’d have saved myself many many hours over the years if people could just ask a question in natural language rather than asking me to write a query.

systemvoltage 4y ago

I think if your SQL becomes spaghetti, it is time to fetch the data out of the database and then process it. SQL databases aren't a data-science studio that lends itself for extreme processing capabilities just because it is fast. Furthermore, it is the last thing that scales typically. Relational processing is what its good at. Doing things like bayesian/statistical analysis on the data in SQL is asking for trouble.

setr 4y ago
Relational databases are perfect for data analysis scenarios, especially when it fits in a single server. Why are you pumping data over the network, and serializing/deserializing it from a table structure usually back into a table structure, when you could just process it , often using only the data in the datastore, and pipe the final computed results over? Hell they even have a name for databases specialized for this usecase — OLAP The problem is that databases, and SQL, are such a pain to extend and define sufficiently general operations that you don’t just have scipy in the database, and thus your complicated analytics have to be fully specified with the few rudimentary analytic functions that are actually provided to you.
- ttfkam 4y ago
  https://madlib.apache.org/ https://madlib.apache.org/ https://docs.timescale.com/timescaledb/latest/how-to-guides/hyperfunctions/function-pipelines/ https://docs.timescale.com/timescaledb/latest/how-to-guides/... https://duckdb.org/2022/05/04/friendlier-sql.html https://duckdb.org/2022/05/04/friendlier-sql.html It's a shame that so many people regard SQL using the reference SQL92 spec when in fact that it is continually evolving and improving like any other language still in use.