5 ms·

While I agree with what you've said, but "skill of SQL" makes me cringe. It sounds like some fundamental skill, like running or swimming. But it's more like "na

by divan 4y ago

While I agree with what you've said, but "skill of SQL" makes me cringe. It sounds like some fundamental skill, like running or swimming. But it's more like "navigating your local govt bureucracy skill" – you generally better off when you have it, but there is nothing to be proud about. SQL as a query language is truly horrible once you get past simple data queries. Most people never question if SQL is even good for its job, or if there is an alternative to the SQL, and blindly work on their SQL skills. It's still better than working on the "MongoDB skills", though. But in 2022 the DB interfaces exploration space is expanding, and there are some good alternatives to SQL to learn too.

ajfriend 4y ago
What are some good alternatives to SQL?
- divan 4y ago
  EdgeQL is surprisingly good in practice
btilly 4y ago
SQL as a query language is truly horrible once you get past simple data queries. Oh, really? I've personally written SQL queries for complex purposes of lengths up to around 1000 lines. I've also had to take what had been SQL queries and rewrite them in a variety of other ways outside of SQL. And vice versa. SQL feels horribly verbose, right until you compare it with equivalent procedural code. I'm confident in my ability to match or beat the SQL optimizer, I know that doing so will take me significant extra work. And I've seen enough code to know that most developers are unable to do so. The only tools that I've seen that come close to SQL are themselves thin wrappers around SQL. And I've yet to see them not lose some important capability that SQL has. For example analytic queries (aka window functions) are not often available in other tools.
- divan 4y ago
  > I've personally written SQL queries for complex purposes of lengths up to around 1000 lines. How does it refute the argument about SQL being horrible for data query? Being proficient in some language or tool doesn't necesarilly mean that thing is not horrible. There are a lot of people proficient in JavaScript, you know. To expand a bit on my definition of horrible, I see a tool as a good choice for its job when two criteria match: 1) tool was deliberatly designed for the job (and not used as an afterthought, as with html/css/js, for example) 2) it matches well the mental model of human thinking in that problem domain. For example, when I need to fetch some nested data from db, I think about my request in terms of relations between objects (like "give me the list of people who are the actors for those movies nominated for Oscars and who are started directors careers recently") instead of JOINS and GROUPS. The fact that I have to reshape my way to think (aka boost my SQL skills) is a testimony to how poorly it scores in my number 2 criteria. Basically whenever you need to reshape your thinking to do some simple stuff with a tool, the tool is horrible. Like describe relatively simple spatial abstract knowledge in terms of class hierarchies (and start thinking in classes, which is not how human brain organizes knowledge). Or build modern UX interfaces thinking in typesetting primitives like "<p>" or "<img 'src='". A lot of popular things are horrible in software engineering unfortunately.
  - btilly 4y ago
    I think you missed the line, "SQL feels horribly verbose, right until you compare it with equivalent procedural code." Having written SQL of varying complexity, and having rewritten it both directions between SQL and procedural, I'm in a good position to compare. And SQL is very good for its purpose. This is why nothing has replaced it.
    - divan 4y ago
      > This is why nothing has replaced it. I don't buy this logic. It's like saying Facebook is a good social network and this is why nothing has replaced it. SQL obviously is an industry standard, the first thing you learn at IT bootcamp or CS classes, the first thing you've being asked at a job as a junior, etc. Decades of experience made most popular SQL engines stupidly effective. Yet, SQL as a query language was designed long time ago, when the data was orders of magnitude smaller and assumptions about data queries needs were way different from today. Let me put it this way – if data query language was designed today from scratch, with all the knowledge and experience we have now - it'll look very different from SQL. I mean, it's just obvious that with more time and real-world practice we get more understanding of how to shape our tools. And then they shape us, of course.
- FreakLegion 4y ago
  Yes, really. I've done (and do) substantial work in SQL as well. Currently almost the entirety of our business logic is implemented in it, both to avoid the delay of round-tripping large data sets and for security reasons. To be clear, this isn't CRUD code. There's a lot of custom ML bits, vector algebra, etc., running in Aurora Postgres where C and Python are off limits. Without PL/pgSQL many of the functions would be impractical. The optimizer isn't the main source of headaches, but it doesn't help. I've caught it doing things it shouldn't be doing -- Aurora's fault, not Postgres' -- and having so much complexity locked away behind a magical "feature" is endlessly frustrating. Not everything is visible in a query plan.
  - btilly 4y ago
    When you're trying to run general purpose code inside of the database, you're beyond what it is good for. I've written a compiler in PL/pgSQL (for a special purpose language for time series data) and agree that there is a limit to what you want to do there. SQL is a query tool. Not a general purpose programming platform. But it is good for surprisingly complex queries on surprisingly large and complex data sets.
    - FreakLegion 4y ago
      We're really just debating the meaning of "simple" from: > SQL as a query language is truly horrible once you get past simple data queries. I think things start to get dicey as soon as you're past basic SELECT, FROM, JOIN, WHERE [1], ORDER BY. CTEs, subqueries, cross joins, case expressions, etc. certainly aren't advanced, but they do surprising things at times, and it can be hard to tell what's going on under the hood with the optimizer. I'd prefer to work with a full-fledged programming language or the underlying primitives the optimizer translates SQL into, but SQL isn't a problem for me personally. The problem is that I've had teammates be utterly defeated by it. 1. And even then you can get in trouble, e.g. not knowing that OR short-circuiting isn't part of the standard, and Postgres for example doesn't do it.
      - btilly 4y ago
kthejoker2 4y ago
As someone who only uses SQL for analytical/OLAP purposes ... I'm cool with people avoiding object relation impedance mismatch if it helps them with app dev. Just remember to stay in your lane.
- btilly 4y ago
  If your need is for CRUD, then using an ORM makes sense. In the case of Python that's easily achieved by using Django. And the skill required in how to use it is directly transferrable to how more complex software is actually written.
Kinrany 4y ago
> But in 2022 the DB interfaces exploration space is expanding, and there are some good alternatives to SQL to learn too. What are your favorite three?

by divan 4y ago

ajfriend 4y ago

What are some good alternatives to SQL?

divan 4y ago
EdgeQL is surprisingly good in practice

btilly 4y ago

SQL as a query language is truly horrible once you get past simple data queries. Oh, really? I've personally written SQL queries for complex purposes of lengths up to around 1000 lines. I've also had to take what had been SQL queries and rewrite them in a variety of other ways outside of SQL. And vice versa. SQL feels horribly verbose, right until you compare it with equivalent procedural code. I'm confident in my ability to match or beat the SQL optimizer, I know that doing so will take me significant extra work. And I've seen enough code to know that most developers are unable to do so. The only tools that I've seen that come close to SQL are themselves thin wrappers around SQL. And I've yet to see them not lose some important capability that SQL has. For example analytic queries (aka window functions) are not often available in other tools.

divan 4y ago
> I've personally written SQL queries for complex purposes of lengths up to around 1000 lines. How does it refute the argument about SQL being horrible for data query? Being proficient in some language or tool doesn't necesarilly mean that thing is not horrible. There are a lot of people proficient in JavaScript, you know. To expand a bit on my definition of horrible, I see a tool as a good choice for its job when two criteria match: 1) tool was deliberatly designed for the job (and not used as an afterthought, as with html/css/js, for example) 2) it matches well the mental model of human thinking in that problem domain. For example, when I need to fetch some nested data from db, I think about my request in terms of relations between objects (like "give me the list of people who are the actors for those movies nominated for Oscars and who are started directors careers recently") instead of JOINS and GROUPS. The fact that I have to reshape my way to think (aka boost my SQL skills) is a testimony to how poorly it scores in my number 2 criteria. Basically whenever you need to reshape your thinking to do some simple stuff with a tool, the tool is horrible. Like describe relatively simple spatial abstract knowledge in terms of class hierarchies (and start thinking in classes, which is not how human brain organizes knowledge). Or build modern UX interfaces thinking in typesetting primitives like "<p>" or "<img 'src='". A lot of popular things are horrible in software engineering unfortunately.
- btilly 4y ago
  I think you missed the line, "SQL feels horribly verbose, right until you compare it with equivalent procedural code." Having written SQL of varying complexity, and having rewritten it both directions between SQL and procedural, I'm in a good position to compare. And SQL is very good for its purpose. This is why nothing has replaced it.
  - divan 4y ago
    > This is why nothing has replaced it. I don't buy this logic. It's like saying Facebook is a good social network and this is why nothing has replaced it. SQL obviously is an industry standard, the first thing you learn at IT bootcamp or CS classes, the first thing you've being asked at a job as a junior, etc. Decades of experience made most popular SQL engines stupidly effective. Yet, SQL as a query language was designed long time ago, when the data was orders of magnitude smaller and assumptions about data queries needs were way different from today. Let me put it this way – if data query language was designed today from scratch, with all the knowledge and experience we have now - it'll look very different from SQL. I mean, it's just obvious that with more time and real-world practice we get more understanding of how to shape our tools. And then they shape us, of course.
FreakLegion 4y ago
Yes, really. I've done (and do) substantial work in SQL as well. Currently almost the entirety of our business logic is implemented in it, both to avoid the delay of round-tripping large data sets and for security reasons. To be clear, this isn't CRUD code. There's a lot of custom ML bits, vector algebra, etc., running in Aurora Postgres where C and Python are off limits. Without PL/pgSQL many of the functions would be impractical. The optimizer isn't the main source of headaches, but it doesn't help. I've caught it doing things it shouldn't be doing -- Aurora's fault, not Postgres' -- and having so much complexity locked away behind a magical "feature" is endlessly frustrating. Not everything is visible in a query plan.
- btilly 4y ago
  When you're trying to run general purpose code inside of the database, you're beyond what it is good for. I've written a compiler in PL/pgSQL (for a special purpose language for time series data) and agree that there is a limit to what you want to do there. SQL is a query tool. Not a general purpose programming platform. But it is good for surprisingly complex queries on surprisingly large and complex data sets.
  - FreakLegion 4y ago
    We're really just debating the meaning of "simple" from: > SQL as a query language is truly horrible once you get past simple data queries. I think things start to get dicey as soon as you're past basic SELECT, FROM, JOIN, WHERE [1], ORDER BY. CTEs, subqueries, cross joins, case expressions, etc. certainly aren't advanced, but they do surprising things at times, and it can be hard to tell what's going on under the hood with the optimizer. I'd prefer to work with a full-fledged programming language or the underlying primitives the optimizer translates SQL into, but SQL isn't a problem for me personally. The problem is that I've had teammates be utterly defeated by it. 1. And even then you can get in trouble, e.g. not knowing that OR short-circuiting isn't part of the standard, and Postgres for example doesn't do it.
    - btilly 4y ago

kthejoker2 4y ago

As someone who only uses SQL for analytical/OLAP purposes ... I'm cool with people avoiding object relation impedance mismatch if it helps them with app dev. Just remember to stay in your lane.

btilly 4y ago
If your need is for CRUD, then using an ORM makes sense. In the case of Python that's easily achieved by using Django. And the skill required in how to use it is directly transferrable to how more complex software is actually written.

Kinrany 4y ago

> But in 2022 the DB interfaces exploration space is expanding, and there are some good alternatives to SQL to learn too. What are your favorite three?