8 ms·

This blog post is calling at least two different things SQL, and it's kind of infuriating. SQL as a query language is never going away. Virtually every databas

This blog post is calling at least two different things SQL, and it's kind of infuriating. SQL as a query language is never going away. Virtually every database has found it necessary to offer a SQL-like query language: Cassandra's CQL, HiveQL, Couchbase query language, and so on. SQL is a human-readable, composable formalism for describing data. What's gone away is the practice of writing complex, highly linked, normalized database schemas with layers of constraints and foreign key references. That was banished to the land of stagnant enterprise 10 years ago and is not coming back. The last 10-15 years have been an evolution from mostly static, deeply linked, highly structured data to shallow schemas, append-only updates, denormalized data, and stream processing. If you data is a stream of updates, there's not as much pressure to roll back. If your data is mostly defined by a series of processing pipelines that live entirely outside your data warehouse, there's not a lot of upside on enforcing constraints in the DB. If anything, we have learned that it's very useful to offer different denormalized views of the data in the DB to different consumers. MS SQL Server is not roaring back. Businesses have just learned to unbundle data processing from data warehouses. Data warehouses now have fewer tasks to focus on, such as scalability. And if your DB is just a dumb replica with a flat schema, whether it's an RDBMS or not is pretty unimportant.

CharlesW 6y ago
> This blog post is calling at least two different things SQL, and it's kind of infuriating. For better or worse, that's not what "NoSQL" means. I understand that the name is a bit infuriating, and I personally map it to "non-relational" as I read. > What's gone away is the practice of writing complex, highly linked, normalized database schemas with layers of constraints and foreign key references. This may be true for some use cases, but in general that's wishful thinking.
- jolux 6y ago
  There still seems to be a decreasing amount of appreciation for the relational model and the power of SQL databases to maintain data integrity. I'm not quite sure why because relational databases are a uniquely powerful tool in software engineering.
- CaptainZapp 6y ago
  > This may be true for some use cases, but in general that's wishful thinking. Why would avoiding implementation of database level constraints be "wishful thinking"? Au contraire: I strongly believe that data level constraints should be tied to the database. Sure, there's a trade off between indexing, constraints and performance. But I rather have an additional unique index on a column than relying on the fact that all developers in an organization always do the right thing. If the database is my responsibility then I'll make damn sure that it's not possible to fuck it up with shlocky code.
deleted 6y ago
[deleted]
jeffbee 6y ago
SQL should go away, though. It is an astonishingly poor method of communicating your query to the database server. Generating and parsing it is problematic and expensive. It really has nothing going for it, except that a lot of people already know it.
- not_knuth 6y ago
  Interesting. I've never seen SQL as a limiting factor, since 99% of the cases I use I just need to get something by it's id or a simple query (select, join, where, order). And it's really easy to learn too. I've seen many people without coding background pick it up and this is definitely a bonus where I work. Otherwise that 1 CS/Data Science guy is pre-occupied with hammering out queries for everyone. What kind of limitations do you run into? And what alternative do you propose (besides GraphQL)?
  - jakear 6y ago
    Not the parent, but I quite like the Kusto query language. It’s Microsoft specific, but the overall concept is nice and could be implemented more broadly. The operations are described as a pipeline, which to me is much more readable (and writable!) than SQL, where it feels like I’m always bouncing my mental cursor around to figure out what a query is doing. I’m sure that’d reduce with more SQL exposure, but understanding Kusto came pretty much instantly for me. Also, each operation being its own line in the pipeline makes modification an absolute breeze, simply comment out, reorder, etc lines and the result will usually also be a valid query. https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/tutorial?pivots=azuredataexplorer https://docs.microsoft.com/en-us/azure/data-explorer/kusto/q...
    - iudqnolq 6y ago
      Kusto looks very similar to the Ecto DSL for SQL queries (elixir library). I really like the pipeline operation, I wish more languages added it. https://hexdocs.pm/ecto/Ecto.Query.html#module-macro-api https://hexdocs.pm/ecto/Ecto.Query.html#module-macro-api
    - 6y ago
ethbr0 6y ago
The DB-as-cathedral didn't scale as our volume of data did. I don't think NoSQL was as much a conscious choice, as much as the only option when even a medium sized business is unable to afford / scale the number of human DBAs they'd need to keep pace. Everything has a trade-off, and I think we accepted more (accessible, scalable) data >> more (pristine) data. (And obviously, the tooling around newer technologies has gotten way better, while SQL was already very mature)
rufus_foreman 6y ago
>> What's gone away is the practice of writing complex, highly linked, normalized database schemas with layers of constraints and foreign key references That hasn't gone away, that is the default for most corporate CRUD apps, for good reasons. Apps that aren't corporate CRUD apps are a rounding error away from not existing. You use the term data warehouse 3 times, are you sure you are describing a general case rather than one you are familiar with? If you want to offer different views of data in a normalized database to different consumers, one way to do that would be to use views.
smadge 6y ago
> SQL is a ... composable formalism I think SQL is a poor language in terms of composability, in contrast to a functional query language (Frankel and Buneman, 1979) or a relational algebra (Codd, 1970).

CharlesW 6y ago

> This blog post is calling at least two different things SQL, and it's kind of infuriating. For better or worse, that's not what "NoSQL" means. I understand that the name is a bit infuriating, and I personally map it to "non-relational" as I read. > What's gone away is the practice of writing complex, highly linked, normalized database schemas with layers of constraints and foreign key references. This may be true for some use cases, but in general that's wishful thinking.

jolux 6y ago
There still seems to be a decreasing amount of appreciation for the relational model and the power of SQL databases to maintain data integrity. I'm not quite sure why because relational databases are a uniquely powerful tool in software engineering.
CaptainZapp 6y ago
> This may be true for some use cases, but in general that's wishful thinking. Why would avoiding implementation of database level constraints be "wishful thinking"? Au contraire: I strongly believe that data level constraints should be tied to the database. Sure, there's a trade off between indexing, constraints and performance. But I rather have an additional unique index on a column than relying on the fact that all developers in an organization always do the right thing. If the database is my responsibility then I'll make damn sure that it's not possible to fuck it up with shlocky code.

deleted 6y ago

[deleted]

jeffbee 6y ago

SQL should go away, though. It is an astonishingly poor method of communicating your query to the database server. Generating and parsing it is problematic and expensive. It really has nothing going for it, except that a lot of people already know it.

not_knuth 6y ago
Interesting. I've never seen SQL as a limiting factor, since 99% of the cases I use I just need to get something by it's id or a simple query (select, join, where, order). And it's really easy to learn too. I've seen many people without coding background pick it up and this is definitely a bonus where I work. Otherwise that 1 CS/Data Science guy is pre-occupied with hammering out queries for everyone. What kind of limitations do you run into? And what alternative do you propose (besides GraphQL)?
- jakear 6y ago
  Not the parent, but I quite like the Kusto query language. It’s Microsoft specific, but the overall concept is nice and could be implemented more broadly. The operations are described as a pipeline, which to me is much more readable (and writable!) than SQL, where it feels like I’m always bouncing my mental cursor around to figure out what a query is doing. I’m sure that’d reduce with more SQL exposure, but understanding Kusto came pretty much instantly for me. Also, each operation being its own line in the pipeline makes modification an absolute breeze, simply comment out, reorder, etc lines and the result will usually also be a valid query. https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/tutorial?pivots=azuredataexplorer https://docs.microsoft.com/en-us/azure/data-explorer/kusto/q...
  - iudqnolq 6y ago
    Kusto looks very similar to the Ecto DSL for SQL queries (elixir library). I really like the pipeline operation, I wish more languages added it. https://hexdocs.pm/ecto/Ecto.Query.html#module-macro-api https://hexdocs.pm/ecto/Ecto.Query.html#module-macro-api
  - 6y ago

ethbr0 6y ago

The DB-as-cathedral didn't scale as our volume of data did. I don't think NoSQL was as much a conscious choice, as much as the only option when even a medium sized business is unable to afford / scale the number of human DBAs they'd need to keep pace. Everything has a trade-off, and I think we accepted more (accessible, scalable) data >> more (pristine) data. (And obviously, the tooling around newer technologies has gotten way better, while SQL was already very mature)

rufus_foreman 6y ago

>> What's gone away is the practice of writing complex, highly linked, normalized database schemas with layers of constraints and foreign key references That hasn't gone away, that is the default for most corporate CRUD apps, for good reasons. Apps that aren't corporate CRUD apps are a rounding error away from not existing. You use the term data warehouse 3 times, are you sure you are describing a general case rather than one you are familiar with? If you want to offer different views of data in a normalized database to different consumers, one way to do that would be to use views.

smadge 6y ago

> SQL is a ... composable formalism I think SQL is a poor language in terms of composability, in contrast to a functional query language (Frankel and Buneman, 1979) or a relational algebra (Codd, 1970).