5 ms·

There is an interaction here between the "what gets measured gets managed" principle and the sunk cost fallacy. The problem with databases is actually complexi

by roenxi 3y ago

There is an interaction here between the "what gets measured gets managed" principle and the sunk cost fallacy. The problem with databases is actually complexity. Any individual feature is more or less safe, but around the time reliability, caching and indexes get matched together there is a complexity explosion and it doesn't (normally, anyhow) make sense to implement a domain-specific DB (call is a DSD?). But, around the time a company has invested in implementing those 3 features and discovered that it has sunk a lot of resources into the DSD, is politically averse to recommending it be stripped out and there is a high real cost to taking out the tech debt in one go. Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own. The NoSQL databases were a good step in that direction, but then they by and large overfocused on big data instead of everyday usefulness. Things like Redis took hold which is nice. Making it easy to run SQL is a reasonable approach, but the problem is that the good databases - I like postgres - are SQL native and it is hard to get efficiency without speaking the DB's language. We really need a PostgresPostSQL database that is a perfect postgres clone but primary parser supports a language with good syntax.

totalhack 3y ago
Have you ever looked at adding a semantic layer on top of a db for those that prefer to avoid direct SQL?
- lmm 3y ago
  You'd need databases that were open to splitting out and layering like that, and I don't think any major SQL databases are. (I mean, most SQL databases are still written in C-family languages; splitting out the PostgreSQL parser into its own library took 20 years and still hasn't really been done properly, they do some weird sync thing from the mainline postgresql codebase rather than being the actual trunk for parser development and having postgresql proper use it as a dependency). Maybe you could do something with H2?
holoduke 3y ago
What is exactly hard about sql? Every dev imho should know it. And sql syntax is good and proven too be long lasting. Maybe investing some time in actually learning it instead of bashing it will help you further.
- lmm 3y ago
  > What is exactly hard about sql? - No non-nullable types (at the expression level). No way to express e.g. normal boolean logic - No real data structures (at the expression level), unless you count rows, which are not first-class values. Even collections aren't first-class - Very awkward control flow constructs. E.g. look at how you write recursive queries. Even if/else is weird. It's a classical Turing Tarpit: everything is possible but nothing is easy - Absurdly complex grammar. Something like 200+ keywords. Bizarre and inconsistent quoting rules - Weird rules about where expressions can be used, and where you have to put variables. Often if you want to extract out a common subexpression and reuse it you have to completely restructure your statement. - Missing basic functionality, e.g. no string concatenation operator (except in some nonstandard dialects) - No libraries. No standard way of reusing code. No unit testing framework, which means in practice most projects have no testing. It's a terrible language by any reasonable standard. Its closest cousin is COBOL, and that's honestly less awful to write.
  - holoduke 3y ago
    If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose. With under the hood a crafty machine that enables atomic principles and depending on the sql tech other different functionalities like scalability and more.. No Libraries? There are a zillion battle tested libs out there. Some are used by companies like Paypal, SpaceX, Tiktok, youtube, Gmail and many more. And why you would unit test datasets in a database? You normally use mocks to test against your data layer. Most if not all software stacks have mock libs available.
    - lmm 3y ago
GuB-42 3y ago
SQL often gets criticized, and I think for good reasons, but why didn't we come up with something better? We have dozens of programming languages in use for general programming, it is a field that is constantly evolving. Even JS, which is hard to change because that's what browsers run and you don't control your client's browser is seeing some evolution, using transpilers, and now WebAssembly. But for databases, there is only one, and that SQL. There are some alternatives, but no one comes close to SQL in terms of usage. So maybe SQL is not that bad after all. Maybe the reason is that the relational model is really good, and attempts to deviate from this will only work in a niche. That a declarative style is also really good, and again, you won't have much success if you deviate from this. And if you end up doing SQL with a different syntax, for most people, it will be too small of an improvement to justify changing their way.
runeks 3y ago
> Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own. I wish Postgres had a stable, more low-level API than SQL. Perhaps something that resembles the query plan I get from EXPLAIN. An application written against this API could implement an SQL DB: (1) parse SQL (2) implement a query planner that spits out a query plan written against this API.

by roenxi 3y ago

totalhack 3y ago

Have you ever looked at adding a semantic layer on top of a db for those that prefer to avoid direct SQL?

lmm 3y ago
You'd need databases that were open to splitting out and layering like that, and I don't think any major SQL databases are. (I mean, most SQL databases are still written in C-family languages; splitting out the PostgreSQL parser into its own library took 20 years and still hasn't really been done properly, they do some weird sync thing from the mainline postgresql codebase rather than being the actual trunk for parser development and having postgresql proper use it as a dependency). Maybe you could do something with H2?

holoduke 3y ago

What is exactly hard about sql? Every dev imho should know it. And sql syntax is good and proven too be long lasting. Maybe investing some time in actually learning it instead of bashing it will help you further.

lmm 3y ago
> What is exactly hard about sql? - No non-nullable types (at the expression level). No way to express e.g. normal boolean logic - No real data structures (at the expression level), unless you count rows, which are not first-class values. Even collections aren't first-class - Very awkward control flow constructs. E.g. look at how you write recursive queries. Even if/else is weird. It's a classical Turing Tarpit: everything is possible but nothing is easy - Absurdly complex grammar. Something like 200+ keywords. Bizarre and inconsistent quoting rules - Weird rules about where expressions can be used, and where you have to put variables. Often if you want to extract out a common subexpression and reuse it you have to completely restructure your statement. - Missing basic functionality, e.g. no string concatenation operator (except in some nonstandard dialects) - No libraries. No standard way of reusing code. No unit testing framework, which means in practice most projects have no testing. It's a terrible language by any reasonable standard. Its closest cousin is COBOL, and that's honestly less awful to write.
- holoduke 3y ago
  If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose. With under the hood a crafty machine that enables atomic principles and depending on the sql tech other different functionalities like scalability and more.. No Libraries? There are a zillion battle tested libs out there. Some are used by companies like Paypal, SpaceX, Tiktok, youtube, Gmail and many more. And why you would unit test datasets in a database? You normally use mocks to test against your data layer. Most if not all software stacks have mock libs available.
  - lmm 3y ago

GuB-42 3y ago

SQL often gets criticized, and I think for good reasons, but why didn't we come up with something better? We have dozens of programming languages in use for general programming, it is a field that is constantly evolving. Even JS, which is hard to change because that's what browsers run and you don't control your client's browser is seeing some evolution, using transpilers, and now WebAssembly. But for databases, there is only one, and that SQL. There are some alternatives, but no one comes close to SQL in terms of usage. So maybe SQL is not that bad after all. Maybe the reason is that the relational model is really good, and attempts to deviate from this will only work in a niche. That a declarative style is also really good, and again, you won't have much success if you deviate from this. And if you end up doing SQL with a different syntax, for most people, it will be too small of an improvement to justify changing their way.

runeks 3y ago

> Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own. I wish Postgres had a stable, more low-level API than SQL. Perhaps something that resembles the query plan I get from EXPLAIN. An application written against this API could implement an SQL DB: (1) parse SQL (2) implement a query planner that spits out a query plan written against this API.