3 ms·

I think this is a poor understanding and laziness. SQL is type and schema first and people hate that because it makes things hard and complicated up front. Tabl

I think this is a poor understanding and laziness. SQL is type and schema first and people hate that because it makes things hard and complicated up front. Table explosions are rare if you know what you are doing. Many people don't any more. As for join heavy, complexity, this is not necessarily a problem in reality. It's incredibly easy to scale this out to huge systems (relatively) cheaply. Believe me as they scale up, they look way less hard and less complicated than arbitrary and poorly enforced schemas in document databases. I could write an essay on how to fuck up MongoDB (or any document store) because I spent nearly 2 years unfucking one.

josephg 2y ago
I’d love to read that essay. But to me sql looks like crap json - it’s like json except if (for purely historical reasons) we didn’t have objects or lists except at the top level of a json blob. Claiming it’s a moral failing on my part for wanting nice, intuitive data structures is a bit rich. Go on, next why don’t you tell me that json is bad and we should use xml. Or c is lazy and we should use assembler. Without more detail, that’s all I hear - moral justifications for a dated design. And the design is dated. As far as I can tell, the reason for the current design is because early sql databases had a fixed byte size for fields. So in that world, it would be impossible to embed a list in a cell. But that limitation hasn’t existed in modern sql databases for decades now. Why can’t we have nice things? I don’t see anything in sql that precludes embedded lists and objects in table cells. Postgres even - sort of - supports it already via json fields.
- cess11 2y ago
  If JSON is enough for your use case, go ahead, use it. The big RDBM systems support it rather well now, so you can bring it straight into the SQL world too. XML is for the non-trivial case, where having schemas and tooling around them for validation and code generation becomes useful. XML also has a default programming language for querying and transformations, allowing you to implement adapters from CSV, JSON or whatever into XML, and then between XML schemas. XML can be used in a JSON-like way if that's what you want: XPath. When programming against XML it's usually not what you want though, in practice it tends to be nicer to use the schema to set up an object hierarchy native to the programming language and unmarshal into that. But there is at least one XPath oriented database, eXist, and it's pretty ergonomic if you find yourself with a lot of XML and isn't sure what it contains or what to use it for. I find it easier to work with than the NoSQL JSON varieties I come across professionally, and when I have Questions For The Customer it craps out PDF:s I can send them, which doesn't seem that easy to do with Mongo or Firestore. As for SQL databases, they come in a lot of shapes and colours. I'm not so sure there is a JSON-first database equivalent of DuckDB, for example. Maybe I'm wrong, it's not a specialty of mine.
- minkles 2y ago
  It's nothing even remotely like JSON. It doesn't encode the same information or semantics at all. Your entire argument is "old stuff is bad and I like JSON". And yes we should use XML. At least it has competent, verifiable, consistent schemas and contracts. At the very least, you can encode xs:decimal in it unlike JSON which is "oh it's some shit that looks like a number and we might stick in a double at the other end because we don't know our butt from our elbow". I ask you with JSON, how do you encode "add a single item to the list attached to a customer" I am so so so tired of dealing with this level of thinking on a daily basis.
  - josephg 2y ago
    I must admit, I'm having more of a go at SQL than I really feel. SQL nailed transactional semantics, indexes and querying. My argument isn't that "old stuff is bad and I like JSON". Its that when I'm modelling data in my program, sometimes I have objects which reference each other (via key or shared pointer). And sometimes I have objects which contain another object. Eg, a struct with a vec inside. Real code is packed full of this stuff. SQL lets me directly model relationships between objects. And it lets me model records with scalar fields. But it does not let me model records with variable size fields. This restriction seems totally arbitrary. There is no technical reason for it that I can think of. And it makes it needlessly complex to map data from my programming language to my SQL database. I've named this problem 3 times in this thread now. For all your moaning about us kids being lazy, you still haven't given a single technical reason for this deficiency in SQL. I largely agree with you about JSON. I don't actually want JSON fields in my database. I want something strongly typed, with well defined semantics and a good query system. SQL has a great track record of providing just that. And it has a perfectly serviceable type system already. So why on earth is it missing this important, obvious feature? I just want my database to be able to express embedded lists and records, like I do in almost every programming language I've ever used, every messaging format I've used (protobuf, msgpack, json, xml, etc). And in many other databases (couchdb, mongo, foundationdb, etc). > I ask you with JSON, how do you encode "add a single item to the list attached to a customer" I can think of dozens of ways to express that. Postgres already supports that operation for embedded JSON fields using this ugly syntax: UPDATE <table> SET <field-name> = <field-name> || '{"a": 1}'::jsonb Mongodb expresses it like this: { $push: { <field1>: <value1>, ... } } Rethinkdb: r.table('marvel').get('IronMan')('equipment').append('newBoots').run(conn, callback) ... Do you need more examples? Basically every database natively supports this except for "raw" SQL databases. Edit: Apparently (according to other comments in this thread) some SQL databases are already moving in this direction anyway. How wonderful!

josephg 2y ago

I’d love to read that essay. But to me sql looks like crap json - it’s like json except if (for purely historical reasons) we didn’t have objects or lists except at the top level of a json blob. Claiming it’s a moral failing on my part for wanting nice, intuitive data structures is a bit rich. Go on, next why don’t you tell me that json is bad and we should use xml. Or c is lazy and we should use assembler. Without more detail, that’s all I hear - moral justifications for a dated design. And the design is dated. As far as I can tell, the reason for the current design is because early sql databases had a fixed byte size for fields. So in that world, it would be impossible to embed a list in a cell. But that limitation hasn’t existed in modern sql databases for decades now. Why can’t we have nice things? I don’t see anything in sql that precludes embedded lists and objects in table cells. Postgres even - sort of - supports it already via json fields.

cess11 2y ago
If JSON is enough for your use case, go ahead, use it. The big RDBM systems support it rather well now, so you can bring it straight into the SQL world too. XML is for the non-trivial case, where having schemas and tooling around them for validation and code generation becomes useful. XML also has a default programming language for querying and transformations, allowing you to implement adapters from CSV, JSON or whatever into XML, and then between XML schemas. XML can be used in a JSON-like way if that's what you want: XPath. When programming against XML it's usually not what you want though, in practice it tends to be nicer to use the schema to set up an object hierarchy native to the programming language and unmarshal into that. But there is at least one XPath oriented database, eXist, and it's pretty ergonomic if you find yourself with a lot of XML and isn't sure what it contains or what to use it for. I find it easier to work with than the NoSQL JSON varieties I come across professionally, and when I have Questions For The Customer it craps out PDF:s I can send them, which doesn't seem that easy to do with Mongo or Firestore. As for SQL databases, they come in a lot of shapes and colours. I'm not so sure there is a JSON-first database equivalent of DuckDB, for example. Maybe I'm wrong, it's not a specialty of mine.
minkles 2y ago
It's nothing even remotely like JSON. It doesn't encode the same information or semantics at all. Your entire argument is "old stuff is bad and I like JSON". And yes we should use XML. At least it has competent, verifiable, consistent schemas and contracts. At the very least, you can encode xs:decimal in it unlike JSON which is "oh it's some shit that looks like a number and we might stick in a double at the other end because we don't know our butt from our elbow". I ask you with JSON, how do you encode "add a single item to the list attached to a customer" I am so so so tired of dealing with this level of thinking on a daily basis.
- josephg 2y ago
  I must admit, I'm having more of a go at SQL than I really feel. SQL nailed transactional semantics, indexes and querying. My argument isn't that "old stuff is bad and I like JSON". Its that when I'm modelling data in my program, sometimes I have objects which reference each other (via key or shared pointer). And sometimes I have objects which contain another object. Eg, a struct with a vec inside. Real code is packed full of this stuff. SQL lets me directly model relationships between objects. And it lets me model records with scalar fields. But it does not let me model records with variable size fields. This restriction seems totally arbitrary. There is no technical reason for it that I can think of. And it makes it needlessly complex to map data from my programming language to my SQL database. I've named this problem 3 times in this thread now. For all your moaning about us kids being lazy, you still haven't given a single technical reason for this deficiency in SQL. I largely agree with you about JSON. I don't actually want JSON fields in my database. I want something strongly typed, with well defined semantics and a good query system. SQL has a great track record of providing just that. And it has a perfectly serviceable type system already. So why on earth is it missing this important, obvious feature? I just want my database to be able to express embedded lists and records, like I do in almost every programming language I've ever used, every messaging format I've used (protobuf, msgpack, json, xml, etc). And in many other databases (couchdb, mongo, foundationdb, etc). > I ask you with JSON, how do you encode "add a single item to the list attached to a customer" I can think of dozens of ways to express that. Postgres already supports that operation for embedded JSON fields using this ugly syntax: UPDATE <table> SET <field-name> = <field-name> || '{"a": 1}'::jsonb Mongodb expresses it like this: { $push: { <field1>: <value1>, ... } } Rethinkdb: r.table('marvel').get('IronMan')('equipment').append('newBoots').run(conn, callback) ... Do you need more examples? Basically every database natively supports this except for "raw" SQL databases. Edit: Apparently (according to other comments in this thread) some SQL databases are already moving in this direction anyway. How wonderful!