6 ms·

I'm making a relational lang (https://tablam.org https://tablam.org) and this was one of the question marks for me. The #1 reason sum types was not presented i

by mamcx 5y ago

I'm making a relational lang (https://tablam.org https://tablam.org) and this was one of the question marks for me. The #1 reason sum types was not presented is because "nobody" thought (seriously) about that for long time. See how much "null" have been with us and still you find defenders of it! The second and more broad problem, is that the relational model was NOT implemented by the RDBMS makers, not in full. To make sum types work, you NEED to support embed relations inside relations: rel Invoice rel DetailInvoice Invoice[invoice_id:1 customer:123 lines: DetailInvoice[ [invoice_id:1 line_id:1 price:10 qty: 1], [invoice_id:1 line_id:2 price:10 qty: 2] ] Then with this, do sum types comes from free: type Option Some ? None [customer_id:1 phone: [tag:Some 123-23456]] [customer_id:2 phone: [tag:None]] This will make things like storing Logs, metrics and event sourcing so much nicer! How store that is another matter, but I think is not that complicated, just none of the major rdbms have put effort on this. This is one of my dreams: Making a rdbms with this and other stuff that push the DBs to the modern way of make apps...

cmrdporcupine 5y ago
Frankly, understanding of what the relational model actually is has been missing from the industry even among many people who have made data management their primary focus. IMHO a lot of the penetration of the NoSQL stuff in the last decade was based on misunderstandings of the capabilities and potentials of the relational model. I think the early history of SQL was too influenced by ISAM-type stores and COBOL flatfiles and the like. Hence the vocabulary around "tables" and "columns" instead of the relations, relvars, tuples, etc. described by the actual relational algebraic model. To many of those with an OO fetish, "relational model" became synonymous with "tabular model", and the industry's early 2000's obsession with object polymorphism, OORMs middleware, etc continued the stall in relational database innovation. I feel like there is now renewed interest in this stuff, though. Your project looks really neat, for example.
- mamcx 5y ago
  I think you analysis is correct. I also add that is related too to the languages, you point with Cobol, and I also add C, Pascal, Java, etc. When they have nulls and limited to zero queries, relationships and data modeling capabilities is not hard to see that how they are is reflected in how the DBs are too. I honestly not know about sum types until a few years ago, neither have a clue how much useful they are, so surely even today many are in the same blind spots...
goto11 5y ago
> To make sum types work, you NEED to support embed relations inside relations If that is the case, sum types would be in violation of first normal form.
- mamcx 5y ago
  Even considering that normal forms are tools... Sum types, how them are stored and how they are used is not the same ie: How are sum types represented in languages like Rust/oCalm? that is tangential as how we experience them. So, support to embed relations is the step to provide the storage and partially, help in the query, but from the POV of the user them are "normal sum types". But with that, you get the ability to store relations, that is usefull as-is :)
- emmelaich 5y ago
  Oddly enough, 1nf is differently defined depending on which person or database you're asking/using. Is it . just to be rectangular? . values are atomic? (and what does that mean) . no such thing as nulls? good luck with that . no duplicate rows (good luck with enforcing that in intermediate tables)
  - goto11 5y ago
    E.F.Codd defines first normal form as eliminating nested relations (in "A Relational Model of Data for Large Shared Data Banks" which introduces the relational model). "Atomic" in this context just means any value which is not a relation. A relation by definition cannot have duplicate rows, since relations are sets. If "rectangular" means that all rows in a table have the same columns (attributes), this also follows from the definition of relations. So these constraints have to be fulfilled before we can even talk about normalization, since normalization operates on relations. (Nulls are its own can of worms, but dosn't really have anything to do with 1NF.) Codd introduces 1NF because he realized supporting nested relations would complicate the model and query language for no benefit. It would need special operators for navigate in and out or nested relations, e.g. if you wanted to join tables nested at different levels in other tables. You would need something like XPath on top of SQL. But you can express exactly the same information using foreign keys between tables and keep the language simpler. The sum-types suggestion shows the problem. If we have sum types as values, presumably they would be composed of other types, right? So sum types could contain other sum types, arbitrary deep. Since you surely would want to query these individual values, you would need operators in SQL to navigate down into sum types. You would need facilities to index and apply constraints, again arbitrary deep down the structure. So the database model and query language gets significantly more complex for dubious benefit.
alexvoda 5y ago
I am interested in opportunities to cooperate on this.
- mamcx 5y ago
  Great!, I have the project at https://tablam.org https://tablam.org
woggy 5y ago
Weird that this pops up today as I was trying to find something like this yesterday. Good luck with it!
tome 5y ago
> To make sum types work, you NEED to support embed relations inside relations You don't need to embed relations. You don't even need to embed tuples or other composite types, you can just model sum types using a pair of fields. The latter is what Opaleye and Rel8 (Postgres libraries for Haskell) do: https://hackage.haskell.org/package/opaleye-0.7.1.0/docs/src/Opaleye.Internal.MaybeFields.html#MaybeFields https://hackage.haskell.org/package/opaleye-0.7.1.0/docs/src...
- alexvoda 5y ago
  You linked to a file with pages of code. Care to summarize how they model sum types using a pair of fields?
- mamcx 5y ago
  Can be show how is done, with data instead of algorithms? (aka "...show me your tables and I won't usually need your flowcharts; they'll be obvious." ~ Fred Brooks") Haskell is hard to decipher for me...
- tome 5y ago
  To reply to both my respondents, the link goes exactly to the source for Opaleye's version of the Maybe data type definition. Neatened up, it is data MaybeFields fields = MaybeFields { mfPresent :: Field SqlBool , mfFields :: fields } That is, an Opaleye Maybe (MaybeFields) is a pair. The first component is an SQL bool indicating whether it's a Just or a Nothing. The second component is the payload of the Just. If it's a Nothing the fields of the payload will be set to NULL.
  - mamcx 5y ago
    Ok, this is for a Maybe/Option, but can't see how can work for any algebraic type. I have this for mine: struct Ag { tag :String, //like "Option.None" data:Vec<Value> //The data inside, if any }
    - tome 5y ago

by mamcx 5y ago

cmrdporcupine 5y ago

Frankly, understanding of what the relational model actually is has been missing from the industry even among many people who have made data management their primary focus. IMHO a lot of the penetration of the NoSQL stuff in the last decade was based on misunderstandings of the capabilities and potentials of the relational model. I think the early history of SQL was too influenced by ISAM-type stores and COBOL flatfiles and the like. Hence the vocabulary around "tables" and "columns" instead of the relations, relvars, tuples, etc. described by the actual relational algebraic model. To many of those with an OO fetish, "relational model" became synonymous with "tabular model", and the industry's early 2000's obsession with object polymorphism, OORMs middleware, etc continued the stall in relational database innovation. I feel like there is now renewed interest in this stuff, though. Your project looks really neat, for example.

mamcx 5y ago
I think you analysis is correct. I also add that is related too to the languages, you point with Cobol, and I also add C, Pascal, Java, etc. When they have nulls and limited to zero queries, relationships and data modeling capabilities is not hard to see that how they are is reflected in how the DBs are too. I honestly not know about sum types until a few years ago, neither have a clue how much useful they are, so surely even today many are in the same blind spots...

goto11 5y ago

> To make sum types work, you NEED to support embed relations inside relations If that is the case, sum types would be in violation of first normal form.

mamcx 5y ago
Even considering that normal forms are tools... Sum types, how them are stored and how they are used is not the same ie: How are sum types represented in languages like Rust/oCalm? that is tangential as how we experience them. So, support to embed relations is the step to provide the storage and partially, help in the query, but from the POV of the user them are "normal sum types". But with that, you get the ability to store relations, that is usefull as-is :)
emmelaich 5y ago
Oddly enough, 1nf is differently defined depending on which person or database you're asking/using. Is it . just to be rectangular? . values are atomic? (and what does that mean) . no such thing as nulls? good luck with that . no duplicate rows (good luck with enforcing that in intermediate tables)
- goto11 5y ago
  E.F.Codd defines first normal form as eliminating nested relations (in "A Relational Model of Data for Large Shared Data Banks" which introduces the relational model). "Atomic" in this context just means any value which is not a relation. A relation by definition cannot have duplicate rows, since relations are sets. If "rectangular" means that all rows in a table have the same columns (attributes), this also follows from the definition of relations. So these constraints have to be fulfilled before we can even talk about normalization, since normalization operates on relations. (Nulls are its own can of worms, but dosn't really have anything to do with 1NF.) Codd introduces 1NF because he realized supporting nested relations would complicate the model and query language for no benefit. It would need special operators for navigate in and out or nested relations, e.g. if you wanted to join tables nested at different levels in other tables. You would need something like XPath on top of SQL. But you can express exactly the same information using foreign keys between tables and keep the language simpler. The sum-types suggestion shows the problem. If we have sum types as values, presumably they would be composed of other types, right? So sum types could contain other sum types, arbitrary deep. Since you surely would want to query these individual values, you would need operators in SQL to navigate down into sum types. You would need facilities to index and apply constraints, again arbitrary deep down the structure. So the database model and query language gets significantly more complex for dubious benefit.

alexvoda 5y ago

I am interested in opportunities to cooperate on this.

mamcx 5y ago
Great!, I have the project at https://tablam.org https://tablam.org

woggy 5y ago

Weird that this pops up today as I was trying to find something like this yesterday. Good luck with it!

tome 5y ago

> To make sum types work, you NEED to support embed relations inside relations You don't need to embed relations. You don't even need to embed tuples or other composite types, you can just model sum types using a pair of fields. The latter is what Opaleye and Rel8 (Postgres libraries for Haskell) do: https://hackage.haskell.org/package/opaleye-0.7.1.0/docs/src/Opaleye.Internal.MaybeFields.html#MaybeFields https://hackage.haskell.org/package/opaleye-0.7.1.0/docs/src...

alexvoda 5y ago
You linked to a file with pages of code. Care to summarize how they model sum types using a pair of fields?
mamcx 5y ago
Can be show how is done, with data instead of algorithms? (aka "...show me your tables and I won't usually need your flowcharts; they'll be obvious." ~ Fred Brooks") Haskell is hard to decipher for me...
tome 5y ago
To reply to both my respondents, the link goes exactly to the source for Opaleye's version of the Maybe data type definition. Neatened up, it is data MaybeFields fields = MaybeFields { mfPresent :: Field SqlBool , mfFields :: fields } That is, an Opaleye Maybe (MaybeFields) is a pair. The first component is an SQL bool indicating whether it's a Just or a Nothing. The second component is the payload of the Just. If it's a Nothing the fields of the payload will be set to NULL.
- mamcx 5y ago
  Ok, this is for a Maybe/Option, but can't see how can work for any algebraic type. I have this for mine: struct Ag { tag :String, //like "Option.None" data:Vec<Value> //The data inside, if any }
  - tome 5y ago