5 ms·

I would definitely prefer an approach where schema is explicit and defined in a minimum number of places. RDBMS/SQL schema do let you define schemas, however an

by fmjrey 8y ago

I would definitely prefer an approach where schema is explicit and defined in a minimum number of places. RDBMS/SQL schema do let you define schemas, however an important aspect of how SQL handles schemas which links back to their inflexibility is how they're defined at the level of a table, meaning a (possibly ordered) set of attributes. A more flexible approach as taken by Datomic is to define schemas at the level of attributes, meaning an entity (table) does not have a schema, only attributes do. This requires attribute names to be within namespaces which can certainly be seen as equivalent to table names, but they don't have to, namespaces could also refer to an entire domain/subdomain/org/whatever. Defining schema at the level of attributes provides a greater level of flexibility because client only declare/specify/depend-on a partial schema that only lists the attributes it needs. It also helps avoiding the "place oriented programming" approach of result sets, whereby attributes must be in a certain order, and if one does not have a value, you have to introduce NULL values.

Scarbutt 8y ago
Datomic is stritcly more(than SQL) between both worlds, you have half of your schema in the database(attributes) and the other half in clojure code (integrity checks, constraints, some validations, etc...).
jayd16 8y ago
Hmm I'm not familiar with Datomic but in SQL you can always break tables apart as it makes sense. NULL values are fine too if you've already decided that is a valid state in your schema but you can also use mapping tables instead of nullable columns. >client only declare/specify/depend-on a partial schema that only lists the attributes it needs. Can you describe what you mean here? It just sounds like specifying your selected columns to someone who is not familiar with the tool.
- fmjrey 8y ago
  Datomic uses datalog which can select any entity based on an expression about attributes, no need to say FROM table name or entity type. Basically entities have an id and can have any attribute. The official doc [1] gives the example of querying all movies by selecting all entities which have a `movie/title` attribute: `[:find ?e :where [?e :movie/title]]`. In that example the `movie` namespace really plays the role of a table name, or entity type, but as I said, namespaces can represent anything and entities can have any attributes, e.g. `[:find ?e :where [?e :audit/last-modified-by "user123"]]` would return any entity last modified by a given user. Note that this latter example is not really how you would capture last modified info in datomic, one would use transaction data [2] instead. Another example is master data management whereby attribute namespaces can refer to the different data sources, e.g. `accounting-system/id`, `supplier/id`, etc. [1] https://docs.datomic.com/on-prem/getting-started/query-the-data.html https://docs.datomic.com/on-prem/getting-started/query-the-d... [2] https://docs.datomic.com/on-prem/transactions.html#reified-transactions https://docs.datomic.com/on-prem/transactions.html#reified-t...
tabtab 8y ago
Some domains or projects require or work better with "soft" schemas, at least initially. I always wanted "dynamic relational" where columns are optional and "create on write" (except for "ID", which is needed for row uniqueness guarantee). If you query for a column that doesn't exist, instead of an error, you get a blank for that column (or null, depending on design decisions or settings). But, one can gradually add rules as the requirements solidify. You can later add a rule that all rows of the Employee table must have a non-blank "last_name" column. You can have a rule/constraint that requires it for new rows, and perhaps have a "cleaning mode" to check existing rows. With enough rules/constraints, it acts pretty much like the current crop of RDBMS. Dynamic relational could still use SQL, with some minor modifications to clarify intended compare types. (SQL has warts, but nothing better has arrived yet that's sufficiently better to justify tossing an established standard.) It's not SQL versus No-SQL, it should be strong-typed SQL versus "soft" SQL. That way one doesn't have to toss all their SQL/RDBMS knowledge to get a dynamic RDBMS. The current dichotomy of choice is unnecessarily stark: it's either an RDBMS with SQL, or something very foreign. The No-SQL movement was driven by hardware & scaling needs, not query language needs. If anyone is interested in starting up a dynamic relational company, I'd be glad to supply you with a draft spec for free.
- scriptkiddy 8y ago
  You may have heard of it already, but if you haven't, I would recommend taking a look at RethinkDb: https://www.rethinkdb.com/docs/architecture/ https://www.rethinkdb.com/docs/architecture/ It is a NoSQL Db, but has a lot of the features you're talking about. It doesn't have any form of "schema", but does allow for SQL style relational data. It definitely supports your "dynamic relational" data idea as stated above. The only issue is that any sort of data constraints need to be addressed at the outside of the Db itself.
  - tabtab 8y ago
    Interesting, but the "outside" constraint processor issue could be a show-stopper. The "gradual tightening" of the DB is a key selling point of dynamic relational. Parts of it could be borrowed to make DR. And it's not clear how one queries it with SQL (or something very close). A translator layer would probably be needed.

by fmjrey 8y ago

Scarbutt 8y ago

Datomic is stritcly more(than SQL) between both worlds, you have half of your schema in the database(attributes) and the other half in clojure code (integrity checks, constraints, some validations, etc...).

jayd16 8y ago

Hmm I'm not familiar with Datomic but in SQL you can always break tables apart as it makes sense. NULL values are fine too if you've already decided that is a valid state in your schema but you can also use mapping tables instead of nullable columns. >client only declare/specify/depend-on a partial schema that only lists the attributes it needs. Can you describe what you mean here? It just sounds like specifying your selected columns to someone who is not familiar with the tool.

fmjrey 8y ago
Datomic uses datalog which can select any entity based on an expression about attributes, no need to say FROM table name or entity type. Basically entities have an id and can have any attribute. The official doc [1] gives the example of querying all movies by selecting all entities which have a `movie/title` attribute: `[:find ?e :where [?e :movie/title]]`. In that example the `movie` namespace really plays the role of a table name, or entity type, but as I said, namespaces can represent anything and entities can have any attributes, e.g. `[:find ?e :where [?e :audit/last-modified-by "user123"]]` would return any entity last modified by a given user. Note that this latter example is not really how you would capture last modified info in datomic, one would use transaction data [2] instead. Another example is master data management whereby attribute namespaces can refer to the different data sources, e.g. `accounting-system/id`, `supplier/id`, etc. [1] https://docs.datomic.com/on-prem/getting-started/query-the-data.html https://docs.datomic.com/on-prem/getting-started/query-the-d... [2] https://docs.datomic.com/on-prem/transactions.html#reified-transactions https://docs.datomic.com/on-prem/transactions.html#reified-t...

tabtab 8y ago

Some domains or projects require or work better with "soft" schemas, at least initially. I always wanted "dynamic relational" where columns are optional and "create on write" (except for "ID", which is needed for row uniqueness guarantee). If you query for a column that doesn't exist, instead of an error, you get a blank for that column (or null, depending on design decisions or settings). But, one can gradually add rules as the requirements solidify. You can later add a rule that all rows of the Employee table must have a non-blank "last_name" column. You can have a rule/constraint that requires it for new rows, and perhaps have a "cleaning mode" to check existing rows. With enough rules/constraints, it acts pretty much like the current crop of RDBMS. Dynamic relational could still use SQL, with some minor modifications to clarify intended compare types. (SQL has warts, but nothing better has arrived yet that's sufficiently better to justify tossing an established standard.) It's not SQL versus No-SQL, it should be strong-typed SQL versus "soft" SQL. That way one doesn't have to toss all their SQL/RDBMS knowledge to get a dynamic RDBMS. The current dichotomy of choice is unnecessarily stark: it's either an RDBMS with SQL, or something very foreign. The No-SQL movement was driven by hardware & scaling needs, not query language needs. If anyone is interested in starting up a dynamic relational company, I'd be glad to supply you with a draft spec for free.

scriptkiddy 8y ago
You may have heard of it already, but if you haven't, I would recommend taking a look at RethinkDb: https://www.rethinkdb.com/docs/architecture/ https://www.rethinkdb.com/docs/architecture/ It is a NoSQL Db, but has a lot of the features you're talking about. It doesn't have any form of "schema", but does allow for SQL style relational data. It definitely supports your "dynamic relational" data idea as stated above. The only issue is that any sort of data constraints need to be addressed at the outside of the Db itself.
- tabtab 8y ago
  Interesting, but the "outside" constraint processor issue could be a show-stopper. The "gradual tightening" of the DB is a key selling point of dynamic relational. Parts of it could be borrowed to make DR. And it's not clear how one queries it with SQL (or something very close). A translator layer would probably be needed.