7 ms·

JSON as a Universal DB Model DSL

9 points by chasingsparks 17y ago

AndrewO 17y ago
Wait, so the author says we should design our own schema definition DSL, write some JSON that uses that DSL to define a database schema, and implement whatever transformations are required to generate a working migration from that DSL? How is that lazy?
- chasingsparks 17y ago
  The author (me) is being lazy by not writing this. I am outsourcing (hopefully) the implementation of an idea. Also, the transformation into a db schema is only one alleged benefit. E.G. class User < ActiveRecord::Base json_def 'user_model.json' end display_form(@obj) etc
  - AndrewO 17y ago
    I hope you take this as constructive criticism (and not me just being a jerk :), but I see a couple of problems: 1) JSON gives no benefit over a Ruby DSL (like ActiveRecord migrations). Is the schema file going to be consumed by client programs written in other languages? (Even so, in that case, you should probably just define ActiveRecord::Base.to_json_schema). Two languages are unnecessary when one will suffice. 2) When you start to define the schema in code[ft. 1], rather than in the database itself, you're leaving the Active Record pattern (as defined in PoEAA) and should probably look at other patterns (and the libraries that implement them). Data Mapper/DataMapper would probably be a good fit here. Also, you're loosing the benefits that migrations give of dealing with changes to the schema over time non-destructively. Dr Nic's Sexy Migrations did something that would diff a single schema file with the existing DB schema, but this is not the default solution. 3) Automatically generating forms from a model has tempted many Rails developers over the years. A rudimentary method would be possible using the existing reflection data (i.e. no need for a JSON schema DSL) and some added metadata. Google "rails admin ui" for some examples. However, "metadata" can easily turn into "writing forms in the model instead of the view" (I've seen it happen). That's why I'd avoid this. Also any forms more advanced than a simple create/edit interface and you're probably going to end up writing a partial anyway, especially if you need to add markup for formatting (e.g. how do you control if the generated form vertically or horizontally oriented? Meant to go in a table with others or independent?). ft. 1: Rails' Migrations are a special case. They should be a seen as a separate system from the ORM that was created later on to manage the complexity of changing schema. ActiveRecord classes still define their fields based on reflection from the database.
andreyf 17y ago
Changing the data format to JSON isn't the solution to most problems involving DB models. Providing a group of operations on that data is: diffs, negation-diffs, and composition of diffs.
andreyf 17y ago
I like the following better: column name | generalized type | validation rule | hidden? ------------------------------------------------------------------ first_name | text | name_regex | false last_name | text | name_regex | false middle_name | text | name_regex | false birth_date | date | over_18 | false location | [double,double] | undefined | true addresses | | has_many | false Parsing it into JSON-equivalent is a programming exercise you can give your interviewees. (my point is that data format doesn't matter - JSON, YAML, XML, |'s and -'s are just different "colors" giving semantic meaning to your data: but it's what you can do with that data which is important)
bokchoi 17y ago
Should change the title to begin with "I want..."
moe 17y ago
Well, apart from the fact that your "proposal" is incomplete and obviously not thought-through in the slightest (to put it politely) we also already have a canonical way to represent relational database schemas; http://en.wikipedia.org/wiki/Data_Definition_Language http://en.wikipedia.org/wiki/Data_Definition_Language Moreover the problem with database schemas has never been to represent them or reflect on them, but to migrate between them. See http://www.post-facto.org/ http://www.post-facto.org/ for an interesting approach (presentation by the author: http://hosting3.epresence.tv/fosslc/1/watch/128.aspx http://hosting3.epresence.tv/fosslc/1/watch/128.aspx).
- thwarted 17y ago
  If I remember correctly, SQLite parses the DDL for tables and indexes each time the database file is opened. I always thought this was kind of weird, and felt a binary representation would be more appropriate, but when you consider that DDL exists to describe the schema, it makes perfect sense. There's no need for another representation, and parsing the DDL is more future-proof as features are added.
jrockway 17y ago
This is what he actually wants: http://www.lukas-renggli.ch/smalltalk/magritte http://www.lukas-renggli.ch/smalltalk/magritte

9 points by chasingsparks 17y ago

Wait, so the author says we should design our own schema definition DSL, write some JSON that uses that DSL to define a database schema, and implement whatever transformations are required to generate a working migration from that DSL? How is that lazy?

chasingsparks 17y ago
The author (me) is being lazy by not writing this. I am outsourcing (hopefully) the implementation of an idea. Also, the transformation into a db schema is only one alleged benefit. E.G. class User < ActiveRecord::Base json_def 'user_model.json' end display_form(@obj) etc
- AndrewO 17y ago
  I hope you take this as constructive criticism (and not me just being a jerk :), but I see a couple of problems: 1) JSON gives no benefit over a Ruby DSL (like ActiveRecord migrations). Is the schema file going to be consumed by client programs written in other languages? (Even so, in that case, you should probably just define ActiveRecord::Base.to_json_schema). Two languages are unnecessary when one will suffice. 2) When you start to define the schema in code[ft. 1], rather than in the database itself, you're leaving the Active Record pattern (as defined in PoEAA) and should probably look at other patterns (and the libraries that implement them). Data Mapper/DataMapper would probably be a good fit here. Also, you're loosing the benefits that migrations give of dealing with changes to the schema over time non-destructively. Dr Nic's Sexy Migrations did something that would diff a single schema file with the existing DB schema, but this is not the default solution. 3) Automatically generating forms from a model has tempted many Rails developers over the years. A rudimentary method would be possible using the existing reflection data (i.e. no need for a JSON schema DSL) and some added metadata. Google "rails admin ui" for some examples. However, "metadata" can easily turn into "writing forms in the model instead of the view" (I've seen it happen). That's why I'd avoid this. Also any forms more advanced than a simple create/edit interface and you're probably going to end up writing a partial anyway, especially if you need to add markup for formatting (e.g. how do you control if the generated form vertically or horizontally oriented? Meant to go in a table with others or independent?). ft. 1: Rails' Migrations are a special case. They should be a seen as a separate system from the ORM that was created later on to manage the complexity of changing schema. ActiveRecord classes still define their fields based on reflection from the database.

andreyf 17y ago

Changing the data format to JSON isn't the solution to most problems involving DB models. Providing a group of operations on that data is: diffs, negation-diffs, and composition of diffs.

andreyf 17y ago

bokchoi 17y ago

Should change the title to begin with "I want..."

moe 17y ago

Well, apart from the fact that your "proposal" is incomplete and obviously not thought-through in the slightest (to put it politely) we also already have a canonical way to represent relational database schemas; http://en.wikipedia.org/wiki/Data_Definition_Language http://en.wikipedia.org/wiki/Data_Definition_Language Moreover the problem with database schemas has never been to represent them or reflect on them, but to migrate between them. See http://www.post-facto.org/ http://www.post-facto.org/ for an interesting approach (presentation by the author: http://hosting3.epresence.tv/fosslc/1/watch/128.aspx http://hosting3.epresence.tv/fosslc/1/watch/128.aspx).

thwarted 17y ago
If I remember correctly, SQLite parses the DDL for tables and indexes each time the database file is opened. I always thought this was kind of weird, and felt a binary representation would be more appropriate, but when you consider that DDL exists to describe the schema, it makes perfect sense. There's no need for another representation, and parsing the DDL is more future-proof as features are added.

jrockway 17y ago

This is what he actually wants: http://www.lukas-renggli.ch/smalltalk/magritte http://www.lukas-renggli.ch/smalltalk/magritte