3 ms·

Roughly: Say you have a relational database with a dozen tables that all connect to each other using foreign keys: A user table has a user id, a message table r

Roughly: Say you have a relational database with a dozen tables that all connect to each other using foreign keys: A user table has a user id, a message table refers to a user, a log table refers to the user and a message, a comment table refers to a message and a user, and so forth. Jailer can, say, grab a few comments from the comment table and at the same time grab any users and messages that were referenced. Then all of those records spread across all of the tables can be exported for use by, for instance, tests. Imagine you found a bug, say, in your app logic, and you wanted to grab an image of the exact data that triggers the bug to repro that bug and use it in a test. This tool makes that easy.

rejectedandsad 5y ago
> Imagine you found a bug, say, in your app logic, and you wanted to grab an image of the exact data that triggers the bug to repro that bug and use it in a test. This tool makes that easy. This is actually a fairly common usecase in my $DAYJOB, what's the typical name for a tool that does this (that isn't the above, since I don't use a SQLdb?)
- rzzzt 5y ago
  "Referential integrity" is when you don't have any dangling foreign keys that can not be resolved to an entity/document/row. "Data subsetting" is the process of extracting the relevant parts out of a larger data set.
- helderoliozuzl 5y ago
  What do you use instead of SQL DB?
  - rejectedandsad 5y ago
    DynamoDB. The data model is relational, despite the fact that it shouldn't be.
- rbobby 5y ago
  I wrote my own for this, specific for my $DAYJOB application. It primarily focused on configuration tables, and optionally a slice of the main data tables. The tricky parts were: 1. Write code to write out DDL. This is easier than it sounds because not every DDL feature is actually used by the application. 2. Write code to dump out DML. Again easier than it sounds because not every datatype is used by the application. The tricky parts where automatic identity columns and large binary table. 3. The code from #1 and #2 has to be done in topological order (least dependent scheme objects to most dependent). After the DDL is write, write the DML. 4. A bit fancy streaming to ensure the webserver doesn't build everything up in memory before zip filing it and beaming it down the wire. 5. Some care needs to be taken for confidential information. User passwords (even bcrypted/scrypted) shouldn't come along. Nor should thinks like outgoing email server settings. Or incoming email server settings. I'm lucky as no PII or credit card details exist in $DAYJOB application. If done carefully the code should be insensitive to simple schema changes (new fields, new configuration tables). The goal behind it all is to produce a single (big) .SQL file that will recreate a working database. So worth the time and effort to create.

rejectedandsad 5y ago

> Imagine you found a bug, say, in your app logic, and you wanted to grab an image of the exact data that triggers the bug to repro that bug and use it in a test. This tool makes that easy. This is actually a fairly common usecase in my $DAYJOB, what's the typical name for a tool that does this (that isn't the above, since I don't use a SQLdb?)

rzzzt 5y ago
"Referential integrity" is when you don't have any dangling foreign keys that can not be resolved to an entity/document/row. "Data subsetting" is the process of extracting the relevant parts out of a larger data set.
helderoliozuzl 5y ago
What do you use instead of SQL DB?
- rejectedandsad 5y ago
  DynamoDB. The data model is relational, despite the fact that it shouldn't be.
rbobby 5y ago
I wrote my own for this, specific for my $DAYJOB application. It primarily focused on configuration tables, and optionally a slice of the main data tables. The tricky parts were: 1. Write code to write out DDL. This is easier than it sounds because not every DDL feature is actually used by the application. 2. Write code to dump out DML. Again easier than it sounds because not every datatype is used by the application. The tricky parts where automatic identity columns and large binary table. 3. The code from #1 and #2 has to be done in topological order (least dependent scheme objects to most dependent). After the DDL is write, write the DML. 4. A bit fancy streaming to ensure the webserver doesn't build everything up in memory before zip filing it and beaming it down the wire. 5. Some care needs to be taken for confidential information. User passwords (even bcrypted/scrypted) shouldn't come along. Nor should thinks like outgoing email server settings. Or incoming email server settings. I'm lucky as no PII or credit card details exist in $DAYJOB application. If done carefully the code should be insensitive to simple schema changes (new fields, new configuration tables). The goal behind it all is to produce a single (big) .SQL file that will recreate a working database. So worth the time and effort to create.