10 ms·

Fork Your Application's Data (Not Just Your Code)

74 points by thinkbohemian 14y ago

TheMakeA 14y ago
Interesting. I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc. I would love to be able to go back to any moment in the projects history and get a full, working build.
- Q6T46nT668w6i3m 14y ago
  I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc. Isn’t that the advantage of Perforce?
  - TheMakeA 14y ago
    It is possible with Perforce, but it is far from ideal. Compare branching in Git to branching in Perforce. There's still a lot of room for improvement.
    - hythloday 14y ago
      You should compare branching in Git to "streaming" in Perforce: http://www.perforce.com/product/product_features/perforce_streams http://www.perforce.com/product/product_features/perforce_st...
- pvh 14y ago
  Yes, wouldn't it be cool if you could fork an entire running app?
thibaut_barrere 14y ago
Also interesting: data clips https://postgres.heroku.com/dataclips https://postgres.heroku.com/dataclips I'll have to wait for them to become SafeHarbor compliant before using them, though (in progress!).
eggbrain 14y ago
One of the biggest problems I have is that I'll branch out to a new feature branch, modify (add/rename columns) in the db, then get sidetracked by something where I have to switch back to master, making everything explode because the db is still modified, while my application is calling the old values. I'd love it if they found a way to link up your database to a repo, so that it could track/rollback changes easily as switching branches (I guess an sqlite db might work, but it's not perfect)
- nlh 14y ago
  It's a great idea and something that even I, humble amateur developer, have encountered (and wanted). Any thoughts about which DB would be best suited for this? I wonder if it could be integrated with git somehow and make the whole process seamless.
  - viaken 14y ago
    You could certainly use SQLite this way, as long as you don't need anything hefty. Add and commit your db file, then fork and modify as needed.
- atomical 14y ago
  I would disagree with this strategy as it adds more complexity than the original problem. It's much easier to rollback the migrations on the feature branch and switch to master. Avoiding situations like this is the best policy though as unexpected things will happen and you can be less sure that things are working properly.
  - eggbrain 14y ago
    "Avoiding situations like this is the best policy though as unexpected things will happen" Yes, but they happen. This seems like replying to someone drowning "well, you should have avoided swimming!" You also make the assumption that we are using Rails (or some framework with migration support) -- many people won't have that luxury. Finally, while the setup might be challenging (as in, figuring out a way to implement db tracking in git), I find it very curious that it would be much easier in anyones mind to do two tasks, doing all the rollbacks and switching branches, vs just switching the branch.
5vforest 14y ago
This is neat. I've always used Amazon RDS in the past because of its ease of import/export, but this may be enough to switch me over to Heroku's offerings.
grimlck 14y ago
So you can't merge changes on the fork back to the main database? Without the ability to merge, shouldn't it just be called 'copying'
arscan 14y ago
Personally I find this interesting from a marketing perspective... its not like cloning a DB is a new concept, but by calling it "forking your data" they make it seem quite a bit sexier. I'm not sure if technically its the best term to use, but it does sound cool.
- scdoshi 14y ago
  Couldn't you always clone a git repo too? But the ability to "fork" (clone plus some setup) on github is still very useful.
CookWithMe 14y ago
What about images? Should one save them in the DB? Is Postgres able to handle high-res images? Last time I checked (for MySQL) it was possible, but not really recommended/wide-spread. Does anyone have practical experience with Postgres (and Heroku)?
sterling312 14y ago
I've been wanting people to post their eigenvector/value on facial recognition training for a while. Imagine a crowdsource trained facial recognition database!

74 points by thinkbohemian 14y ago

Interesting. I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc. I would love to be able to go back to any moment in the projects history and get a full, working build.

Q6T46nT668w6i3m 14y ago
I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc. Isn’t that the advantage of Perforce?
- TheMakeA 14y ago
  It is possible with Perforce, but it is far from ideal. Compare branching in Git to branching in Perforce. There's still a lot of room for improvement.
  - hythloday 14y ago
    You should compare branching in Git to "streaming" in Perforce: http://www.perforce.com/product/product_features/perforce_streams http://www.perforce.com/product/product_features/perforce_st...
pvh 14y ago
Yes, wouldn't it be cool if you could fork an entire running app?

thibaut_barrere 14y ago

Also interesting: data clips https://postgres.heroku.com/dataclips https://postgres.heroku.com/dataclips I'll have to wait for them to become SafeHarbor compliant before using them, though (in progress!).

eggbrain 14y ago

One of the biggest problems I have is that I'll branch out to a new feature branch, modify (add/rename columns) in the db, then get sidetracked by something where I have to switch back to master, making everything explode because the db is still modified, while my application is calling the old values. I'd love it if they found a way to link up your database to a repo, so that it could track/rollback changes easily as switching branches (I guess an sqlite db might work, but it's not perfect)

nlh 14y ago
It's a great idea and something that even I, humble amateur developer, have encountered (and wanted). Any thoughts about which DB would be best suited for this? I wonder if it could be integrated with git somehow and make the whole process seamless.
- viaken 14y ago
  You could certainly use SQLite this way, as long as you don't need anything hefty. Add and commit your db file, then fork and modify as needed.
atomical 14y ago
I would disagree with this strategy as it adds more complexity than the original problem. It's much easier to rollback the migrations on the feature branch and switch to master. Avoiding situations like this is the best policy though as unexpected things will happen and you can be less sure that things are working properly.
- eggbrain 14y ago
  "Avoiding situations like this is the best policy though as unexpected things will happen" Yes, but they happen. This seems like replying to someone drowning "well, you should have avoided swimming!" You also make the assumption that we are using Rails (or some framework with migration support) -- many people won't have that luxury. Finally, while the setup might be challenging (as in, figuring out a way to implement db tracking in git), I find it very curious that it would be much easier in anyones mind to do two tasks, doing all the rollbacks and switching branches, vs just switching the branch.

5vforest 14y ago

This is neat. I've always used Amazon RDS in the past because of its ease of import/export, but this may be enough to switch me over to Heroku's offerings.

grimlck 14y ago

So you can't merge changes on the fork back to the main database? Without the ability to merge, shouldn't it just be called 'copying'

arscan 14y ago

Personally I find this interesting from a marketing perspective... its not like cloning a DB is a new concept, but by calling it "forking your data" they make it seem quite a bit sexier. I'm not sure if technically its the best term to use, but it does sound cool.

scdoshi 14y ago
Couldn't you always clone a git repo too? But the ability to "fork" (clone plus some setup) on github is still very useful.

CookWithMe 14y ago

What about images? Should one save them in the DB? Is Postgres able to handle high-res images? Last time I checked (for MySQL) it was possible, but not really recommended/wide-spread. Does anyone have practical experience with Postgres (and Heroku)?

sterling312 14y ago

I've been wanting people to post their eigenvector/value on facial recognition training for a while. Imagine a crowdsource trained facial recognition database!