3 ms·

I seriously didn't expect this from magnolia. How hard/expensive is it to get an automated daily backup of the DB and sync it up to something like Amazon S3 ?

I seriously didn't expect this from magnolia. How hard/expensive is it to get an automated daily backup of the DB and sync it up to something like Amazon S3 ? I run a similar pet project ( pardon the shameless plug :) http://tagz.in http://tagz.in ), albeit with only a dozen people actively using it. I maintain a 30 day rolling backup of daily db dumps, paying next to nothing for the backups. I've had a number of people asking me how can they trust me if I decide to stop the whole thing, and a part of the plan was to allow people to download their bookmarks in Netscape bookmarks format for atleast 30 days from the day I announce just in case I decide to take it down / run out of money. Not like this is ever gonna happen (To be honest i probably can run it for the next 6 months, even if I lose my job this very day.), But I feel contingencies like these need to be thought about, well in advance when we're dealing with user's data (bookmarks in this case). PS: I use Postgresql and have a script on my laptop which syncs my local db with the latest db snapshot every week, which isn't really a good idea, but works for me, given the tiny scale that I'm working on.

bonaldi 18y ago
Hardish. Expensive when you're talking about terabytes. Regardless of that, though, ma.gnolia had backups; it's just they were untested and were dutifully backing up the corruptions that were being introduced on the software level. It's that bit I'd like to know more about. Were they faithfully following the Rails Way and expecting the model to handle validations, instead of setting up the database to do double validation? How did these errors creep in, and how did they grow to be so catastrophic?
- gensym 18y ago
  This corruption doesn't have anything to do with the "Rails Way". The corruption was at a much lower level then things like nil foreign keys, long fields, etc. that Rails validates. It was the filesystem that was corrupted, and whether he did database validation in ActiveRecord, using database validations, or both would not have made a difference, as far as I can tell. Think about it for a second - if the corruption that caused this was things that could be validated using Ruby or in SQL declarations, it would to more than feasible to import that data into some format form which it could be recovered.

bonaldi 18y ago

Hardish. Expensive when you're talking about terabytes. Regardless of that, though, ma.gnolia had backups; it's just they were untested and were dutifully backing up the corruptions that were being introduced on the software level. It's that bit I'd like to know more about. Were they faithfully following the Rails Way and expecting the model to handle validations, instead of setting up the database to do double validation? How did these errors creep in, and how did they grow to be so catastrophic?

gensym 18y ago
This corruption doesn't have anything to do with the "Rails Way". The corruption was at a much lower level then things like nil foreign keys, long fields, etc. that Rails validates. It was the filesystem that was corrupted, and whether he did database validation in ActiveRecord, using database validations, or both would not have made a difference, as far as I can tell. Think about it for a second - if the corruption that caused this was things that could be validated using Ruby or in SQL declarations, it would to more than feasible to import that data into some format form which it could be recovered.