4 ms·
Rake is just a task runner, it sounds like maybe you had some rake tasks that were constantly ran to fix bad data or take care of recurring problems. This is th
by gt565k 6y ago
Rake is just a task runner, it sounds like maybe you had some rake tasks that were constantly ran to fix bad data or take care of recurring problems. This is the wrong approach. I've seen this done by DBAs who had stored procs to fix issues with data, without ever bothering to check with the dev team and have them fix the integration that was contributing to the faulty data entering the db.
Bad data integrations and poor UI validation bleed garbage data into the data store. That's just the reality of life. If you're constantly fixing records and cleaning data, you need to fix it at the source. Assuming there's a pattern, you can fix the data integration, automated process, or poor UI that's producing those records.
Generally speaking, you want good constraints at the data model layer, in order to prevent bad data from being persisted to the data store by the application layer. It takes a good understanding of database design and data modeling to engineer a good model with constraints. A well structured and properly normalized schema will outlast multiple application layer backends and front-ends, for decades perhaps. Making sure you add proper unique constraints, do not accept NULL values, add required validation to fields, check constraints for various logic, and proper relationship management and normalization of the model.
Once you have the proper constraints on the database table, the app layer will no longer allow poor data to be written that violates those constraints, and the errors will propagate back out to the end user, without ever being persisted in the data store.
Of course constraints can't catch all bad data, the rest really relies on having a good support system and reproducing problems and fixing the processes and integrations that produce poor/unwanted data records.
- andriosr 6y agoCompletely agree. These problems should not happen twice, and if they are frequent there is a problem somewhere else. But as you mention, even with the best validations, it can happen sometimes. What processes/tools have you seen work to fix them without Manual work or untraceable scripts?