3 ms·

>>The development server included a snapshot of our production database, which included encrypted data. >Never, ever do this. (It's been a firing offense at mo

>>The development server included a snapshot of our production database, which included encrypted data. >Never, ever do this. (It's been a firing offense at most shops I've worked at in my career, and that's for good reasons.) Would you use a migration script that "scrambled" certain columns on the way back from Production to Development? That's the only way I've seen it done properly at $EMPLOYER.

rhizome 11y ago
Seeding development with production data is just lazy, unnecessary, and, as we see here, dangerous. Spend some resources on a proper seeding setup.
- voltagex_ 11y ago
  What would you suggest for such a setup? Also, I did not say seed it with production data - I mean jumble PII and then import a section of production data into dev. Sometimes this is necessary.
  - thaumaturgy 11y ago
    A while back a fake DB data generator was posted to HN: https://news.ycombinator.com/item?id=8761953 https://news.ycombinator.com/item?id=8761953 Further back, there's also this one for Python: https://news.ycombinator.com/item?id=7102268 https://news.ycombinator.com/item?id=7102268 And this one for PHP: https://github.com/fzaninotto/Faker https://github.com/fzaninotto/Faker (there were no HN comments on that thread) And this one for Ruby: https://news.ycombinator.com/item?id=86018 https://news.ycombinator.com/item?id=86018 (also no comments) These seem like a good starting point. Also, mining bug reports for data that causes problems in your application is a lot of fun. You can turn your development database into a collection of pathological edge cases.
patio11 11y ago
Would you use a migration script that "scrambled" certain columns on the way back from Production to Development? No, because you'll almost certainly fail to fully anonymize the data and if there is anything "interesting" in your possession it will probably still be recoverable. I wrote a comment on HN long ago about this, and still stand by it. https://news.ycombinator.com/item?id=1142326 https://news.ycombinator.com/item?id=1142326
- voltagex_ 11y ago
  Wow, thank you. It took that kind of crystal clear example for me to understand the risk clearly.

rhizome 11y ago

Seeding development with production data is just lazy, unnecessary, and, as we see here, dangerous. Spend some resources on a proper seeding setup.

voltagex_ 11y ago
What would you suggest for such a setup? Also, I did not say seed it with production data - I mean jumble PII and then import a section of production data into dev. Sometimes this is necessary.
- thaumaturgy 11y ago
  A while back a fake DB data generator was posted to HN: https://news.ycombinator.com/item?id=8761953 https://news.ycombinator.com/item?id=8761953 Further back, there's also this one for Python: https://news.ycombinator.com/item?id=7102268 https://news.ycombinator.com/item?id=7102268 And this one for PHP: https://github.com/fzaninotto/Faker https://github.com/fzaninotto/Faker (there were no HN comments on that thread) And this one for Ruby: https://news.ycombinator.com/item?id=86018 https://news.ycombinator.com/item?id=86018 (also no comments) These seem like a good starting point. Also, mining bug reports for data that causes problems in your application is a lot of fun. You can turn your development database into a collection of pathological edge cases.

patio11 11y ago

Would you use a migration script that "scrambled" certain columns on the way back from Production to Development? No, because you'll almost certainly fail to fully anonymize the data and if there is anything "interesting" in your possession it will probably still be recoverable. I wrote a comment on HN long ago about this, and still stand by it. https://news.ycombinator.com/item?id=1142326 https://news.ycombinator.com/item?id=1142326

voltagex_ 11y ago
Wow, thank you. It took that kind of crystal clear example for me to understand the risk clearly.