4 ms·

Would you consider it evil to use production backups for the QA step in CI/CD? It would catch two birds with one stone, continuously verifying the backups, and

by tinco 6y ago

Would you consider it evil to use production backups for the QA step in CI/CD? It would catch two birds with one stone, continuously verifying the backups, and ensuring the new code works on real world data. We don't have any personal information in our production database, but even if we did, as long as the QA is thoroughly prevented from interacting with the outside world, it can't hurt to use production data right?

antoinealb 6y ago
I think it is a big issue to use anything containing PII in anything else than production environment. For example, do your QA logs have the same kind of management, access control and lifetime than production ? If they don't, you might end up logging PII in a way you should not, and could even be in legal troubles for things like GDPR.
jacquesm 6y ago
Big risk of having your privacy sensitive information stored on other infra than the main servers a path to get that information to your QA servers should not even exist. Many more people that should not have access to that data, in bulk or otherwise suddenly have access which does an end run around all your - hopefully - carefully crafted security. Keep in mind that what you think isn't privacy sensitive may very well be radioactive when combined with other data.
marcosdumay 6y ago
If you have a procedure for deriving QA data from production, change it to derive from the backup. Using it directly is full of dangers, not only of leaking information, but also of corrupting your backups. (Otherwise, why are you testing anything?) And deriving test data from production looks like a good thing to me, but make sure to restrict the access to the test environment and mask your data.
- jacquesm 6y ago
  You should not use production data for QA, period. Use generated data. If you really must use real data make sure that (1) you have explicit direction to do this from those higher up in the food chain so that if there ever is a breach of your test systems you don't end up holding the bag and (2) anonymize a copy of the data before it leaves the production system. That way you minimize the risk. But better: simply don't do it, generate your test data.
- icedchai 6y ago
  Make sure you restrict access to the production environment, too. I once worked at a company where the "dev" office was VPN'd into production at all times. We actually didn't have a QA system for most of the time I was there. Anyway, a developer ran his server with the prod config for a couple of days before anyone noticed that requests were being processed elsewhere.

by tinco 6y ago

antoinealb 6y ago

I think it is a big issue to use anything containing PII in anything else than production environment. For example, do your QA logs have the same kind of management, access control and lifetime than production ? If they don't, you might end up logging PII in a way you should not, and could even be in legal troubles for things like GDPR.

jacquesm 6y ago

Big risk of having your privacy sensitive information stored on other infra than the main servers a path to get that information to your QA servers should not even exist. Many more people that should not have access to that data, in bulk or otherwise suddenly have access which does an end run around all your - hopefully - carefully crafted security. Keep in mind that what you think isn't privacy sensitive may very well be radioactive when combined with other data.

marcosdumay 6y ago

If you have a procedure for deriving QA data from production, change it to derive from the backup. Using it directly is full of dangers, not only of leaking information, but also of corrupting your backups. (Otherwise, why are you testing anything?) And deriving test data from production looks like a good thing to me, but make sure to restrict the access to the test environment and mask your data.

jacquesm 6y ago
You should not use production data for QA, period. Use generated data. If you really must use real data make sure that (1) you have explicit direction to do this from those higher up in the food chain so that if there ever is a breach of your test systems you don't end up holding the bag and (2) anonymize a copy of the data before it leaves the production system. That way you minimize the risk. But better: simply don't do it, generate your test data.
icedchai 6y ago
Make sure you restrict access to the production environment, too. I once worked at a company where the "dev" office was VPN'd into production at all times. We actually didn't have a QA system for most of the time I was there. Anyway, a developer ran his server with the prod config for a couple of days before anyone noticed that requests were being processed elsewhere.