6 ms·

> Use a helm chart and just bring your own little database for dev test and e2e tests. dev, test, and e2e tests should be done against full-size db clones

e12e 4y ago
> dev, test, and e2e tests should be done against full-size db clones Real customer/sensitive data should not exist outside prod (and backup). So generally no, not full-size clones. I'd argue instrumentation in prod should give information on performance - for some tests/development you might need prod-size fake data.
- holografix 4y ago
  Couldn’t agree more. Having full sized and fully speced dev and test DBs is wasteful and not realistic across several independent teams. Monitoring prod closely and understanding what could constitute a costly workload/query which would in turn require a temporary test env with similar sized dataset is the correct approach.
  - samokhvalov 4y ago
    This is too reactive approach, isn't it? Imagine a system that could do this in CI/CD pipelines: 1) first, run all the tests on a tiny DB as usual 2) extract queries 3) run them against full-size DB branch/thin clone (thin provisioning, CoW; PII is not there of course, wiped out for security/compliance) -- auto-guessing parms, that's the trickiest part, but assume it's solved 4) collect all the details about performance, focusing on IO numbers (rows, block read/writes) 5) if some queries are off – say, you forgot LIMIT – post a warning, block the change, do not allow deploying it, letting backend dev fix it. This would be proactive. And it's becoming possible with modern tools.
- Proven 4y ago
  [dead]
Blackthorn 4y ago
You think I'm going to clone a multiple petabyte database just to run some tests?
- samokhvalov 4y ago
  not sure about petabytes (yet), but for dozens of TiB, we are fine – DB branching, thin clones, CoW are to the rescue.
- Proven 4y ago
  [dead]
axlee 4y ago
> dev, test, and e2e tests should be done against full-size db clones that's cute, what is your "full-size"? I don't have 2 days to run a test, and I'm pretty sure every single compliance requirements we are following would get obliterated the second someone hears about us doing that
- samokhvalov 4y ago
  Agreed on compliance part – of course, in many cases (not in all though) PII must not be in dev/test envs. Although, cannot agree with the former part. If your tests are running 2 days on a full size clone, and it's an OLTP case, what about users, do they suffer from long query duration too? It sounds like it's time to optimize queries and/or redesign test sets (or both). If it's bad in testing, it will be bad in prod. That's the idea of testing. Example: how do you check schema changes?
- eddsh1994 4y ago
  You have a tool to keep the structure of data but anonymize it, I’ve seen this a few times in healthcare regulated systems
  - Foobar8568 4y ago
    I am laughing each time I hear this so call anonymous process /data. Each time, there were different ways to link back data or things were badly scraped.

e12e 4y ago

> dev, test, and e2e tests should be done against full-size db clones Real customer/sensitive data should not exist outside prod (and backup). So generally no, not full-size clones. I'd argue instrumentation in prod should give information on performance - for some tests/development you might need prod-size fake data.

holografix 4y ago
Couldn’t agree more. Having full sized and fully speced dev and test DBs is wasteful and not realistic across several independent teams. Monitoring prod closely and understanding what could constitute a costly workload/query which would in turn require a temporary test env with similar sized dataset is the correct approach.
- samokhvalov 4y ago
  This is too reactive approach, isn't it? Imagine a system that could do this in CI/CD pipelines: 1) first, run all the tests on a tiny DB as usual 2) extract queries 3) run them against full-size DB branch/thin clone (thin provisioning, CoW; PII is not there of course, wiped out for security/compliance) -- auto-guessing parms, that's the trickiest part, but assume it's solved 4) collect all the details about performance, focusing on IO numbers (rows, block read/writes) 5) if some queries are off – say, you forgot LIMIT – post a warning, block the change, do not allow deploying it, letting backend dev fix it. This would be proactive. And it's becoming possible with modern tools.
Proven 4y ago
[dead]

Blackthorn 4y ago

You think I'm going to clone a multiple petabyte database just to run some tests?

samokhvalov 4y ago
not sure about petabytes (yet), but for dozens of TiB, we are fine – DB branching, thin clones, CoW are to the rescue.
Proven 4y ago
[dead]

axlee 4y ago

> dev, test, and e2e tests should be done against full-size db clones that's cute, what is your "full-size"? I don't have 2 days to run a test, and I'm pretty sure every single compliance requirements we are following would get obliterated the second someone hears about us doing that

samokhvalov 4y ago
Agreed on compliance part – of course, in many cases (not in all though) PII must not be in dev/test envs. Although, cannot agree with the former part. If your tests are running 2 days on a full size clone, and it's an OLTP case, what about users, do they suffer from long query duration too? It sounds like it's time to optimize queries and/or redesign test sets (or both). If it's bad in testing, it will be bad in prod. That's the idea of testing. Example: how do you check schema changes?
eddsh1994 4y ago
You have a tool to keep the structure of data but anonymize it, I’ve seen this a few times in healthcare regulated systems
- Foobar8568 4y ago
  I am laughing each time I hear this so call anonymous process /data. Each time, there were different ways to link back data or things were badly scraped.