4 ms·

How does this work IRL for side-effectful services.for example, if somebody posts an order and it mirrors to my dev service, does that also hit my dev DB? Then

How does this work IRL for side-effectful services.for example, if somebody posts an order and it mirrors to my dev service, does that also hit my dev DB? Then if there is a live request to access a record detail page, does that expect my dev DB to have it? I’m just trying to wrap my brain about teams use tools like this in practice. Is it mostly for load testing or a better form of E2E?

eyalbukchin 4y ago
Up to you: you’re going to be able to configure what works locally vs what works with the cloud environment. So for example, you could configure all read functionality (incoming traffic, DB reads) to work with the cloud, and all write functionality to work locally. Further down the line, we’re planning to support Copy On Write for databases - so you’d read from the cloud, write locally, then read the records you wrote locally (and records you haven’t written, from the cloud). Disclaimer: we’re still pretty early in the roadmap, so currently only support mirroring incoming traffic. But we’re moving fast :)
opportune 4y ago
I would generally use this for e2e testing since it's too expensive to run a 100% mirror of a large production environment. Well, depends on your scale and whether your production costs are a significant factor in your total expenses - I guess if your production costs are tiny compared to revenue/dev time spent debugging, you could justify a full mirror. For something like an eshop where revenue >> production costs you could probably justify cloning out production multiple times. Sampling is another useful tool but I don't see any sampling in the API for this. If you do sample though you could end up with inconsistent or divergent state if you just sampled a flat % of traffic, so in some contexts it makes sense for sampling to be determined per-user/tenant/etc. rather than per-request. This is pod based so you could easily slice by pod. If you have a pod with a lot less traffic, more impactful traffic, or that is harder to debug (example: pod that periodically aggregates and exports metrics but you believe it's dropping data sometimes) you could just enroll that and have some kind of dev shard of a portion of the stack. I could also see this being very useful if you believe it to be too big a liability to record actual user requests (or implement something special to scrub out sensitive stuff) or let devs peek too close. You have a way of investigating some classes of issues without necessarily looking at sensitive data or requiring extensive back and forth in a support case.
whazor 4y ago
Don't forget that you need to disable CSRF checks, because the randomly generated tokens from prod vs dev would be different.

eyalbukchin 4y ago

Up to you: you’re going to be able to configure what works locally vs what works with the cloud environment. So for example, you could configure all read functionality (incoming traffic, DB reads) to work with the cloud, and all write functionality to work locally. Further down the line, we’re planning to support Copy On Write for databases - so you’d read from the cloud, write locally, then read the records you wrote locally (and records you haven’t written, from the cloud). Disclaimer: we’re still pretty early in the roadmap, so currently only support mirroring incoming traffic. But we’re moving fast :)

opportune 4y ago

I would generally use this for e2e testing since it's too expensive to run a 100% mirror of a large production environment. Well, depends on your scale and whether your production costs are a significant factor in your total expenses - I guess if your production costs are tiny compared to revenue/dev time spent debugging, you could justify a full mirror. For something like an eshop where revenue >> production costs you could probably justify cloning out production multiple times. Sampling is another useful tool but I don't see any sampling in the API for this. If you do sample though you could end up with inconsistent or divergent state if you just sampled a flat % of traffic, so in some contexts it makes sense for sampling to be determined per-user/tenant/etc. rather than per-request. This is pod based so you could easily slice by pod. If you have a pod with a lot less traffic, more impactful traffic, or that is harder to debug (example: pod that periodically aggregates and exports metrics but you believe it's dropping data sometimes) you could just enroll that and have some kind of dev shard of a portion of the stack. I could also see this being very useful if you believe it to be too big a liability to record actual user requests (or implement something special to scrub out sensitive stuff) or let devs peek too close. You have a way of investigating some classes of issues without necessarily looking at sensitive data or requiring extensive back and forth in a support case.

whazor 4y ago

Don't forget that you need to disable CSRF checks, because the randomly generated tokens from prod vs dev would be different.