4 ms·

From my experience building medium scale ecommerce systems, along with innumerate payment integrations of various flavours, this isn’t unreasonable, for a few r

From my experience building medium scale ecommerce systems, along with innumerate payment integrations of various flavours, this isn’t unreasonable, for a few reasons. Firstly, payment service providers honestly suck at providing a coherent staging environment. Either it’ll be out of date, or ahead of production, or full of garbage data that you can’t clear that breaks their outputs, or just plain not representative of the production environment. You’ll have stuff check out perfectly in staging only to be a hot mess on their live environment. Secondly, if you’re doing this stuff at scale, it’s not as simple as “make an API call and get a result” - you’ve got your egress and ingress to worry about, at different levels (NAT, load balancing, packet routing, http(s) proxies), and there’s a host of stuff that can go wrong for subtle reasons. We used to (for they are now just a shopify shop since my departure a decade ago) do exactly as is described - test in staging as much as it is useful, and then go live with an immediate test built into the deployment toolchain, with automatic rollback in case of failure for any reason. It worked. The only payment issues we ever had after having the realisation that testing on staging was damned near meaningless, were on the side of the payment gateway.

michaelt 2y ago
> test in staging as much as it is useful, and then go live with an immediate test built into the deployment toolchain, with automatic rollback in case of failure for any reason I'm curious about the logistics of automatically testing payments after a deployment? Does your automated test place an order with a valid, working credit card? Does your test include going through 3D Secure too? Do you then automatically cancel the order? How do you make sure that whole unusual process doesn't get blocked by unusual activity fraud detection? Whose credit card is it? Have broken tests ever lead to the test order getting fulfilled?
- madaxe_again 2y ago
  > Does your automated test place an order with a valid, working credit card? Yep. Organisation owned card for the purpose, details used by selenium for the test. Details stored securely, I might add, as PCI/DSS and ISO27k1 were important to us. > Does your test include going through 3D Secure too? Yeah. Same bank card always being used meant we could automate the flow. > Do you then automatically cancel the order? It went through the whole despatch process, including label production with couriers etc., and was then cancelled as that tests everything including CANCEL/VOID and the whole critical flow. > How do you make sure that whole unusual process doesn't get blocked by unusual activity fraud detection? By using the same card over and over, placing an order for a normal item, talking to our bank when it did occasionally get flagged. > Whose credit card is it? The businesses. > Have broken tests ever lead to the test order getting fulfilled? Yup. We had a few that appeared on our doorstep, both due to our error, and client error.
altacc 2y ago
People are calling this article fluffy but I agree with you that if people haven't worked with a lot of payment gateways before then it's good advice never to fully trust the test environments. Many years ago I had a big launch turn to chaos when test worked perfectly but the payment gateway's validation on production turned out to be different and declined every payment without returning a meaningful error. A lot of work has gone into getting a customer to make a purchase, so it's the worst time to fail. Nothing beats testing on production with a real card.

michaelt 2y ago

> test in staging as much as it is useful, and then go live with an immediate test built into the deployment toolchain, with automatic rollback in case of failure for any reason I'm curious about the logistics of automatically testing payments after a deployment? Does your automated test place an order with a valid, working credit card? Does your test include going through 3D Secure too? Do you then automatically cancel the order? How do you make sure that whole unusual process doesn't get blocked by unusual activity fraud detection? Whose credit card is it? Have broken tests ever lead to the test order getting fulfilled?

madaxe_again 2y ago
> Does your automated test place an order with a valid, working credit card? Yep. Organisation owned card for the purpose, details used by selenium for the test. Details stored securely, I might add, as PCI/DSS and ISO27k1 were important to us. > Does your test include going through 3D Secure too? Yeah. Same bank card always being used meant we could automate the flow. > Do you then automatically cancel the order? It went through the whole despatch process, including label production with couriers etc., and was then cancelled as that tests everything including CANCEL/VOID and the whole critical flow. > How do you make sure that whole unusual process doesn't get blocked by unusual activity fraud detection? By using the same card over and over, placing an order for a normal item, talking to our bank when it did occasionally get flagged. > Whose credit card is it? The businesses. > Have broken tests ever lead to the test order getting fulfilled? Yup. We had a few that appeared on our doorstep, both due to our error, and client error.

altacc 2y ago

People are calling this article fluffy but I agree with you that if people haven't worked with a lot of payment gateways before then it's good advice never to fully trust the test environments. Many years ago I had a big launch turn to chaos when test worked perfectly but the payment gateway's validation on production turned out to be different and declined every payment without returning a meaningful error. A lot of work has gone into getting a customer to make a purchase, so it's the worst time to fail. Nothing beats testing on production with a real card.