5 ms·

> ..don't test production checkout flows.. That's good advice. Payment providers have test gateways and cards for you to test your code against. While it migh

> ..don't test production checkout flows.. That's good advice. Payment providers have test gateways and cards for you to test your code against. While it might be a minor convenience for you to have 'fake' cards in their production system, the only thing they have to gain is a potentially serious fraud loophole (at best), or an expensive footgun for you. Don't "test" checkout flows in production unless you're using real credit cards :D

rblatz 7y ago
Agreed, test environments exist for a reason, if you don’t trust your test environment to accurately replicate prod then you need to clean that up.
- edwinwee 7y ago
  Yes, testing in production can be dangerous and Stripe purposefully designed as many testing scenarios we could think of to avoid that (https://stripe.com/docs/testing https://stripe.com/docs/testing).
  - rblatz 7y ago
    Yep, I oversaw the implementation of the stripe connect api into our product. You guys have been great to work with.
- seanwilson 7y ago
  Do you mean this in general or for Stripe? For the former, you have to run some tests on production after initial setup and major changes surely? API keys could be wrong for a start and a lot of web services don't give you a programmatic way to sync settings between environments.
  - OJFord 7y ago
    If you want to do some sanity checking in prod to rule out any issue with the test environment setup... Why would you want to set up a special fake test card in prod? Either don't do it, or have everything about it real. e.g. a canary purchase test & then a refund flow test. Yes there's a fee, but there's a fee when you do it for real. Hopefully you have many more real transactions.
octocode 7y ago
I used to run a company card for testing Stripe on prod, and then just processed a refund afterward. It was mildly inconvenient, and eventually I just stopped because there was never actually a difference between their staging and prod behaviour.
Silhouette 7y ago
Payment providers have test gateways and cards for you to test your code against. And yet in 2020, you still can't write an automated test suite that spins up a virgin test environment, simulates all relevant scenarios, and allows you to quickly verify that your integration is responding properly as part of your normal CI process. Working with a single, persistent test environment that offers no facilities to manage events like simulated user actions, API requests and webhooks feels like programming in another, much less productive era. Sadly, this still seems to be the norm for online payment services. It's particularly ironic that online payment services are among the worst offenders for making API changes that require significant changes to integrations or even a whole new integration, and that by their nature they are also at high risk of Very Bad Things happening if an integration breaks. This is exactly the sort of situation where you really want a tight feedback loop and ongoing automated integration testing! Stripe used to be a welcome exception, but even they have dropped the ball badly in recent times.
- polemic 7y ago
  I don't quite understand this comment. In my experience, payment processors (and related services) are somewhat unique in providing test gateways & APIs. Unlike other API services you can write integration tests against their test gateways. What are part of our industries is that normal? Most API-first services don't provide test endpoints at all.
  - Silhouette 7y ago
    In my experience, payment processors (and related services) are somewhat unique in providing test gateways & APIs. They are also somewhat unique in that they handle real money. You don't need a simulated API if you're sending an email or SMS message to a designated recipient or downloading the local weather forecast or traffic news, because there are no significant and irreversible consequences to these actions anyway.
- verletx64 7y ago
  You can’t create an entirely new sandbox, no, but you can, quite easily test most scenarios. Including weird stuff like falling into delinquency - it takes a little bit more thought, but it’s fine.
derrick_jensen 7y ago
One feature I would like to see is the option to create multiple test environments. I have one dedicated for the CICD, and ideally each person testing locally would have their own API key they can use however they wish.

rblatz 7y ago

Agreed, test environments exist for a reason, if you don’t trust your test environment to accurately replicate prod then you need to clean that up.

edwinwee 7y ago
Yes, testing in production can be dangerous and Stripe purposefully designed as many testing scenarios we could think of to avoid that (https://stripe.com/docs/testing https://stripe.com/docs/testing).
- rblatz 7y ago
  Yep, I oversaw the implementation of the stripe connect api into our product. You guys have been great to work with.
seanwilson 7y ago
Do you mean this in general or for Stripe? For the former, you have to run some tests on production after initial setup and major changes surely? API keys could be wrong for a start and a lot of web services don't give you a programmatic way to sync settings between environments.
- OJFord 7y ago
  If you want to do some sanity checking in prod to rule out any issue with the test environment setup... Why would you want to set up a special fake test card in prod? Either don't do it, or have everything about it real. e.g. a canary purchase test & then a refund flow test. Yes there's a fee, but there's a fee when you do it for real. Hopefully you have many more real transactions.

octocode 7y ago

I used to run a company card for testing Stripe on prod, and then just processed a refund afterward. It was mildly inconvenient, and eventually I just stopped because there was never actually a difference between their staging and prod behaviour.

Silhouette 7y ago

Payment providers have test gateways and cards for you to test your code against. And yet in 2020, you still can't write an automated test suite that spins up a virgin test environment, simulates all relevant scenarios, and allows you to quickly verify that your integration is responding properly as part of your normal CI process. Working with a single, persistent test environment that offers no facilities to manage events like simulated user actions, API requests and webhooks feels like programming in another, much less productive era. Sadly, this still seems to be the norm for online payment services. It's particularly ironic that online payment services are among the worst offenders for making API changes that require significant changes to integrations or even a whole new integration, and that by their nature they are also at high risk of Very Bad Things happening if an integration breaks. This is exactly the sort of situation where you really want a tight feedback loop and ongoing automated integration testing! Stripe used to be a welcome exception, but even they have dropped the ball badly in recent times.

polemic 7y ago
I don't quite understand this comment. In my experience, payment processors (and related services) are somewhat unique in providing test gateways & APIs. Unlike other API services you can write integration tests against their test gateways. What are part of our industries is that normal? Most API-first services don't provide test endpoints at all.
- Silhouette 7y ago
  In my experience, payment processors (and related services) are somewhat unique in providing test gateways & APIs. They are also somewhat unique in that they handle real money. You don't need a simulated API if you're sending an email or SMS message to a designated recipient or downloading the local weather forecast or traffic news, because there are no significant and irreversible consequences to these actions anyway.
verletx64 7y ago
You can’t create an entirely new sandbox, no, but you can, quite easily test most scenarios. Including weird stuff like falling into delinquency - it takes a little bit more thought, but it’s fine.

derrick_jensen 7y ago

One feature I would like to see is the option to create multiple test environments. I have one dedicated for the CICD, and ideally each person testing locally would have their own API key they can use however they wish.