6 ms·
[Tl;dr on the below: the changes we make to Checkout are designed to increase our merchants' revenue. If a change doesn't do that, it will be reverted.] The co
by pc 13y ago
[Tl;dr on the below: the changes we make to Checkout are designed to increase our merchants' revenue. If a change doesn't do that, it will be reverted.]
The confusion point you cite was our biggest concern in building the "Remember me" functionality. On the one hand, it's clearly good for our merchants* if customers don't have to constantly retype their card details. On the other hand, it'd be bad if we were scaring them away with something confusing.
So we've been testing Checkout for months. After tons of testing across different sites, types of business, and devices, we're confident that "Remember me" does not harm conversion rate. On the contrary -- all the evidence we have shows that it increases it. If a business has a lot of returning customers, this effect can be very big. For example: we ran an experiment that simply removed the "Remember me" option on Humble Bundle, and determined that they would make less money without it.
To your point about being focused on customers: our rule is to always do what's best for the merchant. I want to really emphasize this point, because I believe it very strongly. The strategy tax example you cite is precisely what we insist we can't do, and where we believe other companies and products have gone wrong.
We've discussed the possibility of allowing users to hide the "Remember me" checkbox, and our belief is that having that option is probably not the best thing for the merchant. We consider it our job to optimize the Checkout in order to maximize our users' revenue. If we add more options, we can easily end up with something confusing and inconsistent. Having a ton of configuration options would also make it harder to pursue other ideas and avenues. The whole point of Checkout is to enable ongoing iteration and improvement. As a result, we'd much rather make the macro guarantee ("we'll work on increasing your revenue"), and build the infrastructure that enables us to do that, rather than forcing merchants to guess what particular tweaks might work best.
Still, even though we've leaned against doing so thus far, we're by no means strongly opposed to removing it. Perhaps, for example, we should have an option that doesn't show "Remember me" but will pre-fill the details if they've been entered elsewhere. Again -- our goal is just to do what's best for the merchant.
* We generally don't like the term merchant, but I'll use it here for clarity.
- pdb123 13y agoIt's awesome that you've tested Checkout and are confident that it helps conversion. Would you guys consider releasing the experiments that your ran? I think it'd be a great lesson in testing out products pre-launch.
- pc 13y agoYes, we'd love to write a blog post about this, and have considered doing something along these lines. Look out for more on this front.
- teej 13y agoI think the trap here, a function of scaling any business, is that what's best for most of your customers isn't necessarily best for all of your customers. Another perspective is that what's right for your 2013 customers isn't necessarily the right thing for your prospective 2014 customers. One possible outcome of your decision is that you're optimizing for customers who have repeat business and against customers with one-off business. That may well be right thing for Stripe. I am a strong supporter of opinionated software but I think that view has to be colored with "this solution will be wrong for some customers" and "we are not building a solution for every customer". I imagine these assumptions are implied in your explanation above.
- pc 13y agoFor sure. We haven't seen evidence yet that there are things that are amazing for some segments and bad for others -- but if we did (or if we do in future -- we're still measuring), that'd certainly inform our approach. Also, it's probably worth emphasizing that the traditional Stripe integration options are all still present and always will be.
- benmathes 13y agoYou could also automatically run split tests on a per-merchant basis, thereby automatically auto-tune the checkout for each merchant.
- pc 13y agoThat's a neat idea.
- benmathes 13y agoAnd you can use it for just 5% of any profits made on it. but seriously, have at it. Guaranteed there's prior art.
- Robin_Message 13y agoIf recurring vs not is important to how Remember Me is perceived, and you've been a/b testing it, you could probably test this particular hypothesis easily.
- thatthatis 13y agoWhy isn't it removable? I know things about my customers that you couldn't. This is the single biggest thing that makes me question whether I trust stripe long term. If you're going to make netwok decision like this now, where's your limit? It stops feeling like payments for developers and starts feeling like a "PayPal that hasn't started fucking over their merchants... yet"
- pc 13y ago> Why isn't it removable? Well, I tried to outline the thinking above, but we're not absolutely attached to it either. If there is some reason it's bad for your customers, we'd love to talk. Can you drop me an email? I'm patrick@stripe.com.
- filearts 13y agoWhat if you could request a semver range for the checkout js file? That way vendors could opt in to fixes, enhancements or major changes and be confident in the level of volatility in UX for their customers.
- pc 13y agoYes, this could be the right approach -- we should perhaps have something like HEAD and STABLE.
- deleted 13y ago[deleted]
- thatthatis 13y agoA few of the sites I'm involved with have very high trust requirements (think childcare) and low average sophistication users. My core user concern is: users losing trust in my site when they see their credit card info pre-filled but they haven't given it to me. Or, another site having cc info pre-filled when they only ever gave me their CC. My users shouldn't have to understand the stripe network model to not lose trust in my website. A site knowing, or claiming to know more than it reasonably should is a sign of fraud. In a high trust industry, you need to be very, very careful at the signals you're sending your users. Some of my users will get confused if they click "remember me" and think I stole their credit cards and sold it to other sites. From a developer perspective, what really concerns me is that this decision was and is clearly being made as "we'll do what's right for stripe and the stripe network in aggregate" not "we'll do right by each merchant." Given that you forced this into the product and then didn't even give an opt out, how can I trust that you're not going to make other changes? E.G. my users now have to create a stripe.com account to use the service in the future? Or, start heavily pushing users to pay by ACH (also something paypal has done)? I don't use paypal checkout because I don't want to force my users to have paypal accounts and I don't trust paypal because they have a long history of these kinds of decisions. That people are saying "that's the way paypal would add features to the product" should set off red flags and alarm bells in your CEO suite. Right now people are suggesting stripe with the same fervor they're telling their friends to never use paypal. That kind of fanaticism is hard to buy, and if you lose it, impossible to get back. Since you added this "feature," I've stopped suggesting stripe emphatically and without reservation. I just don't know if I'm willing to put my reputation next to your future decisions anymore. This hurts, emotionally, because I trusted you guys. And moreover, I was excited that there was finally a payments company that didn't suck. Now, I don't know if I was a rube for trusting and promoting you. The emotional process of going from excited evangelist to caution is worse than never being excited about something to begin with. As a payments provider you act as our agent in the cc world. This "new feature turned on network wide (1) by default (2) without a way to turn it off (3), because we want to test it (4)" makes me trust you less as an agent in 4 separate ways. I didn't opt into this, it just showed up one day with no explanation.
- baudehlo 13y agoThat philosophy works fine with merchants who hope for repeat sales from the same customer. But what about subscription payments (what I use Stripe for). A remember me button doesn't make sense there, as they are never re-entering the data to make another purchase. Perhaps you didn't test it with enough variety of merchants?
- pc 13y agoWe definitely tested it with merchants who take subscription payments. But, yeah, the considerations are different, I agree. The primary motivating use case for Checkout was "standard" checkouts rather than subscription flows (even though it works well for the latter). I suspect we'll take a look at redesigning that flow entirely at some stage, with a product that may look quite different to Checkout.
- baudehlo 13y ago>though it works well for the latter "worked well". Now everyone using Checkout for subscription flow has to do work to change it. You just cost your users a lot of hours work, and my site only brings in enough to pay for the hosting costs. Err, thanks?
- jorde 13y agoIt's great to hear that Stripe is iterating on products and personally I have been happy with Checkout in the past (I think we were the first site to integrated Checkout when @maccaw first build it). After reading other people's feedback here I totally understand their situation given that Stripe isn't a household name among consumers like Paypal (I hope it will be at some point but don't know how given the lack of consumer products). Why not just add "remember me" as the last step after customer has already inputted their information? This would not create any friction pre-payment and you would have more room to explain the feature. This would of course favor merchants and not Stripe, as you would probably gather less cookies. My 2 cents. PS. If Checkout is an experiment still, I would highlight that on documentation. One can't expect people to go and read announcement blog posts dating back a year.
- jusben1369 13y ago
- dirtae 13y agoWe do not have many returning customers, and our customers are not as tech savvy as Humble Bundle customers, by a long shot. We are using Stripe.js with our own form instead. It's not as nice as Checkout, and I really wish I could use Checkout instead, but as a developer I need to be in control of my user experience, and "Remember me" is not appropriate for my application.
- pc 13y agoWell, even if they're not returning for you, they might have entered their payment details on a different site. Out of curiosity, would you be open to running a test to see if "Remember me" has any impact in either direction?
- dirtae 13y agoMaybe, but I'm not sure that a test could change my mind, unless it resulted in a massive boost in revenue. Let's say that the vast majority of my customers do not want to use "Remember me", and that most of them examine the option and leave it unchecked, as is their desire. I've still introduced friction into my payment process, and possibly left a bad taste in the mouths of many of my customers. Maybe not enough that they stop doing business with me, but it could still taint my reputation in their eyes, as a company that is trying to push them down a path that they almost certainly don't want to take. Testing may indicate an increase in revenue, but it can't account for reputational damage, unless we're going to ask every customer how they felt about the checkout process after they're done. So, testing based on revenue alone isn't enough. There is probably plenty of testing that shows that adding an array of annoying up-sells during checkout increases revenue, but that doesn't mean it's something I want to do.
- pc 13y agoAny interest in coming by Stripe for lunch/dinner? I'd love to chat about this more and figure out a good way to accomplish this. (I.e., a product you'd be happy integrating that we iterate on over time.) I'm patrick@stripe.com.
- Chris_Newton 13y agoI’m going to respectfully disagree with almost everything there, based on our experience. TL;DR: The changes do not benefit us or our customers, caught us by surprise, and created customer support issues almost immediately. As a direct result, we are now working on moving to a stripe.js form where Stripe’s branding will be hidden as much as possible and we retain full control of our user experience. I do want to start with my usual caveat that while I’m discussing negatives here in the hope of promoting improvements, we’re generally positive about Stripe, and I’ve waxed lyrical about checkout.js specifically in the past. This appears to have been a blip and hasn’t particularly shaken our general confidence as Stripe customers. I should also acknowledge that I did promise to send the information below to one of Stripe’s people who replied to my e-mail several weeks ago, and I haven’t; mea culpa. I might as well put it here for general discussion at this point. So, we’re planning to dump Checkout for a few reasons, but first among them is that Stripe changed our users’ experience for the worse, without our knowledge or consent. That simply shouldn’t happen. I appreciate the desire to incrementally improve things, but not everyone will share the same views on what is an improvement, so pushing changes to our user experience in an uncontrolled way is not really acceptable to us. For example, our customers typically sign up once for a subscription and then never enter their card details again, so the remember-me changes do absolutely nothing positive for them or us. On the other hand, almost the first customer we had sign up after the changes went live then contacted us to say he’d put in an incorrect phone number and could we please change it to a different one he mailed to us. Our first reaction was that we didn’t collect phone numbers. This is how we discovered Checkout had been changed. Then we looked on the Stripe dashboard to see how to update the information, and it’s not there, so all I can do is mail the Stripe support team to ask for help (and, to be fair, they did, very quickly). So now I’ve gone from having a reasonably streamlined sign-up process to having a paying customer who is distressed because they’ve made a mistake and I can’t even fix it for them. Epic fail. We had similar reactions to the prompt for an e-mail address, when we had chance to watch some users signing up in person a few weeks ago (not watching them actually enter their card details, of course). They’d just entered their name, postal address and e-mail address on our create-account form, clicked through to pay by card, and now the next thing they get is a prompt for much the same details again! I think every person who signed up that day challenged something at this stage in the process, several being suspicious of spam. Perhaps the most surprising thing to me, as someone who’s seen overwhelmingly positive views of Stripe on forums like HN, is that the Stripe brand was a clear negative in the sign-up process. In the UK, Stripe is not a well-known organisation in the way that say a high street bank is, and we appear to be losing some level of business immediately because of the branding. For example, someone mailed us essentially asking why we’d let this Stripe organisation hijack our site. There was clearly some general uncertainty from that particular user — they also thought the Stripe form was not secure, because on their iPad it popped up in a separate tab and they didn’t see the padlock icon in the usual place — but uncertainty is to be expected from non-technical customers. I wanted to send them links to authoritative sources to prove that Stripe was legitimate, but this proved surprisingly difficult. Googling any likely variation of “Stripe PCI” or “Stripe security” turned up more negative links than positive ones on the first SERP (justified, correct, or otherwise, there they were). When I searched for “Stripe hack” I found numerous links to a site called Hacker News, which of course is no surprise to us but more so for our non-geek customers. When I went to look up Stripe’s entry in the Visa Global Registry, as linked from their own site (https://stripe.com/gb/help/faq#pci-compliance https://stripe.com/gb/help/faq#pci-compliance), it only showed them as operating in the US and Canada, but both we and our customer were in the UK. So the bottom line is that we’re going to move away from both Stripe-controlled UX (where the changes have been a clear negative in our case) and the Stripe branding that comes with Checkout. It’s a shame, because the idea is still a great one and obviously it’s more work for us to redo the integration, but when 100% of the customers you’re watching sign up don’t like something about your sign-up process...
- adventured 13y agoI think this is an excellent, well thought-out reply. It still doesn't matter if Remember Me is better for merchants or not. What matters is whether the merchant wants to be able to hide that feature or not. If they choose to shoot themselves in the foot, it should be their choice. There are only a few actually plausible reasons why a service would prevent turning this feature off. A nanny mentality (we know what's best for you), which is insulting to the merchant/s. Or because Stripe is pushing something primarily for their benefit, and all the 'explanations' are nothing but rationalizations to make that smell better, the merchant's wishes be damned. Both scenarios are bad.
- TheCoelacanth 13y agoIf they want to shoot themselves in the foot, can't they just make their own payment form while still using Stripe as the processor? That would allow practically any customization of the form that they want.
- nbouscal 13y agoThe plausible reason has been explained by pc like half a dozen times in this thread. Checkout is not the only option. You can build a form from scratch if you want customizability. Stripe is presenting developers with two options: one where they do the analysis and optimization and present you with a ready-to-go payment solution, and another where you can customize it to your heart's content. You (and others) are complaining because you can't customize the former option. This makes absolutely no sense.
- saurik 13y ago> We consider it our job to optimize the Checkout in order to maximize our users' revenue. Even if you ran a controlled study and determined that this checkbox increases conversion by 100x for 99% of your merchants, it is still quite possible that for someone out there with some business in some selected market it harms them (potentially enough to be devastating). There is a reason why PayPal has tons of tiny little options for manipulating parts of the display given to the user, and it isn't because they are somehow incompetent. Unless you are running, per merchant, some kind of machine learning algorithm that optimizes the display for them, thinking you can optimize the flow for everyone and then should purposely not provide options because the merchant might harm themselves if they don't listen to your advice is nothing but hubris.
- pc 13y agoI agree. I'd go further: even if we were running some kind of crazy ML on a per-merchant basis, I still don't think we could cater to every single use case. I don't think that any UI we build -- no matter how many knobs and whistles it has -- will handle everything. The Checkout page points out that you can still build every pixel of the UI yourself, and Stripe will always support that. Checkout is another option alongside the integration we've always supported, not a replacement.
- iopq 13y agoNo, PayPal is fucking incompetent. That's why they have like 4 APIs that do the same thing, except they don't have feature parity and you need a feature from the Pay API and another feature from Express Checkout and you can't do either one of those because PayPal sucks.
- EGreg 13y agoHere's a suggestion - why not move the "remember me" checkbox to the screen that appears AFTER the user submits the form? If they don't fill the form out, there will be nothing to remember. And when they do, they've invested enough time and effort to WANT to save their work. Plus, they got to the finish line quicker and with less distractions. In short, "remember me" should be done after the user presses a button to submit the form.