7 ms·
Expedia on how one extra data field can cost $12m
- wccrawford 16y agoIt occurs to me that instead of removing the field (which I'm sure people who used Company credits cards are looking for) they could simply have made it clear that it was the user's billing address they were looking for, instead of the bank's. It might have resulted in more revenue if they had done it my way, since there may be people on business trips that now think they can't use a company credit card. Or not. That's the thing with dealing with the public... Without paying really close attention to what's really happening and people are saying, you are just guessing.
- wizard_2 16y agoThey don't really care about the companies name or if it's a company credit card.
- niekmaas 16y agoWhat they should do is a randomized control trial in which they either keep the field and make its meaning more explicit or get rid of the field at all. Then they should look at the differences in sales by comparing the two groups. Nothing wrong with being a bit academic..
- dfox 16y agoProblem is, that this field is question is actually useful for some subset of clients. Making the intent of both "Company" and "billing address" fields is probably most meaningful way to go, possibly hiding the "Company" field behind checkbox like "Bill my employer".
- samd 16y agoI don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies.
- brianbreslin 16y agoWasn't there a famous post a few years back by amazon about their a/b tests resulting in 60million in sales increases, just by changing the button colors? Everyone working on ecommerce sites of this scale should be expected to know about this stuff
- patio11 16y agoI have spoken to a lot of companies who have A/B testing somewhere on their radar, but some combination of "That's nobody's job, so we haven't actually started doing it", (sometimes) organizational infighting (marketing wants to start, engineering has higher priorities and refuses to allocate resources to "twiddling text", etc), and uncertainty about outcomes means they don't start it yet. One of my big tasks at some consulting clients is convincing people to buy into A/B testing. The easy way is to show them how their website is leaking staggering sums of money, and present A/B testing as the cheap, reliable solution to that problem in the future. These are smart software companies who transact most of their business online, by the way. The broader economy which isn't primarily web-focused has many, many, many opportunities to improve their execution.
- leftnode 16y agoYup, it's shocking. However, is this more of a problem with A/B testing or just bad user experience/design? I suppose an A/B test would've been to remove the Company field from the payment process and see how things have performed. I think companies are more concerned with more obvious A/B tests (button color, page headings) than with testing large changes to their site or testing more in depth things like user experience.
- thibaut_barrere 16y agoA/B anything (testing or dual projects at once etc) is seen in some places as a "lack of skills, otherwise you would know beforehand". (which I don't agree with, in case someone doubts!)
- matwood 16y agoI don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies. Yeah, my company has been doing A/B (we call it Champion/Challenger) testing for years.
- al_james 16y agoNow, if only they fix all the other issues on their site. I tried twice at the weekend to make a booking and the whole convoluted process simply did not work (saying there is errors on the form even if there was not etc...).
- gaius 16y agoTo be fair there's absolutely no frickin' way that you could predict that people would enter their bank in place of their employer.
- eitland 16y agoNo, but there should be some way or another that those who create large, international websites should know that their websites are going to be used by non native english speakers :-) Still there is lots of web forms with a Country field and a State field. And in the state field you'll only find American states.
- whatusername 16y agoOr 5 digit "zip" codes. // I wonder how many Internet residents live in 90210? I know I do in a couple of places.
- smiler 16y ago:) Yup that is my standard zip code when I need it. What I find even more frustrating is that some US stores ask for your zip code when you make a purchase. (Is it ok to refuse?). Being English, I just state that I don't live in the US, but it's not a practice I've ever encountered in the UK.
- kenj0418 16y agoIt usually confuses them if you refuse. When they ask, I live in the zip code 90210 and my phone number is 867-5309.
- cullenking 16y agoOh that was mean mean mean. I don't know how long it will take that song to leave my head now :(
- CapitalistCartr 16y ago
- known 16y agoWhile designing the database, it is better to have a notes column in every table so that we can use it in future without changing the schema and breaking the application.
- kprobst 16y agoWow... seriously?
- epochwolf 16y agoWhy not? I'm a big proponent of the SQL K-X model. You have two text fields in each table: key and xml. The flexibility of this model is incredible.
- pavel_lishin 16y agoCan you go into some more detail to explain what you mean? I completely don't understand.
- qjz 16y agoConvenience is king and this is an excellent example of how easy it is to go too far when collecting data from users. I already expect to jump through a few hoops when using a credit card online, so asking me for my life story will only encourage me to bail out and never return.
- corin_ 16y agoYou've clearly not comprehended the subject article - I'd suggest reading it again.
- qjz 16y agoI did read the article, and the title says it all. They lost money because they got greedy for data that was unnecessary to complete the transaction. As a result, another point of failure was added and frustrated users bailed. The body of the article may try to spin it as a case where analytics boosted the bottom line, but the truth is that it was a bad design choice that cost them money from the start, because they ignored the KISS principle.
- corin_ 16y agoAccording to the article the sales they lose were due to customers who didn't understand the form and therefore filled in the wrong information, not because people couldn't be bother to provide the data.
- dmlorenzetti 16y agoThat $12m/yr might be added revenue, but it's hard to see how it could represent added profit (as claimed in the article). Say Expedia earns $200 profit per transaction (total WAG, but intended to sound high). Then $12e6 means 60,000 people a year - or 165 people a day - were failing to complete their transactions because they thought "company" meant "bank name". Similarly, $500 profit per transaction means 66 people a day were thrown off by an input field that's actually pretty common on the web. Neither of those seems even vaguely plausible to me. Take another tack. According to this article (http://www.reuters.com/article/idUSTRE69R56X20101028 http://www.reuters.com/article/idUSTRE69R56X20101028), Expedia had $177m net income this past quarter. That's framed as a big jump from the past, but say they make 4 x $175m or $700m a year. Before this change, they would have made $688m per year. So 12/688, or 1.7%, of their customers were being thrown off by the "company" field? I find that hard to believe. If, on the other hand, that $12m is added revenue, then (taking the article's $6.89b revenue per quarter figure) we have 0.044% of their customers failing to complete due to this problem. That sounds a bit more plausible.
- aresant 16y agoDon't forget Amazon's $300,000,000 button: http://www.uie.com/articles/three_hund_million_button/ http://www.uie.com/articles/three_hund_million_button/
- dcnstrct 16y agoFor those curious Expedia actually captures a stupendous amount of information down to session level activity. They can play back failed sessions to discover areas of opportunity. This was discussed at an analytics conference I attended on how they implement testing across a range of systems to find possible areas of friction and improve the business.
- afterburner 16y agoI wonder what the percentage was of customers who made that mistake and then corrected it in a second try. The increase in sales after correcting the problem implies there were plenty who didn't, but how many were stubborn enough to try again after making the bank mistake?
- jwcacces 16y agoConfusing customers costs money: news at 11