3 ms·

Writing Decisions: Headline tests on the Highrise signup page

32 points by brm 18y ago

diN0bot 18y ago
the moral of the story is that advertising in place of information gets consumers/users to act.
- arockwell 18y ago
  I don't follow you... There's not much difference in the amount of information between the different versions.
- jwesley 18y ago
  Advertising is a medium of information, not a substitute for it. I think the moral is that people love free trials and it's essential to test your landing pages.
arockwell 18y ago
That's a huge difference in sales from subtle changes in wording and font sizes. Definitely shows the importance of A/B testing.
auston 18y ago
Here is similar article related to design - etc (http://www.boalt.com/blog/2008/10/great-design-is-a-bottom-line-business/ http://www.boalt.com/blog/2008/10/great-design-is-a-bottom-l...)
teej 18y ago
I'm glad that Jason has decided to start throwing data behind his design decisions. It's certainly easy to go overboard on data collection, but a little good data is better than no data at all. Quick story - a company I used to work for decided to A/B test their signup page. Everyone in the company could submit their "version" of the page. In the end, the best performer was the secretary, while the worst performer was the marketing guy.
- pstinnett 18y ago
  Agreed. It also gives a good place to point back to when recommending to go simple rather than extravagant on design decisions.
- staunch 18y ago
  Not surprising. To do sales well you really have to be able to step back and think like someone who knows nothing about what you're selling. Hard for someone who's been buried deep in a project for a long time. One of the reasons I love trade shows. By the end of a few days talking to potential customers at a booth you'll have a rock solid pitch.
vColin 18y ago
Interesting – I wonder what the variance in the “kind” of sign-up might be. Perhaps those designs that highlight the “free” aspects will encourage a lot of try-outs that then bail after the 30 days; while the designs that don’t will see more longer term sign-ups. The results of a follow-up after 30 days to see how many stay could also be enlightening.
aneesh 18y ago
One caveat in A/B tests -- you have to be careful in deciding what metric you use to pick the winner. They used the number of immediate conversions. But maybe they just increased signups of freeloaders who will abandon the account within 30 days. And maybe not. Another, possibly better, metric would be the number of paying customers after 30 days, rather the number of conversions today.
dpifke 18y ago
We played with multivariate testing at the last company I was at, using an expensive platform built by Optimost (now part of Interwoven): http://www.interwoven.com/components/pagenext.jsp?topic=PRODUCT::OPTIMOST http://www.interwoven.com/components/pagenext.jsp?topic=PROD... (We also paid a company that did optimization via Taguchi methods, and found them to be a bit of a scam.) Besides this and Google Website Optimizer, I'd be curious if anyone on HN has used other AB or multivariate testing tools? I would absolutely love something open source that I could plug into Django templates on the server side, rather than depend on external Javascript code to rejigger the page.

32 points by brm 18y ago

diN0bot 18y ago

the moral of the story is that advertising in place of information gets consumers/users to act.

arockwell 18y ago
I don't follow you... There's not much difference in the amount of information between the different versions.
jwesley 18y ago
Advertising is a medium of information, not a substitute for it. I think the moral is that people love free trials and it's essential to test your landing pages.

arockwell 18y ago

That's a huge difference in sales from subtle changes in wording and font sizes. Definitely shows the importance of A/B testing.

auston 18y ago

Here is similar article related to design - etc (http://www.boalt.com/blog/2008/10/great-design-is-a-bottom-line-business/ http://www.boalt.com/blog/2008/10/great-design-is-a-bottom-l...)

teej 18y ago

I'm glad that Jason has decided to start throwing data behind his design decisions. It's certainly easy to go overboard on data collection, but a little good data is better than no data at all. Quick story - a company I used to work for decided to A/B test their signup page. Everyone in the company could submit their "version" of the page. In the end, the best performer was the secretary, while the worst performer was the marketing guy.

pstinnett 18y ago
Agreed. It also gives a good place to point back to when recommending to go simple rather than extravagant on design decisions.
staunch 18y ago
Not surprising. To do sales well you really have to be able to step back and think like someone who knows nothing about what you're selling. Hard for someone who's been buried deep in a project for a long time. One of the reasons I love trade shows. By the end of a few days talking to potential customers at a booth you'll have a rock solid pitch.

vColin 18y ago

Interesting – I wonder what the variance in the “kind” of sign-up might be. Perhaps those designs that highlight the “free” aspects will encourage a lot of try-outs that then bail after the 30 days; while the designs that don’t will see more longer term sign-ups. The results of a follow-up after 30 days to see how many stay could also be enlightening.

aneesh 18y ago

One caveat in A/B tests -- you have to be careful in deciding what metric you use to pick the winner. They used the number of immediate conversions. But maybe they just increased signups of freeloaders who will abandon the account within 30 days. And maybe not. Another, possibly better, metric would be the number of paying customers after 30 days, rather the number of conversions today.

dpifke 18y ago

We played with multivariate testing at the last company I was at, using an expensive platform built by Optimost (now part of Interwoven): http://www.interwoven.com/components/pagenext.jsp?topic=PRODUCT::OPTIMOST http://www.interwoven.com/components/pagenext.jsp?topic=PROD... (We also paid a company that did optimization via Taguchi methods, and found them to be a bit of a scam.) Besides this and Google Website Optimizer, I'd be curious if anyone on HN has used other AB or multivariate testing tools? I would absolutely love something open source that I could plug into Django templates on the server side, rather than depend on external Javascript code to rejigger the page.