10 ms·

“Merry Christmas" vs. "Happy Holidays" - nearly double the conversion rate.

117 points by aresant 16y ago

Vivtek 16y ago
Now that's interesting. I wonder if it's because businesses tend to use HH while personal mail tends towards MC? I just love to see this kind of experimental results.
jordan0day 16y ago
That's a pretty interesting finding, I wonder if anyone else has done similar testing and would care to share their results? Also, I can't help but grin about "conversion" and the religiously-tinged "Merry Christmas" vs. "Happy Holidays" debate. Perhaps the already-converted are more likely to... convert?
- fecklessyouth 16y ago
  Or maybe Jesus rewards those who keep "Christ in Christmas" with higher conversion rates.
jeffreyg 16y ago
And I bet if you sent 'Happy Hanukkah' vs. 'Happy Holidays' you'd get much more traffic from Jewish recipients. Repeat with any holiday specific to a group.
- aresant 16y ago
  That’s a great point – clearly the demographic you’re sending to will influence the results. The experiment was, however, about as controlled for general USA populous as we were able to do - big box retailer, drives most customer acquisition via SEM / SEO so these type of results should be duplicatable if your customers find you through search engines.
- electromagnetic 16y ago
  The conversion rate could be much better because people rarely see Merry Christmas any more. I'd have bought something from Walmart if they just said 'Merry Christmas' over 'Happy Holidays'. I'm sorry, I'm not celebrating 'happy holidays' I'm celebrating mother fucking Christmas. I'm not religious, it's just the goddamn holiday. I don't say 'happy holidays' to a Jew, because it's flat out rude. They're Jewish, it's their religious holiday and I respect that. I'll say happy Kwanzaa if I knew someone celebrated it. I'll say happy noodle day to a spaghetti monsterist. I like respect. If I'm spending money, I want some damn respect or I'll go to the little store downtown and buy something that's actually celebrating the holiday I'm buying for: Christmas.
  - DannoHung 16y ago
    So if it's before/during Hannukah and you're not sure if they're Jewish or not... Oh, and what if you're talking to a Pagan? Do you identify them by the sacrificial blood they are so clearly wearing?
    - electromagnetic 16y ago
      I usually detect my Pagans with a divining rod, I find it the much more applicable way. Similarly I detect Jews by throwing ham at people and judging their reaction, it becomes a little difficult with the Arab Israelis, but you can judge them by carrying around a selection of Halal non-Kosher products. Note: This method isn't 100% accurate due to the lack of verified studies into the field of divination, and the fact that a large subset of people would hit you if you pelt them with seemingly random foods.
      - HeyLaughingBoy 16y ago
        Considering that the person who introduced me to bacon and garlic pizza was Jewish, I can attest that this method does not always work.
  - ceejayoz 16y ago
    > I'm not religious, it's just the goddamn holiday. They're saying "Happy Holidays", not "Happy Holiday". There are all sorts of holidays in December, including New Years.
- jawee 16y ago
  I know about five Jewish families that also celebrate Christmas as a non-religious holiday--the only Jewish families I know. I´d say the secular Christmas holiday is common enough in the American culture to not offend all that many people.
jrockway 16y ago
I think people think the "Merry Christmas $name" email is not spam (it looks like a message from a friend), so they open and read. "Happy Holidays" is something that only people trying to sell you something say, so you mentally filter that email. The conversion rate is just a depressing example of how people will buy anything you tell them to. Once you get people to read your email, you'll much more likely to convert. (You get Viagra spam because somebody clicks it and buys.) This is basically why I give everyone a unique email address, and /dev/null the address after I stop needing it. People like the author of this post try to abuse my attention, and that is not OK with me.
- jordan0day 16y ago
  While I very much agree with your sentiment (email marketing is usually a bad thing), this article seems like a small example of "psychological hacking" and I hate to admit how interesting this sort of thing really is. It frightens me to know that there are people who spend as much time optimizing the performance of their marketing messages as we do on algorithms. The key difference (at least this is what I tell myself) is that while my work improves the actual product, the marketers work does no such thing.
  - space-monkey 16y ago
    Two thoughts: For many consumer products, the value of the product is the experience created by the product in conjunction with the marketing. I think hackers tend to focus to exclusion on "core" or "objective" value. If the consumer market thought like that, everyone would be buying off-brand cola. Second, I would imagine there's at least one order of magnitude more effort going into marketing optimization than there is into computer algorithms.
  - nitrogen 16y ago
    Even after reading HN for 662 days now, I still have to force myself to concede that not everyone is motivated by product quality or code cleverness. It still feels immoral to me for a company to sell an inferior product with superior marketing, but I have to ask myself: "Who am I to argue with what people want?" Or, more technically, "Who am I to argue with the neurochemistry of the majority of the population?"
elvirs 16y ago
I hear Happy New Year more than Happy Holidays
mindstab 16y ago
Google agrees, "Merry Christmas" reigns supreme: http://www.google.com/trends?q=merry+christmas,+happy+holidays http://www.google.com/trends?q=merry+christmas,+happy+holida...
civilian 16y ago
"Happy holidays" smacks of politically-correct pandering and pussyfooting. (I'm a Christmas-celebrating atheist.) Merry Christmas, HN!
- tptacek 16y ago
  I'm a Christmas-celebrating Catholic and I say "Happy Holidays". This notion that the words "Happy Holidays" are a political statement is bullshit. People said them long before Bill O'Reilly declared that there was a war on Christmas. It's not "politically incorrect" or offensive to say "Merry Christmas" to someone who isn't celebrating Christmas (or, as is far more likely now, is instead preparing to celebrate New Years). It's stupid.
  - walkon 16y ago
    Saying "Happy Holidays" to someone who isn't celebrating any holidays this time of year (e.g. a Chinese Buddhist) is also stupid.
    - tptacek 16y ago
      If I was in China, I wouldn't say it. In North America, I'm safe in assuming they're at least celebrating New Years, because they're probably getting the day off.
- ido 16y ago
  I'm a (non-practicing) Jew and I appreciate it when people acknowledge I might be celebrating a different holiday than Christmas.
- bitskits 16y ago
  I'm an atheist who feels the opposite. I'd much rather have someone wish me "Happy Holidays". Assuming people celebrate a certain (your) religious holiday is presumptuous, IMO. Aside from that, with "Happy Holidays", New Year's is included along with Christmas, Hanukah, Festivus, Kwanzaa and all others. Why leave out new year's? Headline: THE WAR ON NEW YEARS HAS BEGUN! :)
btilly 16y ago
An important note about email conversion rates. There is a conversion funnel to actual paying customer. For this kind of test they are testing conversion from one step to the next. At other steps in funnels that I've seen, I've found that the increase in conversion at that step goes fairly straightforwardly to conversion to paying customer. However for email in particular at a couple of different places I've seen that improvements in clickthrough rates do not convert linearly to improvements in revenue. While 100% more people may have clicked through, I'd be willing to bet that actual purchases went up by substantially less than 100%. This is not to discount the value of, say, a 20% increase in sales. But you really should measure that instead of clicks. (That said, people like this company like to measure clicks because it is very easy to do so.)
- aresant 16y ago
  Valid point. In this case the results were nearly linear, this is addressed in the article: "While we haven’t included the buy rate in the chart above due to client confidentiality, the results were similarly impressive."
  - reinhardt 16y ago
    "similarly impressive" is deliberately vague; I certainly wouldn't take it to imply nearly linear.
- jeffreyrusso 16y ago
  Very important point. Driving increases in CTR is a million times easier than driving increases in actual conversion rate. While the author mentioned that they saw "similarly impressive" increases in conversion, that isn't the norm. In a lot of cases, the extra traffic drawn in by jumping through hoops to increase CTR converts at a proportionately poorer rate - it's fickle, poor quality traffic.
deleted 16y ago
[deleted]
tejaswiy 16y ago
I'm possibly wrong, but for me, Merry Christmas has a direct association with buying, sales, or e-mails about a sale etc., while happy holidays doesn't conjure up the same meaning, and hence a higher probability of me opening an e-mail? Maybe fewer percentage of the people even opened the e-mail and the click-through percentage of the people that actually opened the e-mail was the same?
rwhitman 16y ago
Um, what was the unsubscribe rate on this particular campaign? While they converted 16% they probably alienated another 16%
kmfrk 16y ago
Just to take a step away from the particular discussion at hand, I found the very similar season greetings cluttering my inbox to be a nuisance. It's worth remembering that the conversion rate only applies to the people who clicked at all - remember the surly curmudgeons like us who don't. :) Just before you think about applying this to your business.
jimmyjazz14 16y ago
"Merry Christmas" always seemed more personal to me than "Happy Holidays". Family members wish me "Merry Christmas", corporations wish me "Happy Holidays". Its not really a religious thing in my mind its more a matter of an meaningful gesture vs an empty statement.
alanh 16y ago
Who cares about offending people who consider Christmas exclusionary when your CTR doubles? Most Americans still consider Christmas a “religious holiday” so this email means, to most people, your company is “picking” Christianity. I personally disagree with this practice. There’s enough Christian privilege to go around as it is.
dfhjtyjyt 16y ago
http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX
qeorge 16y ago
One problem: you did this test on Dec 21, well after Hanukkah had passed (it was December 1-9 this year). So for a big portion of the folks who are supposed to appreciate your PC-ness you missed the holiday by almost a month. Hardly surprising the remaining segment liked "Merry Christmas" better.
- aresant 16y ago
  Lots and lots more agnostics and aethiests than Jewish people in the USA: http://en.wikipedia.org/wiki/Religion_in_the_United_States#Main_religious_preferences_of_Americans http://en.wikipedia.org/wiki/Religion_in_the_United_States#M... But fair point - would be interesting to test a happy holidays message earlier in the season.
  - qeorge 16y ago
    That's fine, but plenty of atheists/agnostics/other non-Christian folk celebrate Christmas (at least in the gift buying sense, which is what you care about). So for people who do celebrate Christmas, regardless of religion, of course the more specific "Merry Christmas" would beat the generic "Happy Holidays." Of the folks who don't celebrate Christmas, but do celebrate a gift-buying holiday, Hannukah is pretty important. My gut tells me that Jewish folks are a big portion of this segment, and running the experiment 3 weeks after Hannukah would pretty much kill any chance of conversion for them. Still, the takeaway from your experiment is solid: if you're running an email campaign 4 days before a major holiday, go ahead and focus on those who celebrate it.
- deleted 16y ago
  [deleted]
jordanlyall 16y ago
What the writer is referring to is the "Click Through Rate". A "conversion" is when the user takes an action post click (submits info, downloads, purchases, etc.). He also incorrectly calculate the CTR. The CTR on an email campaign is clicks divided by opens. He divided clicks over sends, which since fewer people opened the "Happy Holidays" email, fewer would have naturally clicked. "Merry Christmas" still performed better, but not at the rates he's suggesting.
warrenwilkinson 16y ago
'Happy Christmas' remains awkward.
- waqf 16y ago
  "Happy Christmas" is accepted in the UK, just not in the US.
Misha_B 16y ago
From 30 Rock: HAPPY HOLIDAYS is what terrorists say. Merry Christmas, Avery and Jack.

117 points by aresant 16y ago

Now that's interesting. I wonder if it's because businesses tend to use HH while personal mail tends towards MC? I just love to see this kind of experimental results.

jordan0day 16y ago

That's a pretty interesting finding, I wonder if anyone else has done similar testing and would care to share their results? Also, I can't help but grin about "conversion" and the religiously-tinged "Merry Christmas" vs. "Happy Holidays" debate. Perhaps the already-converted are more likely to... convert?

fecklessyouth 16y ago
Or maybe Jesus rewards those who keep "Christ in Christmas" with higher conversion rates.

jeffreyg 16y ago

And I bet if you sent 'Happy Hanukkah' vs. 'Happy Holidays' you'd get much more traffic from Jewish recipients. Repeat with any holiday specific to a group.

aresant 16y ago
That’s a great point – clearly the demographic you’re sending to will influence the results. The experiment was, however, about as controlled for general USA populous as we were able to do - big box retailer, drives most customer acquisition via SEM / SEO so these type of results should be duplicatable if your customers find you through search engines.
electromagnetic 16y ago
The conversion rate could be much better because people rarely see Merry Christmas any more. I'd have bought something from Walmart if they just said 'Merry Christmas' over 'Happy Holidays'. I'm sorry, I'm not celebrating 'happy holidays' I'm celebrating mother fucking Christmas. I'm not religious, it's just the goddamn holiday. I don't say 'happy holidays' to a Jew, because it's flat out rude. They're Jewish, it's their religious holiday and I respect that. I'll say happy Kwanzaa if I knew someone celebrated it. I'll say happy noodle day to a spaghetti monsterist. I like respect. If I'm spending money, I want some damn respect or I'll go to the little store downtown and buy something that's actually celebrating the holiday I'm buying for: Christmas.
- DannoHung 16y ago
  So if it's before/during Hannukah and you're not sure if they're Jewish or not... Oh, and what if you're talking to a Pagan? Do you identify them by the sacrificial blood they are so clearly wearing?
  - electromagnetic 16y ago
    I usually detect my Pagans with a divining rod, I find it the much more applicable way. Similarly I detect Jews by throwing ham at people and judging their reaction, it becomes a little difficult with the Arab Israelis, but you can judge them by carrying around a selection of Halal non-Kosher products. Note: This method isn't 100% accurate due to the lack of verified studies into the field of divination, and the fact that a large subset of people would hit you if you pelt them with seemingly random foods.
    - HeyLaughingBoy 16y ago
      Considering that the person who introduced me to bacon and garlic pizza was Jewish, I can attest that this method does not always work.
- ceejayoz 16y ago
  > I'm not religious, it's just the goddamn holiday. They're saying "Happy Holidays", not "Happy Holiday". There are all sorts of holidays in December, including New Years.
jawee 16y ago
I know about five Jewish families that also celebrate Christmas as a non-religious holiday--the only Jewish families I know. I´d say the secular Christmas holiday is common enough in the American culture to not offend all that many people.

jrockway 16y ago

I think people think the "Merry Christmas $name" email is not spam (it looks like a message from a friend), so they open and read. "Happy Holidays" is something that only people trying to sell you something say, so you mentally filter that email. The conversion rate is just a depressing example of how people will buy anything you tell them to. Once you get people to read your email, you'll much more likely to convert. (You get Viagra spam because somebody clicks it and buys.) This is basically why I give everyone a unique email address, and /dev/null the address after I stop needing it. People like the author of this post try to abuse my attention, and that is not OK with me.

jordan0day 16y ago
While I very much agree with your sentiment (email marketing is usually a bad thing), this article seems like a small example of "psychological hacking" and I hate to admit how interesting this sort of thing really is. It frightens me to know that there are people who spend as much time optimizing the performance of their marketing messages as we do on algorithms. The key difference (at least this is what I tell myself) is that while my work improves the actual product, the marketers work does no such thing.
- space-monkey 16y ago
  Two thoughts: For many consumer products, the value of the product is the experience created by the product in conjunction with the marketing. I think hackers tend to focus to exclusion on "core" or "objective" value. If the consumer market thought like that, everyone would be buying off-brand cola. Second, I would imagine there's at least one order of magnitude more effort going into marketing optimization than there is into computer algorithms.
- nitrogen 16y ago
  Even after reading HN for 662 days now, I still have to force myself to concede that not everyone is motivated by product quality or code cleverness. It still feels immoral to me for a company to sell an inferior product with superior marketing, but I have to ask myself: "Who am I to argue with what people want?" Or, more technically, "Who am I to argue with the neurochemistry of the majority of the population?"

elvirs 16y ago

I hear Happy New Year more than Happy Holidays

mindstab 16y ago

Google agrees, "Merry Christmas" reigns supreme: http://www.google.com/trends?q=merry+christmas,+happy+holidays http://www.google.com/trends?q=merry+christmas,+happy+holida...

civilian 16y ago

"Happy holidays" smacks of politically-correct pandering and pussyfooting. (I'm a Christmas-celebrating atheist.) Merry Christmas, HN!

tptacek 16y ago
I'm a Christmas-celebrating Catholic and I say "Happy Holidays". This notion that the words "Happy Holidays" are a political statement is bullshit. People said them long before Bill O'Reilly declared that there was a war on Christmas. It's not "politically incorrect" or offensive to say "Merry Christmas" to someone who isn't celebrating Christmas (or, as is far more likely now, is instead preparing to celebrate New Years). It's stupid.
- walkon 16y ago
  Saying "Happy Holidays" to someone who isn't celebrating any holidays this time of year (e.g. a Chinese Buddhist) is also stupid.
  - tptacek 16y ago
    If I was in China, I wouldn't say it. In North America, I'm safe in assuming they're at least celebrating New Years, because they're probably getting the day off.
ido 16y ago
I'm a (non-practicing) Jew and I appreciate it when people acknowledge I might be celebrating a different holiday than Christmas.
bitskits 16y ago
I'm an atheist who feels the opposite. I'd much rather have someone wish me "Happy Holidays". Assuming people celebrate a certain (your) religious holiday is presumptuous, IMO. Aside from that, with "Happy Holidays", New Year's is included along with Christmas, Hanukah, Festivus, Kwanzaa and all others. Why leave out new year's? Headline: THE WAR ON NEW YEARS HAS BEGUN! :)

btilly 16y ago

An important note about email conversion rates. There is a conversion funnel to actual paying customer. For this kind of test they are testing conversion from one step to the next. At other steps in funnels that I've seen, I've found that the increase in conversion at that step goes fairly straightforwardly to conversion to paying customer. However for email in particular at a couple of different places I've seen that improvements in clickthrough rates do not convert linearly to improvements in revenue. While 100% more people may have clicked through, I'd be willing to bet that actual purchases went up by substantially less than 100%. This is not to discount the value of, say, a 20% increase in sales. But you really should measure that instead of clicks. (That said, people like this company like to measure clicks because it is very easy to do so.)

aresant 16y ago
Valid point. In this case the results were nearly linear, this is addressed in the article: "While we haven’t included the buy rate in the chart above due to client confidentiality, the results were similarly impressive."
- reinhardt 16y ago
  "similarly impressive" is deliberately vague; I certainly wouldn't take it to imply nearly linear.
jeffreyrusso 16y ago
Very important point. Driving increases in CTR is a million times easier than driving increases in actual conversion rate. While the author mentioned that they saw "similarly impressive" increases in conversion, that isn't the norm. In a lot of cases, the extra traffic drawn in by jumping through hoops to increase CTR converts at a proportionately poorer rate - it's fickle, poor quality traffic.

deleted 16y ago

[deleted]

tejaswiy 16y ago

I'm possibly wrong, but for me, Merry Christmas has a direct association with buying, sales, or e-mails about a sale etc., while happy holidays doesn't conjure up the same meaning, and hence a higher probability of me opening an e-mail? Maybe fewer percentage of the people even opened the e-mail and the click-through percentage of the people that actually opened the e-mail was the same?

rwhitman 16y ago

Um, what was the unsubscribe rate on this particular campaign? While they converted 16% they probably alienated another 16%

kmfrk 16y ago

Just to take a step away from the particular discussion at hand, I found the very similar season greetings cluttering my inbox to be a nuisance. It's worth remembering that the conversion rate only applies to the people who clicked at all - remember the surly curmudgeons like us who don't. :) Just before you think about applying this to your business.

jimmyjazz14 16y ago

"Merry Christmas" always seemed more personal to me than "Happy Holidays". Family members wish me "Merry Christmas", corporations wish me "Happy Holidays". Its not really a religious thing in my mind its more a matter of an meaningful gesture vs an empty statement.

alanh 16y ago

Who cares about offending people who consider Christmas exclusionary when your CTR doubles? Most Americans still consider Christmas a “religious holiday” so this email means, to most people, your company is “picking” Christianity. I personally disagree with this practice. There’s enough Christian privilege to go around as it is.

dfhjtyjyt 16y ago

http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX http://✂.ws/yX

qeorge 16y ago

One problem: you did this test on Dec 21, well after Hanukkah had passed (it was December 1-9 this year). So for a big portion of the folks who are supposed to appreciate your PC-ness you missed the holiday by almost a month. Hardly surprising the remaining segment liked "Merry Christmas" better.

aresant 16y ago
Lots and lots more agnostics and aethiests than Jewish people in the USA: http://en.wikipedia.org/wiki/Religion_in_the_United_States#Main_religious_preferences_of_Americans http://en.wikipedia.org/wiki/Religion_in_the_United_States#M... But fair point - would be interesting to test a happy holidays message earlier in the season.
- qeorge 16y ago
  That's fine, but plenty of atheists/agnostics/other non-Christian folk celebrate Christmas (at least in the gift buying sense, which is what you care about). So for people who do celebrate Christmas, regardless of religion, of course the more specific "Merry Christmas" would beat the generic "Happy Holidays." Of the folks who don't celebrate Christmas, but do celebrate a gift-buying holiday, Hannukah is pretty important. My gut tells me that Jewish folks are a big portion of this segment, and running the experiment 3 weeks after Hannukah would pretty much kill any chance of conversion for them. Still, the takeaway from your experiment is solid: if you're running an email campaign 4 days before a major holiday, go ahead and focus on those who celebrate it.
deleted 16y ago
[deleted]

jordanlyall 16y ago

What the writer is referring to is the "Click Through Rate". A "conversion" is when the user takes an action post click (submits info, downloads, purchases, etc.). He also incorrectly calculate the CTR. The CTR on an email campaign is clicks divided by opens. He divided clicks over sends, which since fewer people opened the "Happy Holidays" email, fewer would have naturally clicked. "Merry Christmas" still performed better, but not at the rates he's suggesting.

warrenwilkinson 16y ago

'Happy Christmas' remains awkward.

waqf 16y ago
"Happy Christmas" is accepted in the UK, just not in the US.

Misha_B 16y ago

From 30 Rock: HAPPY HOLIDAYS is what terrorists say. Merry Christmas, Avery and Jack.