11 ms·

We Experiment On Human Beings

367 points by dochtman 12y ago

rockdiesel 12y ago
Nice to see a new blog post from OkCupid. They haven't updated their blog since April 2011.
- zabcik 12y ago
  I hope it sticks around! They're so fascinating.
- cschmidt 12y ago
  Interesting that the author has a plug at the bottom for his new book, which he says he's been working on for the past 3 years. I guess his book writing got in the way of blogging.
  - jkaunisv1 12y ago
    It seemed to me the company being acquired by Match.com got in the way of the blogging.
  - chimeracoder 12y ago
    > I guess his book writing got in the way of blogging. There were a number of factors (which I've explained elsewhere in my comment history). That's part of it, but another factor was that, in 2010, there were 2.5 people working full-time[0] on doing research for OkTrends. The blog posts took a lot of work. "The Real Stuff White People Like"[1] took almost two months of my time, plus some from Max and Christian as well. (Much like the product design process, since we didn't start each post off with a clear end result in mind, not all the work was visible in the final product). I left to go back to school. Max ended up taking on more responsibility for other data/stats work, which slowed the pace a bit, and he left at the beginning of 2012 to do his own stuff. People asked me for the last three years whether the reason OkTrends hadn't posted since 2011 was because of the Match.com acquisition and whether Match shut them down and I had to tell everyone "No, trust me, they're still around! It's just a coincidence!". Thankfully I no longer have to. :) [0] 2.5 full-time means: Two of us full-time, as well as Christian, though he split his work time between OkTrends (the blog) and other stuff. [1] http://blog.okcupid.com/index.php/the-real-stuff-white-people-like/ http://blog.okcupid.com/index.php/the-real-stuff-white-peopl...
    - harimau 12y ago
      I had only heard about the studies but never took time to read the blog posts. I think I've spent the better part of the past two hours on there. Great stuff and always interesting to see people doing experiments like this (and making them open and 'fun'). A blog post every few months with as much in-depth research is something to look forward to.
- ShaneOG 12y ago
  I immediately dropped everything to read this blog post. I have been waiting for 3 years! I am fascinated with the data and science behind human behaviour, and their blog posts were (now are) always really interesting.
  - heyalexej 12y ago
    This!
- adamzerner 12y ago
  Yeah, I just started reading all the older blog posts and they are really interesting!
  - Ntrails 12y ago
    I'm still mad they removed the blog post on the success rates of paid dating services a few days after being acquired by Match.com (I think it was)
    - jabagawee 12y ago
      In case you wanted to read it again, here's a Wayback Machine link [1]. [1]: http://web.archive.org/web/20101006104124/http://blog.okcupid.com/index.php/why-you-should-never-pay-for-online-dating/ http://web.archive.org/web/20101006104124/http://blog.okcupi...
    - discardorama 12y ago
      It is still true, and even before they wrote the blog post, that used to be my answer to friends' question, "which is better?". I would say, think about it: Match makes money the longer you stay in the system! Then it would dawn on them that they were the product.
- discardorama 12y ago
  > They haven't updated their blog since April 2011. I think that's around the time they were bought out by Match.com .
foolrush 12y ago
I suspect that the cavalier sentiment OKCupid has approached the discourse with will not go well for them. Will be interesting to observe the sociological takedowns of this post.
- tomjen3 12y ago
  Unlikely. Facebook went overboard with it after hiding it from their users. OKC has always done this with their data, although I am more than a little disturbed by them showing deliberately bad data to their users. Also who the hell though it was a good idea to have a dating site without pictures (even for 24 hours?)
  - ghaff 12y ago
    And, probably more to the point, Facebook is Facebook and OKCupid is OKCupid. And even if, as OKCupid says in their post "But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work." the fact that Facebook did it ignited the Internet's collective anti-Facebook sentiment. (Of course, it's also the case that Facebook's experiments were more oriented toward their own advertising than improving the site experience for users.)
  - Mandelbug 12y ago
    Unfortunately, while showing what we would consider "intuitively bad data" may seem like a bad idea, only actually empirically testing it can we measure and quantify the exact impact that bad data (either shown purposefully or accidentally, or unknowingly) will have on users. It may have in fact turned out that what we intuitively think as bad data results in better matches or better experiences. I think experimenting is worthwhile, so long as it is done in the open as they have been doing.
    - nobodysfool 12y ago
      I remember reading about this concerning Netflix. They used to have a competition on who can come up with a better recommendation algorithm. They eventually decided not to implement the best one because what they were getting was good enough, and since most people were streaming instead of getting a dvd, the more recommendations the better. They could just try and if they don't want to see the whole movie, rate it and instantly pick another.
andrey-p 12y ago
I really enjoyed reading this. On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error. For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore. [1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them. Edit: tweaked phrasing as per comments.
- Chinjut 12y ago
  I believe you are supposed to get notified when rated 4 or 5 regardless of whether you rated the other person at all.
  - wldcordeiro 12y ago
    Didn't they recently remove the 4-5 star system in favor of just a like button? I've been using the mobile version lately but it just has a star now for like and no longer shows a 1-5 rating.
- delecti 12y ago
  You can always get notified that somebody scored you 4-5, and you get notified of specifically who it was if you score them 4-5 back.
  - andrey-p 12y ago
    Fair point - that's what I meant. Let me rephrase that.
tomjen3 12y ago
>But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does. Hm, that is unexpected. Wonder how long it will last.
- sliverstorm 12y ago
  Nah, we like people who like us. So if a trusted source tells us someone else either likes or will like us, we tend to like them.
  - surement 12y ago
    From Modern Seinfeld [0]: George swipes right for every woman on Tinder. E:"What if you're not attracted to her?" G:"If she's attracted to me, I might be!" [0] https://twitter.com/SeinfeldToday https://twitter.com/SeinfeldToday
interstitial 12y ago
OKC experimented with the matching process -- the express purpose of their site. Facebook experimented with mood manipulation -- something they have no permission to do -- regardless of the fine print. Facebook is a very bad landlord, but people don't want to move.
- yarrel 12y ago
  They experimented through misrepresentation. It's the latter that's the problem, not the former. If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
  - Dylan16807 12y ago
    'match percentage' is a fuzzy subjective measure. To a large extent it's asking of OkCupid thinks two people having a conversation is a good idea, which is basically impossible to be a lie. And they're doing it to fight against users being hideously misrepresentative, which is hilarious.
    - chillingeffect 12y ago
      The percentage is insanely fuzzy. I often endure rage fits from one of my buddies who shows me example after example of cases where, for example, he answered "often" to a question a lady answered "usually," resulting in a mismatch. The only good thing about it are key individual questions that let you judge someone's intelligence and determine if they're racist. The scalar number is a crock.
      - GFK_of_xmaspast 12y ago
        If your pal is having "rage fits" about an online dating site, he shouldn't be on there in the first place.
- azakai 12y ago
  Facebook didn't experiment with "mood manipulation" any more than OKC. Facebook experimented with changing the site layout and which content and how much to show. Exactly the same stuff as all sites test. Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation. But OKC's changes also affected people's moods. More than Facebook's, I would guess. All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.
rabbyte 12y ago
I don't think the sticky point is whether or not we should be experimenting with people to find what works and what doesn't. It's about respecting users as humans and not simply data points. How does OkCupid know that conversations "went deeper" or how often emails and numbers were exchanged? Were the participants aware of that level of scrutiny or is that tucked away in the ToS? Most people I know aren't even comfortable talking about their OkCupid, they use it as a means to an end and place trust in the service to be discrete. Do these experiments fall in line with user expectations? I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.
- herge 12y ago
  For the emails and numbers, they can probably search for those in the messages quite effectively without many false-positives. I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
  - rabbyte 12y ago
    Sure, my comment wasn't to say they're doing anything nefarious. I'm actually not taking a position here either way. I'm just pointing out that people aren't upset that services use data to improve their services which seems to be the tone and premise of OkCupid's post.
adamzerner 12y ago
Is anyone surprised that only 1/5 people who are 90% matches have a "conversation"? I don't use dating sites, but I'd think that people who do want to converse, and who better to converse with than 90% matches? But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me. Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.
- cm2012 12y ago
  It's quite possible to have 100 90% matches - can't contact them all!
- Someone1234 12y ago
  If you want to understand this better I'd recommend going back through that Blog's archives, they talk extensively about this (TL;DR: Women are the minority on the site, and only respond to a small subset of messages, plus matches are personalty not looks and as this article indicates that is only 10% of people's opinions).
- javert 12y ago
  Women don't respond to most messages, even 90%+. 90%+ match is not that special.
  - danudey 12y ago
    90% matches are common because of the huge number of people on the site. And they seem common because those are the kinds of people you're likely to be shown. Either that or they're usually rare and I'm just so generic I get along with everyone.
    - mkr-hn 12y ago
      It might be different for the gay side of the site. It's rare that I get a percentage above 60.
blt 12y ago
Heh heh. I recently got a message from okcupid saying "Because of a diagnostic test, your match percentage with <user> was misstated as 31%. It is actually 91%. We wanted to let you know!" I had a feeling something like this might be involved. (I messaged that person anyway, because they had a funny username, so I guess I supported their point.)
zach 12y ago
I thought Experiment 2 was a great example of the pain of trying to fit your users' feelings into an "ideal" model. When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing. But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept. Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively. So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly. So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it. PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.
- boomzilla 12y ago
  This phenomenon is actually quite common. In retail, it's called the customer funnel: http://en.wikipedia.org/wiki/Purchase_funnel http://en.wikipedia.org/wiki/Purchase_funnel. The idea is you want to capture as much awareness with something really simple, usually it's a brand name, e.g. Apple. When customers express more interest, by coming in Apple store/website, they are presented with more design/technical details. Finally, only when they show purchase intent, they are shown price/configuration/delivery options.
deleted 12y ago
[deleted]
ssl_love 12y ago
Hey OkCupid – How about some SSL Love? "For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network." https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-some-ssl-love https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-... The Heartbreaking Truth About Online Dating Privacy: https://www.eff.org/press/releases/heartbreaking-truth-about-online-dating-privacy https://www.eff.org/press/releases/heartbreaking-truth-about...
- 3pt14159 12y ago
  Is this really still the case?
  - handsomeransoms 12y ago
    Oh, yeah :/ The login page is over SSL, but everything after that is not (!!) This protects your password but not your OkCupid session cookies. It's an improvement over no SSL at all (which was the case for a long time) but still leaves a lot to be desired.
nostrademons 12y ago
When I was still on OKCupid, I once wrote a Chrome extension to hide the pictures. I found that I had, in general, a much better experience with the site - I'd actually read people's profiles, I sent better messages, and I got more responses. I eventually gave up on it when a site redesign changed some of the #ids I was depending on and I didn't feel like revamping the code. But interestingly, my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site. Setting up our first date, she was like "We don't need to exchange numbers, you know what I look like", and I was like "Actually, I'm not sure I do, here's mine." (Okay, I didn't actually tell her her pictures were terrible until we'd been dating for six months or so, but that was the general sentiment.) I think it's one more example of when people's emotions and desires lead them to suboptimal outcomes. Most of the cues we associate with beauty and sexual desire evolved back in the savanna when health and fertility were very real risks; being able to pick up on which potential mates would be able to carry healthy offspring to term and nurture them until adulthood was very important then. Nowadays, the far greater risk is that you'll hate each other and fight all the time, but this has only been a concern since people started living long enough and in close enough proximity to care.
- jessriedel 12y ago
  > my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site. This is not what people mean when they say someone looks terrible in their pictures. The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
  - nostrademons 12y ago
    Previous OKTrends blog posts and their MyBestFace feature have shown that putting up poor-quality photos has the same effect as being ugly. People just assume you're ugly.
  - 12y ago
ianstallings 12y ago
For the record not every site does these type of experiments. I've been programming websites since the 90s and I've never experimented on my users like this. The most I've done is swap in and out user interface features to see which one has a better response in a typical A/B test. It's not the same as telling a lie to a user no matter how you split hairs about it. They're overstating their case because they happen to be in the business of matching people up.
wcdolphin 12y ago
But you never did so with the intention of eliciting a negative psychological response. That to me is a distinct difference, both in practicality and morality.
krallja 12y ago
Doesn't load completely in IE11.
iamleppert 12y ago
What a bunch of pointless data and hubris to prove that, yes, people are predominately fueled by the laws of physical attraction.
- xj9 12y ago
  It might be hard to accept, but people are shallow. Everybody has an attractiveness threshold that must be reached before someone will spark their interest. It might be lower or higher, depending on the person, but everybody has one. Obviously, sustained interest and the development of romance is more complicated and involves aspects of personality, shared interests, &c. However, that doesn't change the fact that if they weren't good looking enough for you in the first place, you never would have made the effort to find out about the stuff you actually care about.
  - rabbyte 12y ago
    Being able to relate attractiveness to a linear scale doesn't mean attraction operates on linear terms. In reality, there is no threshold nor is there a higher or lower. I agree with you in the sense that people will disregard personality or shared interests if they don't find the person physically attractive but there's no answer that's more correct than another.
pervycreeper 12y ago
I'm surprised no one has commented on the ethics of "Experiment #3". Feeding false data to users is not something that should be taken lightly.
- spott 12y ago
  I think the premise you are assuming is wrong: You assume that it is sending false data to them to say "this person (which our algorithm has deemed is a bad match), is a good match". The problem is that until the experiment is done, you don't how good a match that person actually is. So you aren't sending them "false data," because you don't know that it is "false data".
  - pervycreeper 12y ago
    If it is being reported as known, but is actually unknown, that still would be considered as 'false information'. For instance, if I claim to be clairvoyant, and predict that the next coin you toss will come up 'heads', that would be a false claim, regardless of the outcome of the coin toss.
lifeisstillgood 12y ago
Basically, people are exactly as shallow as their technology allows them to be. Just a quote that resonates ... The design and architecture of our online environments will affect our happiness as much as anything else.
jessriedel 12y ago
> I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar). I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.) It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.
galfarragem 12y ago
I feel very "bearish" towards dating sites. People convince themselves they are saving time meeting more possible "matches" in less time but they forget that meeting online is not the same as in person, "turn-based" chat is different than "real-time" chat. Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again. edit: Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.
bshoemaker 12y ago
#clickbait
tragomaskhalos 12y ago
TL;DR; people are shallow idiots. Cool article though.

367 points by dochtman 12y ago

rockdiesel 12y ago

Nice to see a new blog post from OkCupid. They haven't updated their blog since April 2011.

zabcik 12y ago
I hope it sticks around! They're so fascinating.
cschmidt 12y ago
Interesting that the author has a plug at the bottom for his new book, which he says he's been working on for the past 3 years. I guess his book writing got in the way of blogging.
- jkaunisv1 12y ago
  It seemed to me the company being acquired by Match.com got in the way of the blogging.
- chimeracoder 12y ago
  > I guess his book writing got in the way of blogging. There were a number of factors (which I've explained elsewhere in my comment history). That's part of it, but another factor was that, in 2010, there were 2.5 people working full-time[0] on doing research for OkTrends. The blog posts took a lot of work. "The Real Stuff White People Like"[1] took almost two months of my time, plus some from Max and Christian as well. (Much like the product design process, since we didn't start each post off with a clear end result in mind, not all the work was visible in the final product). I left to go back to school. Max ended up taking on more responsibility for other data/stats work, which slowed the pace a bit, and he left at the beginning of 2012 to do his own stuff. People asked me for the last three years whether the reason OkTrends hadn't posted since 2011 was because of the Match.com acquisition and whether Match shut them down and I had to tell everyone "No, trust me, they're still around! It's just a coincidence!". Thankfully I no longer have to. :) [0] 2.5 full-time means: Two of us full-time, as well as Christian, though he split his work time between OkTrends (the blog) and other stuff. [1] http://blog.okcupid.com/index.php/the-real-stuff-white-people-like/ http://blog.okcupid.com/index.php/the-real-stuff-white-peopl...
  - harimau 12y ago
    I had only heard about the studies but never took time to read the blog posts. I think I've spent the better part of the past two hours on there. Great stuff and always interesting to see people doing experiments like this (and making them open and 'fun'). A blog post every few months with as much in-depth research is something to look forward to.
ShaneOG 12y ago
I immediately dropped everything to read this blog post. I have been waiting for 3 years! I am fascinated with the data and science behind human behaviour, and their blog posts were (now are) always really interesting.
- heyalexej 12y ago
  This!
adamzerner 12y ago
Yeah, I just started reading all the older blog posts and they are really interesting!
- Ntrails 12y ago
  I'm still mad they removed the blog post on the success rates of paid dating services a few days after being acquired by Match.com (I think it was)
  - jabagawee 12y ago
    In case you wanted to read it again, here's a Wayback Machine link [1]. [1]: http://web.archive.org/web/20101006104124/http://blog.okcupid.com/index.php/why-you-should-never-pay-for-online-dating/ http://web.archive.org/web/20101006104124/http://blog.okcupi...
  - discardorama 12y ago
    It is still true, and even before they wrote the blog post, that used to be my answer to friends' question, "which is better?". I would say, think about it: Match makes money the longer you stay in the system! Then it would dawn on them that they were the product.
discardorama 12y ago
> They haven't updated their blog since April 2011. I think that's around the time they were bought out by Match.com .

foolrush 12y ago

I suspect that the cavalier sentiment OKCupid has approached the discourse with will not go well for them. Will be interesting to observe the sociological takedowns of this post.

tomjen3 12y ago
Unlikely. Facebook went overboard with it after hiding it from their users. OKC has always done this with their data, although I am more than a little disturbed by them showing deliberately bad data to their users. Also who the hell though it was a good idea to have a dating site without pictures (even for 24 hours?)
- ghaff 12y ago
  And, probably more to the point, Facebook is Facebook and OKCupid is OKCupid. And even if, as OKCupid says in their post "But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work." the fact that Facebook did it ignited the Internet's collective anti-Facebook sentiment. (Of course, it's also the case that Facebook's experiments were more oriented toward their own advertising than improving the site experience for users.)
- Mandelbug 12y ago
  Unfortunately, while showing what we would consider "intuitively bad data" may seem like a bad idea, only actually empirically testing it can we measure and quantify the exact impact that bad data (either shown purposefully or accidentally, or unknowingly) will have on users. It may have in fact turned out that what we intuitively think as bad data results in better matches or better experiences. I think experimenting is worthwhile, so long as it is done in the open as they have been doing.
  - nobodysfool 12y ago
    I remember reading about this concerning Netflix. They used to have a competition on who can come up with a better recommendation algorithm. They eventually decided not to implement the best one because what they were getting was good enough, and since most people were streaming instead of getting a dvd, the more recommendations the better. They could just try and if they don't want to see the whole movie, rate it and instantly pick another.

andrey-p 12y ago

I really enjoyed reading this. On the flip side (as an OKC user), I've also had moderate amounts of fun trying to figure out their algorithms by trial and error. For a while, I discovered I could figure out who scored me highly on their Quickmatch [1] feature by visiting it repeatedly - the ones who gave me a high score always came up first. This doesn't seem to work anymore. [1]: If you haven't used OKC, the idea is: you score potential matches 1-5. You can tell if someone scored you highly (but not who) and if both of you give each other 4 or 5 you get a message "introducing" you both. If you're an A-list (paid) member, you get to see who scored you highly without scoring them. Edit: tweaked phrasing as per comments.

Chinjut 12y ago
I believe you are supposed to get notified when rated 4 or 5 regardless of whether you rated the other person at all.
- wldcordeiro 12y ago
  Didn't they recently remove the 4-5 star system in favor of just a like button? I've been using the mobile version lately but it just has a star now for like and no longer shows a 1-5 rating.
delecti 12y ago
You can always get notified that somebody scored you 4-5, and you get notified of specifically who it was if you score them 4-5 back.
- andrey-p 12y ago
  Fair point - that's what I meant. Let me rephrase that.

tomjen3 12y ago

>But we took the analysis one step deeper. We asked: does the displayed match percentage cause more than just that first message—does the mere suggestion cause people to actually like each other? As far as we can measure, yes, it does. Hm, that is unexpected. Wonder how long it will last.

sliverstorm 12y ago
Nah, we like people who like us. So if a trusted source tells us someone else either likes or will like us, we tend to like them.
- surement 12y ago
  From Modern Seinfeld [0]: George swipes right for every woman on Tinder. E:"What if you're not attracted to her?" G:"If she's attracted to me, I might be!" [0] https://twitter.com/SeinfeldToday https://twitter.com/SeinfeldToday

interstitial 12y ago

OKC experimented with the matching process -- the express purpose of their site. Facebook experimented with mood manipulation -- something they have no permission to do -- regardless of the fine print. Facebook is a very bad landlord, but people don't want to move.

yarrel 12y ago
They experimented through misrepresentation. It's the latter that's the problem, not the former. If your bank experimented with not completing transactions or your email provider experimented with not delivering emails, the problem wouldn't be that they experimented.
- Dylan16807 12y ago
  'match percentage' is a fuzzy subjective measure. To a large extent it's asking of OkCupid thinks two people having a conversation is a good idea, which is basically impossible to be a lie. And they're doing it to fight against users being hideously misrepresentative, which is hilarious.
  - chillingeffect 12y ago
    The percentage is insanely fuzzy. I often endure rage fits from one of my buddies who shows me example after example of cases where, for example, he answered "often" to a question a lady answered "usually," resulting in a mismatch. The only good thing about it are key individual questions that let you judge someone's intelligence and determine if they're racist. The scalar number is a crock.
    - GFK_of_xmaspast 12y ago
      If your pal is having "rage fits" about an online dating site, he shouldn't be on there in the first place.
azakai 12y ago
Facebook didn't experiment with "mood manipulation" any more than OKC. Facebook experimented with changing the site layout and which content and how much to show. Exactly the same stuff as all sites test. Facebook did theorize that the changes it was making would affect people's moods in a certain way. That might make it sound like intentional mood manipulation. But OKC's changes also affected people's moods. More than Facebook's, I would guess. All these sites are constantly experimenting on humans. That's what changing the site content means. Everything that affects us affects our moods and everything else.

rabbyte 12y ago

I don't think the sticky point is whether or not we should be experimenting with people to find what works and what doesn't. It's about respecting users as humans and not simply data points. How does OkCupid know that conversations "went deeper" or how often emails and numbers were exchanged? Were the participants aware of that level of scrutiny or is that tucked away in the ToS? Most people I know aren't even comfortable talking about their OkCupid, they use it as a means to an end and place trust in the service to be discrete. Do these experiments fall in line with user expectations? I'm not saying you have to find these experiments upsetting just that when people are upset it tends to be about treating users like honorable guests and not about whether or not we should be allowed to tinker with services to find the right solution.

herge 12y ago
For the emails and numbers, they can probably search for those in the messages quite effectively without many false-positives. I'm all for the expectations of privacy, but if you type something into a search box on a website owned by somebody else, well, you get what you pay for.
- rabbyte 12y ago
  Sure, my comment wasn't to say they're doing anything nefarious. I'm actually not taking a position here either way. I'm just pointing out that people aren't upset that services use data to improve their services which seems to be the tone and premise of OkCupid's post.

adamzerner 12y ago

Is anyone surprised that only 1/5 people who are 90% matches have a "conversation"? I don't use dating sites, but I'd think that people who do want to converse, and who better to converse with than 90% matches? But I guess there's a lot of matches, and you can't talk to everyone, which would explain the low numbers. Still a bit surprising to me. Edit: What you guys said makes sense, thanks. My lack of knowledge of dating sites lead to a bad intuition.

cm2012 12y ago
It's quite possible to have 100 90% matches - can't contact them all!
Someone1234 12y ago
If you want to understand this better I'd recommend going back through that Blog's archives, they talk extensively about this (TL;DR: Women are the minority on the site, and only respond to a small subset of messages, plus matches are personalty not looks and as this article indicates that is only 10% of people's opinions).
javert 12y ago
Women don't respond to most messages, even 90%+. 90%+ match is not that special.
- danudey 12y ago
  90% matches are common because of the huge number of people on the site. And they seem common because those are the kinds of people you're likely to be shown. Either that or they're usually rare and I'm just so generic I get along with everyone.
  - mkr-hn 12y ago
    It might be different for the gay side of the site. It's rare that I get a percentage above 60.

blt 12y ago

Heh heh. I recently got a message from okcupid saying "Because of a diagnostic test, your match percentage with <user> was misstated as 31%. It is actually 91%. We wanted to let you know!" I had a feeling something like this might be involved. (I messaged that person anyway, because they had a funny username, so I guess I supported their point.)

zach 12y ago

I thought Experiment 2 was a great example of the pain of trying to fit your users' feelings into an "ideal" model. When we started LALife.com, it was with the idea of making a real estate site with lots of great statistics and quantitative data. We gave grades for how safe things were, how good the schools were, multiple heatmaps, assessor data, census data, nearby amenities -- a buffet of data. And sure, people who also fit in the mold of trivia collectors thought this was amazing. But the more we talked to people, the more we found out that they really didn't use this data, even when they said they appreciated it. We had really overshot the market -- users almost never delved into the statistics, yet took our grades as absolute authority. What people wanted first of all was just insight about whether it was a "nice place" or not, a maddeningly vague concept. Ultimately, we trashed countless tables and statistics and scaled it back to one number. Yes, one 0-99 number that shows you, well, how "nice" a place is to live. We were so afraid of generalizing things like this because everyone is different and has different priorities and so on. But trying to make things custom for everyone is a losing game, and it turned out "is it nice?" is something everyone already knew intuitively. So insanely, we went from having 50 extra data points on a home for sale to having one "superscore". But a funny thing happened, which is it became much more successful in the site's actual mission, to help people understand a home's neighborhood without having to visit in person. And now we're providing an insight that is compact enough that we can put it everywhere and people can digest it instantly. So although we would ultimately like to give people a more data-conscious mindset, the tool and the user need to agree on that commitment. So we're accommodating the user's actual mental model while we work towards expanding it. PS: We have our scoring model working nationwide, but as you have seen from the Zillow thread, it's hard to get nationwide home listings. We're working on getting homes for rent on http://www.padrank.com/ http://www.padrank.com/ so you can sign up there if you want to see how it works throughout the US.

boomzilla 12y ago
This phenomenon is actually quite common. In retail, it's called the customer funnel: http://en.wikipedia.org/wiki/Purchase_funnel http://en.wikipedia.org/wiki/Purchase_funnel. The idea is you want to capture as much awareness with something really simple, usually it's a brand name, e.g. Apple. When customers express more interest, by coming in Apple store/website, they are presented with more design/technical details. Finally, only when they show purchase intent, they are shown price/configuration/delivery options.

deleted 12y ago

[deleted]

ssl_love 12y ago

Hey OkCupid – How about some SSL Love? "For the hundreds of thousands of users searching for that special someone through one of the largest free online dating sites, the love fest may be coming to an end. OkCupid is putting users’ privacy in danger by failing to support secure access to its entire website through HTTPS. Every OkCupid email, chat session, search, clicked link, page viewed, and username is transmitted over the Internet in unencrypted plaintext, where it can be intercepted and read by anyone on the network." https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-some-ssl-love https://www.eff.org/deeplinks/2012/02/hey-okcupid-how-about-... The Heartbreaking Truth About Online Dating Privacy: https://www.eff.org/press/releases/heartbreaking-truth-about-online-dating-privacy https://www.eff.org/press/releases/heartbreaking-truth-about...

3pt14159 12y ago
Is this really still the case?
- handsomeransoms 12y ago
  Oh, yeah :/ The login page is over SSL, but everything after that is not (!!) This protects your password but not your OkCupid session cookies. It's an improvement over no SSL at all (which was the case for a long time) but still leaves a lot to be desired.

nostrademons 12y ago

When I was still on OKCupid, I once wrote a Chrome extension to hide the pictures. I found that I had, in general, a much better experience with the site - I'd actually read people's profiles, I sent better messages, and I got more responses. I eventually gave up on it when a site redesign changed some of the #ids I was depending on and I didn't feel like revamping the code. But interestingly, my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site. Setting up our first date, she was like "We don't need to exchange numbers, you know what I look like", and I was like "Actually, I'm not sure I do, here's mine." (Okay, I didn't actually tell her her pictures were terrible until we'd been dating for six months or so, but that was the general sentiment.) I think it's one more example of when people's emotions and desires lead them to suboptimal outcomes. Most of the cues we associate with beauty and sexual desire evolved back in the savanna when health and fertility were very real risks; being able to pick up on which potential mates would be able to carry healthy offspring to term and nurture them until adulthood was very important then. Nowadays, the far greater risk is that you'll hate each other and fight all the time, but this has only been a concern since people started living long enough and in close enough proximity to care.

jessriedel 12y ago
> my eventual girlfriend had terrible pictures - the grainy, multiple-people-in-the-background sort you're never supposed to put on a dating site. This is not what people mean when they say someone looks terrible in their pictures. The point of the experiment is to withhold information about whether someone is physically attract, not about whether they have good photography skills. The only reason to avoid putting up grainy pictures is because people will assume you're not good looking and are trying to disguise this fact.
- nostrademons 12y ago
  Previous OKTrends blog posts and their MyBestFace feature have shown that putting up poor-quality photos has the same effect as being ugly. People just assume you're ugly.
- 12y ago

ianstallings 12y ago

For the record not every site does these type of experiments. I've been programming websites since the 90s and I've never experimented on my users like this. The most I've done is swap in and out user interface features to see which one has a better response in a typical A/B test. It's not the same as telling a lie to a user no matter how you split hairs about it. They're overstating their case because they happen to be in the business of matching people up.

wcdolphin 12y ago

But you never did so with the intention of eliciting a negative psychological response. That to me is a distinct difference, both in practicality and morality.

krallja 12y ago

Doesn't load completely in IE11.

iamleppert 12y ago

What a bunch of pointless data and hubris to prove that, yes, people are predominately fueled by the laws of physical attraction.

xj9 12y ago
It might be hard to accept, but people are shallow. Everybody has an attractiveness threshold that must be reached before someone will spark their interest. It might be lower or higher, depending on the person, but everybody has one. Obviously, sustained interest and the development of romance is more complicated and involves aspects of personality, shared interests, &c. However, that doesn't change the fact that if they weren't good looking enough for you in the first place, you never would have made the effort to find out about the stuff you actually care about.
- rabbyte 12y ago
  Being able to relate attractiveness to a linear scale doesn't mean attraction operates on linear terms. In reality, there is no threshold nor is there a higher or lower. I agree with you in the sense that people will disregard personality or shared interests if they don't find the person physically attractive but there's no answer that's more correct than another.

pervycreeper 12y ago

I'm surprised no one has commented on the ethics of "Experiment #3". Feeding false data to users is not something that should be taken lightly.

spott 12y ago
I think the premise you are assuming is wrong: You assume that it is sending false data to them to say "this person (which our algorithm has deemed is a bad match), is a good match". The problem is that until the experiment is done, you don't how good a match that person actually is. So you aren't sending them "false data," because you don't know that it is "false data".
- pervycreeper 12y ago
  If it is being reported as known, but is actually unknown, that still would be considered as 'false information'. For instance, if I claim to be clairvoyant, and predict that the next coin you toss will come up 'heads', that would be a false claim, regardless of the outcome of the coin toss.

lifeisstillgood 12y ago

Basically, people are exactly as shallow as their technology allows them to be. Just a quote that resonates ... The design and architecture of our online environments will affect our happiness as much as anything else.

jessriedel 12y ago

> I found a similar thing: once they got to the date, they had a good time more or less regardless of how good-looking their partner was. Here’s the female side of the experience (the male is very similar). I'm skeptical it was similar, at least if you were to measure the thing that actually matters: whether or not the person wants to go on a second date. ("Did you have a good time" is probably equating to "Did the person avoid doing something terrible" in the survey.) It's ludicrous on its face to think that men, if they choose a blind date partner based on conversations and profiles, will want to go on a second date just as often with a beautiful woman as with a plain woman.

galfarragem 12y ago

I feel very "bearish" towards dating sites. People convince themselves they are saving time meeting more possible "matches" in less time but they forget that meeting online is not the same as in person, "turn-based" chat is different than "real-time" chat. Some years ago randomly I met a girl online, that most of men (based on her picture) would wife up. I was living abroad at the time and we were only able to meet in person some months later. After few minutes in the date I could see that we would not go anywhere. There was a lack of "real-time" empathy and I hated some details in her personality that I couldn't realise before meeting her (not her look, surprisingly she was even more beautiful in person than at photos). If I had met her in person instead of online, I would have saved months of my life.. I will not do it ever again. edit: Because of her I met my gf of 5 years, but that was just serendipity working, not because of a dating site.

bshoemaker 12y ago

#clickbait

tragomaskhalos 12y ago

TL;DR; people are shallow idiots. Cool article though.