7 ms·
What makes one appear smarter and more sociable?
- simonster 14y agoThese graphs could use some error bars.
- slashcom 14y agoIndeed. While the results are very interesting, and some of them are clearly very strong differences, I'd like to see some t-tests.
- celer 14y agoWhen you mention the strong differences, are you accounting for the y axis starting at varying locations? Once I realized this, the significance of the results went down significantly.
- mindstab 14y agoThat's usually when you have some idea of the error in what you're measuring. They are just reporting on a social poll they did, isn't that usually a bit different. I mean of course this isn't a rigorous scientific study, but that doesn't mean it's useless either.
- thisisnotmyname 14y agoThey're presenting averages, which should always be accompanied by the standard deviations, otherwise they're close to meaningless. I actually think box plots would be better here.
- capnrefsmmat 14y agoIndeed. It's hard to tell if any of the differences are significant, particularly when you compare with so many traits -- the chances of a false positive are very high. I have a statistics final exam coming up; if the complete data set were available, I'd love to play with it for practice.
- sparkie 14y agoI'm confused as to what Perceived Smartness `div` Extroversion is meant to represent. Or is it implying that extroverts are perceived as smart?
- dpark 14y agoThey phrased that poorly, but the graphs make it pretty clear. Every picture was rated for "looks intelligent" and "looks extroverted". The two are orthogonal in presentation to the voter and as presented in the graphs. The blue bars are for the "smartness" ratings and the red bars are for the "extroversion" ratings.
- drewwwwww 14y agothe y axes vary a great deal. there's no information on the distribution of ratings for each class. really difficult to tell if there's anything meaningful or even interesting here at all.
- Mz 14y agoI read just enough to decide it isn't really worth reading. I love the articles OK Cupid does with hard statistical data backing up their inferences about similar social stuff. This does not strike me as of that ilk. I am disappointed. I was recently thinking about how people are judged based on looks (and blogged about it) so was hoping for/looking forward to something meatier.
- tsumnia 14y agoAwesome analysis, though I agree they results look like error bars. I know you mention a random sample of 1000 images, but what were your overall metrics? Did you have a good data set across the board (ie as many Hispanic females as Caucasian males)? What kind of advertising did you do as well? Reason I ask is I've been working on trying build a face-morpher based on different criteria (make you look 80, fat, African) and these are some of the questions I've got bouncing in my head about how to collect the data.
- pflats 14y agoWhen you don't start your y-axis at 0, you skew the interpretation of your data. At best, this is a significant mistake, and at worst, this is intentionally misleading. Take this graph: http://i.imgur.com/bBzCK.png http://i.imgur.com/bBzCK.png It looks like women are rated as more than twice as smart as men. Huge difference. Except until you run the numbers. Women are rated about 4.3% "smarter" than men. Not twice as smart, like the graph implies. Not 20% smarter. Not even 5%. Please, pay attention to your graphs. They're great tools, but they can mislead as much as they can help elucidate.
- aero142 14y agoAnd worse than that, the y-axis range shifts. For example, the range on the Fat vs. Normal chart is much smaller than the surrounding charts, giving the illusion that the spread is much larger than it is. (site is down or I'd link).
- bvi 14y agoTrue, and I was having a tough time deciding which option to take (whether to start the y-axis at 0, or 5). As far as averages are concerned, the differences are in the tenths and hundredths - so starting the y-axis at 0 would have it more difficult to see any differences across the graphs.
- mhartl 14y agostarting the y-axis at 0 would have it more difficult to see any differences across the graphs. That's the point: the differences are small.
- scott_s 14y agoI would go further: the differences probably are not significant. (Small differences may still be significant. In this case, I doubt it.)
- DevX101 14y ago| so starting the y-axis at 0 would have it more difficult to see any differences across the graphs Then maybe there isn't a real difference. See statistical significance.
- kami8845 14y agoCoral Cache to the rescue http://judg.me.nyud.net/blog/judgment-day/ http://judg.me.nyud.net/blog/judgment-day/
- larrys 14y ago"thousands of photos have been uploaded and judged by users since." Who are the users that are judging? What is the breakdown of those users (age,sex,location,education etc.)? What can possibly be inferred from this without knowing that info?
- bvi 14y agoIt's entirely anonymous - I know nothing about the users who are uploading the photos (apart from the email addresses they use when uploading a photo), and nothing about users who judge the photos. The entire premise of the site is for the user to be judged by strangers. Why would age/sex/location/education of the person doing the judging matter?
- olalonde 14y ago> Why would age/sex/location/education of the person doing the judging matter? It can bias the results. How did your users find your site?
- DanBC 14y agoResearch is tricky. Unless you're very careful with adjusting for factors (like age, location, etc) and careful with statistics you end up with garbage. Either do something that's just purely fun, or do something properly and call it research. But don't do something wrong and call it research, because there are so many people ignorant about how science works and ignorant about numbers.
- oskarth 14y agoBecause it's most likely skewed. Without knowing anything about where your votes come from, I'm pretty sure the WEIRD demographic is over-represented (white, educated, industrialized, rich, democratic). Not trying to be overly critical - I like the concept and execution, but these things are really important in statistics.
- saraid216 14y agoThe judges have IP addresses that you can map to a geographic region, such as a state or country. That would be useful data, and you'd be able to actually say with authority whether or not at least the location of the person doing the judging matters.
- nates 14y agoYour graphs appear to be very misleading. There is little to be learned from the data. Learn some Data analysis and learn how to not provide bias via graphs.
- Danieru 14y agoI think they drove too far into the details considering their sample size. It is an interesting study so I hope they update the post once they have been in business longer.
- MarlonPro 14y agoBe asian and bald = smart! It ain't that simple!
- antihero 14y agoAnd female?
- hoop 14y agoOh good, just what the Internet needed: two-axis hot-or-not
- ori_b 14y agoI have no idea what the standard deviation on this is. Lots of the numbers look close enough to be noise. Others in this thread have pointed out other missing information that makes this a fairly poor survey.
- andrewfelix 14y agoAgreed. What's the margin of error on these? Sample group of 1000 is also far too small to make any kind of conclusion considering all the variables.
- boredguy8 14y agoInter-rater reliability is super important for tests like this. http://en.wikipedia.org/wiki/Inter-rater_reliability http://en.wikipedia.org/wiki/Inter-rater_reliability The gist is: you can't simply mark one person as "asian" and assume that categorization is correct. In that respect, the data would reveal more about the person sorting the photos than it would reveal about the perceptions of those that are rating the photos. Second, there is a huge problem with causality here. So for instance, the author writes: "Be Asian if you want to appear smart; Latino if you want to appear extroverted." The problem is that there is a methodological flaw. On the first photo I saw on judge.me, I was presented with this image: http://images.judg.me/82e7fcbd988dbdcac0d00bd53fb93e96.jpg http://images.judg.me/82e7fcbd988dbdcac0d00bd53fb93e96.jpg This would appear to me to be a latino or hispanic male at a party. I'm highly inclined to rate them highly on the extrovert scale: they're at a party. But that doesn't indicate stereotypically latino or hispanic features indicate extroversion. It could be that people with stereotypically latino or hispanic features were more likely to upload photos in which the image portrayed a more stereotypically extroverted activity. Third, it appears that users can upload a photo to the site and see their feedback from votes. It seems highly possible that users self-select a photo that will best affirm the image of themselves they wish to cultivate. In that respect, there's both a huge confirmation bias and huge self-selection bias. If I want to think of myself as an academic, I'll upload a picture of me at my desk studying and watch the "intellectual" ratings pour in. Then I can feel assured that other people perceive me the way I want to be perceived. Additionally, if one wants to conform to social expectations (and things like Asch's line test http://en.wikipedia.org/wiki/Asch_conformity_experiments http://en.wikipedia.org/wiki/Asch_conformity_experiments indicate conformity is common), this data might really be nothing more than showing the degree to which people post photos affirming their conformity to their social expectations (i.e. 'smart' ethnicities posting 'smart-looking' photos) and be saying nothing at all about how people actually perceive ethic cues. There are huge methodological concerns for this 'study'. Instead, the revelation of this data might actually be the insight that "pictures of yourself at social events makes you look more social." Taking much of anything at all away from this data set would be rather unwise.
- hej 14y agoWhile inter-rater reliability is a concern for identifying ethnicities, other traits should be more reliable (gender, hair color). Still, stuff like that needs to be pre-tested. You basically let the coders code a limited set of photos and check the correlation between their codes. The higher the correlation, the better, >.7 is the convention in social science (but still pretty bad, higher would be better). You should also check intra-coder reliability, i.e. give the same person the same set of photos with two weeks or so between. You can then again calculate the correlation. This tells you wether your categories are too fuzzy (e.g. what exactly is medium length hair?). All in all this has serious methodical flaws, from a social science perspective it’s not salvageable, and I haven’t even talked about the complete lack of statistical tests (which, to be honest, would just be like polishing a turd).
- mc32 14y agoI had to stop reading after the following entered their glossary: sistas, ladies. Sorry but that is a turn-off to me.
- geraldfong 14y agoThis is cool data, but it would be best if you could release numbers about the distribution more than just the average, ie standard deviation, quartile, medians. It is hard to determine significance from these graphs, especially as pflats commented that the y-axis are skewed.
- MichailP 14y agoProf. Dan Ariely mentioned Hot or Not website in one of his books. He used the website to get his attractiveness score and other interesting data. The book is a great read, and analysis of how people percieve you by the looks. As for the website, I think that judg.me looks very promising as a source of social data, which is otherwise very difficult to obtain.
- Kiro 14y agoThe comments here are depressing. Why can't you just enjoy it for what it is? No-one believes this is pioneering research so you don't need to analyze it as such.
- bvi 14y agoThat's okay. :) I'm learning quite a bit from the comments, and it's always good to know what others think.
- Mz 14y agoUpvoted. I hope you do turn this into a source for solid analysis of this sort. Best of luck.
- mynameishere 14y agoI just wish I could downmod such stark idiocy.
- andrewfelix 14y agoI come here to read informed analysis and thoughts. If you prefer misguided positive commentary instead of critical analysis I suggest you don't read HN comments.
- kvh 14y agoWhat a tragic waste of data and time. Not one mention of confidence intervals (are _any_ of these differences statistically significant??), selection bias (who was more likely to submit photos, and why did they choose a specific photo??), or sampling errors (who rated the attributes, and how consistent were they?). The OK Cupid blog posts are a great source for similar (but statistically sound) studies.
- deleted 14y ago[deleted]
- carguy1983 14y agoSo in other words (for men) to get the most responses from dating sites, be happy and white and wear sunglasses in an outdoor setting with a 5 o'clock shadow and show your fit body? In other words be a rugged, outdoorsy, all-american white guy. Pretty sure this is only confirming what was already common knowledge.
- blt 14y agohere is a 2D scatter plot of the smart/social ratings: https://docs.google.com/spreadsheet/oimg?key=0AmoarnvJ2W0ndFh5aTF2Ti1Va2VKUnNDNTE0Vm1WX1E&oid=2&zx=t13973iv1gp2%22 https://docs.google.com/spreadsheet/oimg?key=0AmoarnvJ2W0ndF... . It is still very misleading without axes going to zero, but at least you can see the purported differences clearly.
- jcc80 14y agoFor someone in his early 30s whose hair is starting to thin out the results are interesting, though expected. I won't lose many social points but will pick up a good amount of perceived smartness when the baldness battle is finally lost. And to throw the "I'm a fun-loving extrovert" vibe out there for special occasions, I just throw on some shades.
- pessimizer 14y agoThe problem with this blog is so obvious that I'm suprised that I haven't seen it in the comments yet (I probably missed it), but you can't use a random selection of photographs for this if you want to expect gross ratings to mean something. You would have to normalise each trait that you were comparing against every other trait. Otherwise, when you were trying to isolate how smart people judge black people to be, and black people were wearing caps a quarter more ofter than the average person, you would think you were getting interesting data for blacks when really you were getting interesting data for caps. If you didn't plan to use gross ratings like this blog did (I think), then I'm pretty sure that you could do a post-normalization by analyzing the frequencies in the sample and determining how much you'd expect each of the traits to affect the rating for every other trait, then trying to determine the if the deviations from that were statistically significant in a universe that contained only those traits. Honestly - just take the original data and assign every trait a 5 rating, then pick a random trait and pull that value up or down, then check and see what the gross ratings now say about the other traits. I apoligize if the methodology was more complicated than it looks, and I hope there's a link to the spreadsheet of the original distribution somewhere in the blog that I missed, so someone could make sense of this data.
- craigmoore 14y agoLong live Hot or Not! I don't want to like this sort of site. I'm ashamed that I read the whole post (and found it interesting).
- lunchbox 14y agoThe authors conflate "extroversion" and "social skills". For example, based on his pic I'd rate this guy high on extroversion but low on social skills: http://madconfessionsofaman.files.wordpress.com/2011/05/douchebags2.jpg http://madconfessionsofaman.files.wordpress.com/2011/05/douc... Similarly, being introverted doesn't mean you have low social skills.
- paulhauggis 14y ago"Similarly, being introverted doesn't mean you have low social skills." I don't know how you can say that. The definition here: http://dictionary.reference.com/browse/introvert?s=t http://dictionary.reference.com/browse/introvert?s=t implies low social skills. Even when people discuss it here, they talk about wanting to be left alone, not going to parties because they aren't interested in socializing, and feeling "weak" after socializing for a short amount of time. With all of this, I don't know how your social skills could ever be considered high.
- lunchbox 14y agoEnjoying socializing (extraversion) != being good at it (social skills)
- mmcdan 14y agoAs an introvert, I can be a social butterfly at parties but need to leave after an hour because it felt like work. As an extrovert, I can be a wallower but want to stay out all night because the ambiance is energizing.
- noahc 14y agoIntrovert has a definition which is colloquial, which is what you're pointing to. It also has a definition used in psychology as well, which is something completely different, than the term used colloquially.
- B-Con 14y agoNo, that definition you linked to does not imply low social skills. Social skills are the ability to make conversation, to make people feel comfortable, to engage others, etc. Extroversion is the desire to do those things. Yes, they are clearly correlated, but by absolutely no means interchangeable. There are plenty of introverts who have perfectly decent social skills, and plenty of extroverts who have terrible social skills, because the desire for something is not the same as having a skill for it. For example, an adolescent extrovert may feel strong motivation to be outgoing, loud, talkative, and active, but that doesn't mean he will have good social skills and be polite. The most common psychological definition of extrovert and introvert that we currently use was largely shaped by Carl Jung, and his definition involves far more than just sociability. However, sociability is the easiest application. What it really comes down to (as the link you provided points out) is a focus on the "external world" vs the "internal world", so I use the word "are" in those two definitions above very loosely.
- dmvaldman 14y agoError bars! Law of large numbers says that your error should scale like 1/sqrt(N) where N is the sample size. In this case N = 1000, so 1/sqrt(N) ~ 3% This measures 1 STD (68% of values lie in an interval of 3% of reported value). To be on the safe side you should take 2 or 3 STDs for the error bars. This already nullifies most of the results!
- neckpunch 14y agoBooze. Especially whiskey. At least if the appearance is self-assessed.
- Jebus 14y agoThanks for the tl;dr's, +1
- antsam 14y agoI guess all future photos of me will have to be full body shots, with five o'clock shadow, outside, smiling, and died gray hair.
- AngryParsley 14y agoAccording to this, the smartest most sociable people should be smiling bald indian women with glasses and 5 o'clock shadows.
- roarktoohey 14y agoWhat would be interesting to see is the likely decline in intelligence as the user base increases.
- mahrain 14y agoSo, please mark what data is statistically significant at a 95% level?
- scotty79 14y agoWhy people assume that parameters are independent? If most black woman that sent their photos are fat and people don't rate fat woman high then the black women will be rated low not because of the race but because being black woman and being fat woman is correlated in the sample data. Owner of such sites have large sample of some data and they assume that large equals representative and they go on slicing their data by different parameters not controlling for anything and making statements that are only technically true with respect to their data but strongly misleading in many ways.
- lninyo 14y agoThis should be renamed to "stereotype.me" ... don't believe.me? check.out the scores.me for the blacks.me
- inDesperateZone 14y agoOkay, that's it, I'm cutting of my hair. Everyone seems to hate long hair on man. But I wonder how many long haired man were in the sample. They are quite rare and a few ponytail grad students might lower the score.
- willpearse 14y agoI'm sorry, but there's no 'unsexy data crunching' here - just a series of ratios compared against one another. There is a whole body of statistical literature about how to do anything of this kind, and they haven't done any of it. I'd quite happily believe that none of these differences have any kind of significance in the statistical sense (i.e., it's due to background variation). But then again, I wasn't given any information to know whether they've even looked. So I can't say...
- B-Con 14y agoThese people have a wrong definition of extroversion. The actual site rates extroversion vs introversion, but the analysis here mistakenly uses the term social scale, implying that extroversion and sociability are interchangeable. They are correlated, but by absolutely no means are they interchangeable. This analysis should have stuck with the original vocabulary more consistently.
- superslug 14y agoNothing says sociable like an iced grill ..