5 ms·

20,000 is a massive sample. Political polls are based on a few thousand respondents even though there are millions of voters. It's basic statistics. The MyTy

20,000 is a massive sample. Political polls are based on a few thousand respondents even though there are millions of voters. It's basic statistics. The MyType sample was weighted to reflect the age, location, gender and personality distribution of the US. (Note that very few samples are weighted by personality. This is an important consideration because certain personality types are more prone to respond to surveys.) A better critique is that the survey was held exclusively on Facebook. But even then, every online survey has to be held somewhere. As the most popular site on the web, Facebook has minimal sample bias. The post also notes how the MyType results are in line with Forrestor and Sybase survey results, so the data appear to be very valid.

slantyyz 16y ago
There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. Having said that, I'm of the opinion that most polls are useless, even more so if they're done by an analyst firm like Forrester. EDIT: On a side note, I wonder how many of the Facebookers lied about owning an iPad.
- bad_user 16y ago
  > There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. When you're doing such an assertion, you also need hard numbers. Quite the contrary, my intuition says that most iPad owners do have an account on Facebook, but again ... I can't prove it. The real question ... is the chosen sample biased or not? ... and you can't answer with a yes/no just because the survey was done on Facebook, simply because the demographics of Facebook is pretty large and diverse (and for this assertion there is data available). > Having said that, I'm of the opinion that most polls are useless, even more so if they're done by an analyst firm like Forrester. That's because when doing statistics it's pretty easy to introduce bias ... but the error margin can be calculated ... by people knowing how to do their job of course. That said, electoral polls are pretty accurate in general with an error margin of +/- 1.5%, so many of them aren't useless.
  - slantyyz 16y ago
    I am assuming people were asked to participate in this survey, and that the survey mentioned that it was for the iPad (I couldn't see any place where it was mentioned that it the topic of the survey was anything more generalized such as "portable computing devices"). Because opinions on the iPad were quite polarized, especially among the techies who felt like the iPad was too inflexible, I tend to think the opportunity to participate in such a survey created a self-selection bias for the so-called "independent geeks" and for the Apple fanboys who want to ensure that the outcome of any satisfaction survey yields a positive opinion score for the device. Those are the two groups who dominate in the comments sections of any blog post or news article about Apple products, so is it even surprising that the two dominant profiles are "Selfish Elites" and "Independent Geeks"?
    - bad_user 16y ago
      When you put it that way, you may have a point :)
    - mytype 16y ago
      This certainly would be a good point if it were true. In reality people come to MyType to take a personality quiz. We just randomly inserted the iPad question into the personality quiz for 20,000 people. They're not particularly savvy on technology, just the average Facebook user. We realized that as a personality typing application we have a very good platform for surveying any sort of opinion. Our user base is fairly reflective of the general US population, and none of them are coming to answer a survey about x. We plan to look at the personality traits that drive preferences in beer, politics, and much more. Subscribe to the blog if you haven't already :)
      - slantyyz 16y ago
        Fair enough. As I said, I "assumed" the details of how you invited participation in the survey, and you proved my assumption to be incorrect. I did look at your pie chart for the question, and it still doesn't strike me as a properly constructed interval type of question (are "think it's a silly product" and "don't care that much" mutually exclusive? I could agree with both statements). "It's a silly product" is also not the kind of phrasing you would use to get objective results. The survey probably would have benefited from additional questions. In any case, I'll stick to my original statement and say "take it with a grain of salt", as I would with the results of any poll.
- dlsspy 16y ago
  > There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. Facebook has somewhere around 120 million active users in the US alone. The US has around 307 million active residents. 1/3 of the population of the US is on facebook. A good number of the remaining 2/3 probably don't have their own means to buy things (e.g. that includes nearly 60 million people who are too young to be eligible for a facebook account).
  - slantyyz 16y ago
    You forgot about too old or tech-illiterate. I can't be the only person to have said "the iPad is the perfect device for my -insert older relative here-". And I would bet there's a significant portion of that segment isn't on Facebook. Don't forget that there's a percentage of Facebook users that are businesses, fake users, "employer friendly" duplicate accounts, and people who lie about their location. That's not counting people who lie about their demographic information, either. I don't doubt there's a large number of Facebook users, but you can't take everything at face value.
    - flatulent1 16y ago
      Years ago I read about a study that went on and on with interesting things about the people of Beverly Hills California. What the people doing the study hadn't figured out was that virtually everyone in it had made up their info and wasn't in Beverly Hills at all. In picking a ZIP code other than their own to use the first thing that came to mind was 90210, the name of a tv show.
xenophanes 16y ago
A random sample of 20,000 is massive. A non-random sample of even a million can easily be worthless. Political polls are very careful about getting a sample representative of the country which is why they can have some value. They also have very detailed models, which a lot of thought have gone into, about what the general population looks like and how to get accurate results. The weighting from this facebook poll is not based a bunch of well-developed knowledge but just guesses at what to do.
- slantyyz 16y ago
  The problem even with political polls is unless an expected result is obviously hard to fudge, you still need a lot of healthy skepticism, because the money behind a poll tends to subtly skew the results. That's why GOP poll results tend to conflict with DEM results.
- mytype 16y ago
  Our sampling rigor cannot be compared to political polls, it's true. However we do weight the sample not based on guesses but on official data about the gender, age, and personality distribution in the US. We did not use geographic weights because our respondents were more or less properly distributed. Any poll can be attacked. Even political polls are subject to bias in who picks up the phone, who responds to the surveyor, etc. This bias is partially driven by personality type, which we are able to nearly eliminate through weighting by personality traits. How many polls can do that? Most polling is much less random than political polls, and yet the results are still treated as worthwhile. Some firms put up surveys on sites (site audience bias), or have carefully selected paid volunteers (bias to incorporate people who are willing to do that), etc. The fact that our results are corroborated by Forrestor and Sybase, to the extent that we can be compared, is very satisfying to me personally. Sure, we may be a few percentage points off on some things, but are the overall results invalid? No. I challenge anyone to point to any result in the study and give a reasonable explanation why that result is likely way off. The only things that are suspect to me are the major outliers, subsets like native americans for which we did not have much data. I'll give you that. Anything else? Let me know.
  - 16y ago
btilly 16y ago
However within the 20,000 they focused on slices of critics (11%) and owners (3%). Within those groups they focused on geeks (33% of the 11%) and selfish elites (18% of the 3%). Sure, they started off with a good sample size, but from something like 108 people they are painting a vivid image of all iPad owners. Even though their sample suggests that over 80% of iPad owners don't fit that image! (The geek image was not as poorly fitting, but it still didn't fit well.)
- mytype 16y ago
  We focused on the 11% and 3% slices, yes. But from there we looked for the most over-represented groups. The fact that the independent geeks and selfish elites are 3 and 6 times more prominent in those slices is statistically significant, and it's a statement about the strongest tendencies of owners and critics. But yes most owners are not selfish elites, and most critics are not independent geeks. It's difficult to say any group is "mostly" any particular psychographic profile. We're highlighting correlations and tendencies, not stereotyping. Admittedly, the title doesn't convey this, but if it tried to no one here would be reading the post, it would sound like something out of an academic journal.

slantyyz 16y ago

There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. Having said that, I'm of the opinion that most polls are useless, even more so if they're done by an analyst firm like Forrester. EDIT: On a side note, I wonder how many of the Facebookers lied about owning an iPad.

bad_user 16y ago
> There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. When you're doing such an assertion, you also need hard numbers. Quite the contrary, my intuition says that most iPad owners do have an account on Facebook, but again ... I can't prove it. The real question ... is the chosen sample biased or not? ... and you can't answer with a yes/no just because the survey was done on Facebook, simply because the demographics of Facebook is pretty large and diverse (and for this assertion there is data available). > Having said that, I'm of the opinion that most polls are useless, even more so if they're done by an analyst firm like Forrester. That's because when doing statistics it's pretty easy to introduce bias ... but the error margin can be calculated ... by people knowing how to do their job of course. That said, electoral polls are pretty accurate in general with an error margin of +/- 1.5%, so many of them aren't useless.
- slantyyz 16y ago
  I am assuming people were asked to participate in this survey, and that the survey mentioned that it was for the iPad (I couldn't see any place where it was mentioned that it the topic of the survey was anything more generalized such as "portable computing devices"). Because opinions on the iPad were quite polarized, especially among the techies who felt like the iPad was too inflexible, I tend to think the opportunity to participate in such a survey created a self-selection bias for the so-called "independent geeks" and for the Apple fanboys who want to ensure that the outcome of any satisfaction survey yields a positive opinion score for the device. Those are the two groups who dominate in the comments sections of any blog post or news article about Apple products, so is it even surprising that the two dominant profiles are "Selfish Elites" and "Independent Geeks"?
  - bad_user 16y ago
    When you put it that way, you may have a point :)
  - mytype 16y ago
    This certainly would be a good point if it were true. In reality people come to MyType to take a personality quiz. We just randomly inserted the iPad question into the personality quiz for 20,000 people. They're not particularly savvy on technology, just the average Facebook user. We realized that as a personality typing application we have a very good platform for surveying any sort of opinion. Our user base is fairly reflective of the general US population, and none of them are coming to answer a survey about x. We plan to look at the personality traits that drive preferences in beer, politics, and much more. Subscribe to the blog if you haven't already :)
    - slantyyz 16y ago
      Fair enough. As I said, I "assumed" the details of how you invited participation in the survey, and you proved my assumption to be incorrect. I did look at your pie chart for the question, and it still doesn't strike me as a properly constructed interval type of question (are "think it's a silly product" and "don't care that much" mutually exclusive? I could agree with both statements). "It's a silly product" is also not the kind of phrasing you would use to get objective results. The survey probably would have benefited from additional questions. In any case, I'll stick to my original statement and say "take it with a grain of salt", as I would with the results of any poll.
dlsspy 16y ago
> There are a lot of iPad owners not on Facebook, who are arguably a major market segment for Apple. Facebook has somewhere around 120 million active users in the US alone. The US has around 307 million active residents. 1/3 of the population of the US is on facebook. A good number of the remaining 2/3 probably don't have their own means to buy things (e.g. that includes nearly 60 million people who are too young to be eligible for a facebook account).
- slantyyz 16y ago
  You forgot about too old or tech-illiterate. I can't be the only person to have said "the iPad is the perfect device for my -insert older relative here-". And I would bet there's a significant portion of that segment isn't on Facebook. Don't forget that there's a percentage of Facebook users that are businesses, fake users, "employer friendly" duplicate accounts, and people who lie about their location. That's not counting people who lie about their demographic information, either. I don't doubt there's a large number of Facebook users, but you can't take everything at face value.
  - flatulent1 16y ago
    Years ago I read about a study that went on and on with interesting things about the people of Beverly Hills California. What the people doing the study hadn't figured out was that virtually everyone in it had made up their info and wasn't in Beverly Hills at all. In picking a ZIP code other than their own to use the first thing that came to mind was 90210, the name of a tv show.

xenophanes 16y ago

A random sample of 20,000 is massive. A non-random sample of even a million can easily be worthless. Political polls are very careful about getting a sample representative of the country which is why they can have some value. They also have very detailed models, which a lot of thought have gone into, about what the general population looks like and how to get accurate results. The weighting from this facebook poll is not based a bunch of well-developed knowledge but just guesses at what to do.

slantyyz 16y ago
The problem even with political polls is unless an expected result is obviously hard to fudge, you still need a lot of healthy skepticism, because the money behind a poll tends to subtly skew the results. That's why GOP poll results tend to conflict with DEM results.
mytype 16y ago
Our sampling rigor cannot be compared to political polls, it's true. However we do weight the sample not based on guesses but on official data about the gender, age, and personality distribution in the US. We did not use geographic weights because our respondents were more or less properly distributed. Any poll can be attacked. Even political polls are subject to bias in who picks up the phone, who responds to the surveyor, etc. This bias is partially driven by personality type, which we are able to nearly eliminate through weighting by personality traits. How many polls can do that? Most polling is much less random than political polls, and yet the results are still treated as worthwhile. Some firms put up surveys on sites (site audience bias), or have carefully selected paid volunteers (bias to incorporate people who are willing to do that), etc. The fact that our results are corroborated by Forrestor and Sybase, to the extent that we can be compared, is very satisfying to me personally. Sure, we may be a few percentage points off on some things, but are the overall results invalid? No. I challenge anyone to point to any result in the study and give a reasonable explanation why that result is likely way off. The only things that are suspect to me are the major outliers, subsets like native americans for which we did not have much data. I'll give you that. Anything else? Let me know.
- 16y ago

btilly 16y ago

However within the 20,000 they focused on slices of critics (11%) and owners (3%). Within those groups they focused on geeks (33% of the 11%) and selfish elites (18% of the 3%). Sure, they started off with a good sample size, but from something like 108 people they are painting a vivid image of all iPad owners. Even though their sample suggests that over 80% of iPad owners don't fit that image! (The geek image was not as poorly fitting, but it still didn't fit well.)

mytype 16y ago
We focused on the 11% and 3% slices, yes. But from there we looked for the most over-represented groups. The fact that the independent geeks and selfish elites are 3 and 6 times more prominent in those slices is statistically significant, and it's a statement about the strongest tendencies of owners and critics. But yes most owners are not selfish elites, and most critics are not independent geeks. It's difficult to say any group is "mostly" any particular psychographic profile. We're highlighting correlations and tendencies, not stereotyping. Admittedly, the title doesn't convey this, but if it tried to no one here would be reading the post, it would sound like something out of an academic journal.