6 ms·

I think asking "are you aware" is a different question. Most non-tech people have barely done any research on this and probably have never even looked into it.

by jonfw 8y ago

I think asking "are you aware" is a different question. Most non-tech people have barely done any research on this and probably have never even looked into it. You're sharing information and asking questions about it at the same time. Sort of a cross-contamination IMO.

fixermark 8y ago
Such cross-contamination, when done maliciously, is called "push-polling" and is a known tactic for shaping political discourse by taking advantage of being possibly the first to mention a topic in a person's field-of-view to paint that topic in the worst possible light. I don't think that's what's being done here, but it's interesting to note that what you're seeing is definitely an issue regarding polls in general.
- mandevil 8y ago
  Quite honestly, most things that receivers call "push polls" are not push polls. They are honest attempts to figure out which lines of attack will resonate better with the public at large. So you try several different attacks on a relatively small sample, see what makes people respond most strongly, and then scale up the attacks with paid media, speeches etc. Yes, most of these will have to inform people of the bad stuff you are going to be attacking over, but that's because if the general public was widely aware of the attack, you wouldn't need to mention it- or at least not to pay pollsters to figure out if it is a good attack. The hallmarks of a push poll are that it makes no attempt to gauge your response level, and that it tries to call far more people than a statistical survey would need to do. The most famous case, in recent US history, would have to be "John McCain's illegitimate Black baby" (https://www.vanityfair.com/news/2004/11/mccain200411 https://www.vanityfair.com/news/2004/11/mccain200411)
  - dahart 8y ago
    Interesting use of the word "honest". If there's any agenda being pushed during a survey at all, then it's not an honest survey, statistically speaking. An honest attempt to figure out a line of attack isn't an honest poll. So, any "survey" that is trying to sway or push opinions at all is a push poll in my book, even if they're also collecting information & using the information collected. You're trying to draw the line at one extreme, all push and no poll, but I believe the term "push poll" has long been used by many people for surveys that do both. Since you can't influence the thing you're sampling and get an accurate sample at the same time, don't you think the line actually needs to be at 100% poll and 0% push, rather than requiring 0% polling before you can call it a push poll?
    - kortilla 8y ago
      It’s an “honest” survey because the authors are using it to extract the data they are looking for (the most effective trigger phrasing to use in speeches, etc). It’s working as designed to provide useful information to the authors. A push poll doesn’t provide information to authors. They don’t care about the results.
      - dahart 8y ago
        > A push poll doesn’t provide information to authors. They don’t care about the results. That's not necessarily true, there are polls in the middle, where the authors attempt to both push agenda, and also collect information on whether the push is working. I'm saying that any amount of pushing constitutes a push poll; the defining characteristic is not whether the authors care about the results, but whether they're attempting to influence the results. Examples of what I'm talking about are even listed in the Wikipedia article on push polling: https://en.wikipedia.org/wiki/Push_poll https://en.wikipedia.org/wiki/Push_poll If a poll is really asking whether a certain line of negative attack is going to work better than another, and not attempting to influence the answer, not attempting to sway the respondent, then yes that would be an "honest" survey in some sense. However, there aren't actually many polls related to negative campaigns that do this, and it's also very difficult to do without influencing the respondent. Negative attack campaigns are rarely if ever interesting in doing careful neutral statistics. You'd have to be scientific about it and include controls as well as positive and neutral statements in addition to the negative ones. So, I still don't buy the original claim that most polls on negative lines of attack aren't really push polls. They are push polls, but they might indeed be softer pushes than other more extreme push polls.
        mandevil 8y ago
        The difference, as so often, is scale. If you poll 500 people in a congressional district and ask them what their reaction to Candidate X's drinking problem is, that's an information gathering poll, not one intended to sway a significant number of voters. It might not be "scientific", but it is intended to gather information about broader voter sentiment to guide your campaign strategy lines, because even changing 500 voters minds isn't going to have much effect on the race. If you did asked the same questions, and even made the same decisions based on the data, but did it to 100,000 voters then it does start to look more like a hybrid or closer to the push-poll side of things.
- icebraining 8y ago
  Well exemplified by Yes, Prime Minister: https://www.youtube.com/watch?v=5xC2bNpXdAo https://www.youtube.com/watch?v=5xC2bNpXdAo
scott_s 8y ago
I had a similar reaction as redwards510, but you bring up a good point. You could do an "are you aware" question first, then follow-up with "are you okay with" right after. Another alternative is "are you surprised to learn," but that's still a different question, as very knowledgeable people would not be surprised to learn it, but may not be okay with it. Surveys are hard.
- jonfw 8y ago
  The point of the survey is to test whether or not people are aware of what google is doing with their data, not whether or not they're okay with it. Transparency is at the center of the argument of the morality of surveillance. There is nothing inherently immoral about collecting data from a consenting party, but it's impossible to gain consent from consumers who don't even know that they're under surveillance. If you are attempting to test peoples existing knowledge of a situation, you don't want to offer new information before you gauge their existing knowledge.
  - tremon 8y ago
    The point of the survey is to test whether or not people are aware of what google is doing with their data But this is a difficult question as well. Speaking for myself, I cannot say that I am aware of what Google is doing with my data. I fully expect them to use my data for any and all purposes they can think of, and I'm not okay with any of that -- but I'm still not aware of what they're actually doing.
  - scott_s 8y ago
    > The point of the survey is to test whether or not people are aware of what google is doing with their data, not whether or not they're okay with it. I'm not sure that's true. From the article itself: "DCN surveyed a nationally representative sample to find out what people expect from Google — and, as with a similar study we conducted last year about Facebook, the results were unsettling." Since the notion of "expect" is baked into the purpose, I think some manner of "okay with it" is fundamental to the survey. (Because when people don't expect something, that's usually not okay.)
- 8y ago
Fishkins 8y ago
I agree with this, but I also that "Do you expect" is a bad way to start the question. "Expect" can either mean what you think will happen, or what you think should happen. Given that, I wouldn't be surprised if different respondents were answering different questions. I'd say "Do you think Google is currently..."

by jonfw 8y ago

fixermark 8y ago

Such cross-contamination, when done maliciously, is called "push-polling" and is a known tactic for shaping political discourse by taking advantage of being possibly the first to mention a topic in a person's field-of-view to paint that topic in the worst possible light. I don't think that's what's being done here, but it's interesting to note that what you're seeing is definitely an issue regarding polls in general.

mandevil 8y ago
Quite honestly, most things that receivers call "push polls" are not push polls. They are honest attempts to figure out which lines of attack will resonate better with the public at large. So you try several different attacks on a relatively small sample, see what makes people respond most strongly, and then scale up the attacks with paid media, speeches etc. Yes, most of these will have to inform people of the bad stuff you are going to be attacking over, but that's because if the general public was widely aware of the attack, you wouldn't need to mention it- or at least not to pay pollsters to figure out if it is a good attack. The hallmarks of a push poll are that it makes no attempt to gauge your response level, and that it tries to call far more people than a statistical survey would need to do. The most famous case, in recent US history, would have to be "John McCain's illegitimate Black baby" (https://www.vanityfair.com/news/2004/11/mccain200411 https://www.vanityfair.com/news/2004/11/mccain200411)
- dahart 8y ago
  Interesting use of the word "honest". If there's any agenda being pushed during a survey at all, then it's not an honest survey, statistically speaking. An honest attempt to figure out a line of attack isn't an honest poll. So, any "survey" that is trying to sway or push opinions at all is a push poll in my book, even if they're also collecting information & using the information collected. You're trying to draw the line at one extreme, all push and no poll, but I believe the term "push poll" has long been used by many people for surveys that do both. Since you can't influence the thing you're sampling and get an accurate sample at the same time, don't you think the line actually needs to be at 100% poll and 0% push, rather than requiring 0% polling before you can call it a push poll?
  - kortilla 8y ago
    It’s an “honest” survey because the authors are using it to extract the data they are looking for (the most effective trigger phrasing to use in speeches, etc). It’s working as designed to provide useful information to the authors. A push poll doesn’t provide information to authors. They don’t care about the results.
    - dahart 8y ago
      > A push poll doesn’t provide information to authors. They don’t care about the results. That's not necessarily true, there are polls in the middle, where the authors attempt to both push agenda, and also collect information on whether the push is working. I'm saying that any amount of pushing constitutes a push poll; the defining characteristic is not whether the authors care about the results, but whether they're attempting to influence the results. Examples of what I'm talking about are even listed in the Wikipedia article on push polling: https://en.wikipedia.org/wiki/Push_poll https://en.wikipedia.org/wiki/Push_poll If a poll is really asking whether a certain line of negative attack is going to work better than another, and not attempting to influence the answer, not attempting to sway the respondent, then yes that would be an "honest" survey in some sense. However, there aren't actually many polls related to negative campaigns that do this, and it's also very difficult to do without influencing the respondent. Negative attack campaigns are rarely if ever interesting in doing careful neutral statistics. You'd have to be scientific about it and include controls as well as positive and neutral statements in addition to the negative ones. So, I still don't buy the original claim that most polls on negative lines of attack aren't really push polls. They are push polls, but they might indeed be softer pushes than other more extreme push polls.
      - mandevil 8y ago
        The difference, as so often, is scale. If you poll 500 people in a congressional district and ask them what their reaction to Candidate X's drinking problem is, that's an information gathering poll, not one intended to sway a significant number of voters. It might not be "scientific", but it is intended to gather information about broader voter sentiment to guide your campaign strategy lines, because even changing 500 voters minds isn't going to have much effect on the race. If you did asked the same questions, and even made the same decisions based on the data, but did it to 100,000 voters then it does start to look more like a hybrid or closer to the push-poll side of things.
icebraining 8y ago
Well exemplified by Yes, Prime Minister: https://www.youtube.com/watch?v=5xC2bNpXdAo https://www.youtube.com/watch?v=5xC2bNpXdAo

scott_s 8y ago

I had a similar reaction as redwards510, but you bring up a good point. You could do an "are you aware" question first, then follow-up with "are you okay with" right after. Another alternative is "are you surprised to learn," but that's still a different question, as very knowledgeable people would not be surprised to learn it, but may not be okay with it. Surveys are hard.

jonfw 8y ago
The point of the survey is to test whether or not people are aware of what google is doing with their data, not whether or not they're okay with it. Transparency is at the center of the argument of the morality of surveillance. There is nothing inherently immoral about collecting data from a consenting party, but it's impossible to gain consent from consumers who don't even know that they're under surveillance. If you are attempting to test peoples existing knowledge of a situation, you don't want to offer new information before you gauge their existing knowledge.
- tremon 8y ago
  The point of the survey is to test whether or not people are aware of what google is doing with their data But this is a difficult question as well. Speaking for myself, I cannot say that I am aware of what Google is doing with my data. I fully expect them to use my data for any and all purposes they can think of, and I'm not okay with any of that -- but I'm still not aware of what they're actually doing.
- scott_s 8y ago
  > The point of the survey is to test whether or not people are aware of what google is doing with their data, not whether or not they're okay with it. I'm not sure that's true. From the article itself: "DCN surveyed a nationally representative sample to find out what people expect from Google — and, as with a similar study we conducted last year about Facebook, the results were unsettling." Since the notion of "expect" is baked into the purpose, I think some manner of "okay with it" is fundamental to the survey. (Because when people don't expect something, that's usually not okay.)
8y ago

Fishkins 8y ago

I agree with this, but I also that "Do you expect" is a bad way to start the question. "Expect" can either mean what you think will happen, or what you think should happen. Given that, I wouldn't be surprised if different respondents were answering different questions. I'd say "Do you think Google is currently..."