4 ms·

Was there something wrong with the way the exit survey was asked/written? Because that is remarkable. Was maybe part of the issue that people leaving when the

Was there something wrong with the way the exit survey was asked/written? Because that is remarkable. Was maybe part of the issue that people leaving when the exit survey was being run (at "night") not representative of the visitors as a whole?

asdff 6y ago
Past elections have told us exit polling is a load of horseshit. These are useless metrics.
- D-Coder 6y ago
  If you are referring to 2016, the national results were pretty accurate. Clinton did win the popular vote. The Electoral College gives slightly more power to small states than they would get under a popular vote, and occasionally that extra power makes a difference.
blakesterz 6y ago
Great questions, I wondered how they got it so wrong, I still wonder how they got it so wrong. It never occurred to me it may have been time of day they asked the questions.
- heavenlyblue 6y ago
  Or people who answer surveys have more money
- jerf 6y ago
  There's a big difference between spending hypothetical dollars and real dollars, no matter how much you ask people to please try really really hard to pretend they're spending real dollars. It's a recurring element in the recurring "I Tried To Start A SaaS Business And It Crashed And Burned Taking My Will To Live With It" posts on HN... "I asked some people if they'd be willing to buy it and they said yes, were very encouraging, until I asked them to actually buy it."
- caseysoftware 6y ago
  Three things strike me here: audience, proximity, and predicting behavior. - The audience coming out of the waterpark at the end of the day are different than the people entering the overall park at the beginning of the day. If nothing else, the larger group will include people who have no interest in going to the waterpark and therefore find no value in it at all. The smaller audience is more highly qualified. - The proximity comes down to when you received/perceived value. Polling people immediately after they had a fun filled day is terrible timing. Think of the last time you went to a movie you LOVED. On the way out, if someone had asked "want to see it again" you would have said "yes!" but a week later, much less so. Now a year later would you buy the same movie on streaming? - And finally, people are terrible about predicting their own behavior, especially if it requires work or not-immediately visible value. That's why most New Year's resolutions are dead by February. After hitting this lots of times, I stay away from "would" questions: https://caseysoftware.com/blog/the-problem-with-would https://caseysoftware.com/blog/the-problem-with-would But without going through that pain and suffering it first hand, it's hard to predict when/if it will occur. There are lots of other packaging & pricing strategies to address this intersection of economics & psychology.
- rconti 6y ago
  "Would you pay extra for the water park" at the end of a fun day is a proxy for "did you appreciate/receive value from this thing you just finished enjoying?" Whereas, on a random Tuesday, announcing: "Want to come to the water park? PAY US MORE MONEY." Is a proxy for "we're taking something away from your admission ticket and trying to backdoor a new charge".
clairity 6y ago
this is a well-known issue in market research, and survey design in particular. stated preferences are nearly useless in understanding customer segments and behavior. respondents have hundreds of different reasons for not answering with their true preferences (in this case, to not to pay more/separately). instead, you want to reveal preferences using techniques like conjoint analysis (not always feasible), which better simulates the actual choices respondents might make in real situations.
- travisjungroth 6y ago
  It’s pretty obvious when you look at things from an awkward level of first principles. “Based on this forced social interaction of questioning strangers as they left the park, we have forecasted how different people entering the park will react to a commercial interaction.” Seems shocking that it would ever work at all.
mjevans 6y ago
It could have also been expectations. The people reading a poorly written survey might have thought that the park wanted to SPLIT the fee structure. * Same $$ Both Parks * Fraction, One Park There are so many ways it could have been taken incorrectly, even the context of the other questions matters too.

asdff 6y ago

Past elections have told us exit polling is a load of horseshit. These are useless metrics.

D-Coder 6y ago
If you are referring to 2016, the national results were pretty accurate. Clinton did win the popular vote. The Electoral College gives slightly more power to small states than they would get under a popular vote, and occasionally that extra power makes a difference.

blakesterz 6y ago

Great questions, I wondered how they got it so wrong, I still wonder how they got it so wrong. It never occurred to me it may have been time of day they asked the questions.

heavenlyblue 6y ago
Or people who answer surveys have more money
jerf 6y ago
There's a big difference between spending hypothetical dollars and real dollars, no matter how much you ask people to please try really really hard to pretend they're spending real dollars. It's a recurring element in the recurring "I Tried To Start A SaaS Business And It Crashed And Burned Taking My Will To Live With It" posts on HN... "I asked some people if they'd be willing to buy it and they said yes, were very encouraging, until I asked them to actually buy it."
caseysoftware 6y ago
Three things strike me here: audience, proximity, and predicting behavior. - The audience coming out of the waterpark at the end of the day are different than the people entering the overall park at the beginning of the day. If nothing else, the larger group will include people who have no interest in going to the waterpark and therefore find no value in it at all. The smaller audience is more highly qualified. - The proximity comes down to when you received/perceived value. Polling people immediately after they had a fun filled day is terrible timing. Think of the last time you went to a movie you LOVED. On the way out, if someone had asked "want to see it again" you would have said "yes!" but a week later, much less so. Now a year later would you buy the same movie on streaming? - And finally, people are terrible about predicting their own behavior, especially if it requires work or not-immediately visible value. That's why most New Year's resolutions are dead by February. After hitting this lots of times, I stay away from "would" questions: https://caseysoftware.com/blog/the-problem-with-would https://caseysoftware.com/blog/the-problem-with-would But without going through that pain and suffering it first hand, it's hard to predict when/if it will occur. There are lots of other packaging & pricing strategies to address this intersection of economics & psychology.
rconti 6y ago
"Would you pay extra for the water park" at the end of a fun day is a proxy for "did you appreciate/receive value from this thing you just finished enjoying?" Whereas, on a random Tuesday, announcing: "Want to come to the water park? PAY US MORE MONEY." Is a proxy for "we're taking something away from your admission ticket and trying to backdoor a new charge".

clairity 6y ago

this is a well-known issue in market research, and survey design in particular. stated preferences are nearly useless in understanding customer segments and behavior. respondents have hundreds of different reasons for not answering with their true preferences (in this case, to not to pay more/separately). instead, you want to reveal preferences using techniques like conjoint analysis (not always feasible), which better simulates the actual choices respondents might make in real situations.

travisjungroth 6y ago
It’s pretty obvious when you look at things from an awkward level of first principles. “Based on this forced social interaction of questioning strangers as they left the park, we have forecasted how different people entering the park will react to a commercial interaction.” Seems shocking that it would ever work at all.

mjevans 6y ago

It could have also been expectations. The people reading a poorly written survey might have thought that the park wanted to SPLIT the fee structure. * Same $$ Both Parks * Fraction, One Park There are so many ways it could have been taken incorrectly, even the context of the other questions matters too.