4 ms·
There are multiple levels to data. If 57% of respondents to a self reported survey say they are going to do X, it doesn't mean that 57% of people are actually g
by jjk166 2y ago
There are multiple levels to data. If 57% of respondents to a self reported survey say they are going to do X, it doesn't mean that 57% of people are actually going to do X. But if you do the same survey again and now only 43% of respondents say they will do X, then that is clear evidence something has clearly changed, even if you don't know exactly what effect that something will have on X. That is very useful.
The problem is when people only look at self reporting. For example if in the previous scenario 57% responded X but only 54% actually did X, then someone might naively assume that after 43% respond X that 40% will actually do X. Or just as naively they could say 54% will still do X. Or they could apply any of an infinite number of other models. There exists a model that will spit out any given answer for any given input, so without that followup work to actually verify and understand the underlying mechanism, models are worthless.