6 ms·
‘“I’m re-running the analysis this weekend using Uber’s more optimistic assumptions and should have new results and a public response acknowledging the discrepa
by stochastic_monk 9y ago
‘“I’m re-running the analysis this weekend using Uber’s more optimistic assumptions and should have new results and a public response acknowledging the discrepancy by Monday,” he wrote.‘
Re-running analysis with more optimistic assumptions is not how science works.
- dekhn 9y agoNo, that's precisely how it works. All of science is based on making assumptions.
- xiphias 9y agoIf an assumption can give 2x difference, the whole study is flowed. He should extend the study to check the assumption itself.
- nopriorarrests 9y agoOK, let's imagine he we will re-run the study with optimistic assumptions and the result would be that Uber drivers are compensated lavishly (and if we take more moderate assumptions, they are making adequate money). And with another set of assumptions result was that they are serverly underpaid. So, exactly what we have learned here, other then different assumptions yield different results?
- Eridrus 9y agoWe learn that the outcome we care about is highly sensitive to the initial assumptions and we should get data on those assumptions.
- 1337biz 9y agoWe learn that there are no absolutes when using quantitative methods to measure human actions.
- twic 9y agoNothing. But learning that different assumptions yield different results is useful in its own right. It tells you that this study is not capable of answering the question it set out to answer, and you will need to do a better study. Perhaps you have come across interval arithmetic: https://en.wikipedia.org/wiki/Interval_arithmetic https://en.wikipedia.org/wiki/Interval_arithmetic
- deleted 9y ago[deleted]
- dekhn 9y agonot much, which is a big part of the problem.
- pdpi 9y agoIf the re-run concludes that even more optimistic assumptions still lead to underpaid drivers, though, that’s useful information. The key aspect here is the study’s author is willing to change his study based on criticism, to try and determine whether it still produces the same conclusions. That is precisely what science is about!
- stochastic_monk 9y agoBut should those assumptions really be based on angry tweets from someone with a colossal vested interest?
- makomk 9y agoUber's complaint was that he assumed that when the questionnaire asked about the hours per week people worked across all on-demand services, their answer only included the hours they worked driving for companies like Uber, but that for the next question about money earned across all on-demand services their answer was their total income and should be multiplied by the percentage of their total income that came from driving given in a later question to get their Uber income. Those assumptions are basically incompatible with each other. See https://medium.com/uber-under-the-hood/an-analysis-of-ceeprs-paper-on-the-economics-of-ride-hailing-1c8bfbf1081d https://medium.com/uber-under-the-hood/an-analysis-of-ceeprs...
- stochastic_monk 9y agoIt's quite clear that the survey was flawed. However, I'd rather consider all responses tainted and require re-surveying rather than trying to account for it post hoc.
- dekhn 9y agoIf somebody with a colossal vested interest makes an angry tweet that also happens to point out the study author made basic errors (that's what this is about: the author of the study likely made invalidating errors; no amount of changing assumptions is going to fix that).
- luispedrocoelho 9y agoThe reply in the uber blog is less "angry tweet" than "good peer review": it points to a very specific methodological weakness which has bearing on the final conclusions.
- subroutine 9y agoI can't tell if this is sarcasm. In case not, the scientific process is about gathering data to test the validity of hypotheses. Your phrasing makes it sound like science is mainly about spewing conjecture.
- dekhn 9y agoAssumptions are an essential part of science. Any modelling study (that's what this was) starts with a collection of assumptions (even true for ab initio simulations). Those assumptions are literally coded into the analysis. They should also be clearly stated in the analysis writeup.
- subroutine 9y agoCalling this 'a study' is a stretch. This seems more like pure analysis to me. No non-trivial assumptions are necessary to compute the average wage of Uber employees. In this case is was simply a calculation error.
- mannykannot 9y agoA hypothesis is a conjecture - we hope not a fanciful one, but a conjecture nevertheless.
- amelius 9y agoI find that statement a bit disturbing too, but do we need "science" to determine people's average pay now?
- stochastic_monk 9y agoI agree that it’s an awkward label, but “good research practices” felt worse. It was median, not average, as well, which makes sense as potentially being lower than an average wage.
- walshemj 9y agoThe median is better for things like pay as it removes the extreme outliers - I doubt you'd like it if your negotiation for a pay increase was denied because your lavishly remunerated c level execs skewed the average.
- stochastic_monk 9y agoExactly. Which is why the responses which focus on how the "average" pay is higher than the "median" pay the report mentioned is misleading at best.
- r3bl 9y ago> Re-running analysis with more optimistic assumptions is not how science works. String theory is nothing more than an assumption.
- stochastic_monk 9y agoAnd is untestable.
- richardknop 9y agoString theory is still just a speculation by some theoretical physicists, albeit some of them high profile. It's not part of a model accepted my mainstream physics so bad comparison.
- ballenf 9y agoThe assumption is with regard to how respondents interpreted a vaguely worded question. Not an assumption of what 2+2 equals. This study has a huge human component since no single party has all the data needed to calculate the answer. Uber has a lot of data, but even they don't know total hours spent before or after shifts in prep for work nor expenses with regard to vehicle maintenance.
- mseebach 9y ago> Uber has a lot of data, but even they don't know total hours spent before or after shifts in prep for work nor expenses with regard to vehicle maintenance. I would think Uber has exactly the data they would need to calculate this meaningfully, or at least the points you list aren't the decisive shortcomings. Most jobs require that you get yourself to work and prepared (to some degree, at least, but one that seems to apply fine to Uber drivers) on your own time. Also, in the blog post they state that they agree with the methodology around calculating cost of vehicle use, which regardless is not an area of great contention. There are stable and largely uncontroversial numbers published by tax authorities, and there are rental agencies that provide vehicles at fixed rates.
- walshemj 9y agoIts economics its not called the dismal science for nothing