4 ms·

Having implemented several projects in the world of analyzing data to make predictions, I think a bigger issue exists: the philosophical (existential?) differen

Having implemented several projects in the world of analyzing data to make predictions, I think a bigger issue exists: the philosophical (existential?) difference between frequentists and Bayesians. Essentially these two approaches have 'irreconcilable differences'. This is where the 'not enough evidence' objection (objection 3 in the article) usually rears its ugly head. In my observation, the reason silicon valley gets a deserved reputation for using data to make decisions well is that they are (mostly) Bayesian. Startups have to make decisions with very small data sets. If they were to wait until there was a large sample size of users, they would never get over the chicken-egg problem of scale. So they all out of necessity go talk to their small set of users, find out what they like, then make prior assumptions about what a slightly larger group of users might like.

ISL 9y ago
Good scientists are conversational in both languages of probability, even if they are fluent and most-comfortable in one. Both approaches have strengths and weaknesses; the application of the appropriate tool at an appropriate time tends to have the greatest success.
- derefr 9y ago
  The parent isn't talking about paradigm/approach; they're talking about how a certain class of people ("statistics-users in or from academia", let's call them) tend to believe in things like the "statistical power" of an experiment, which—while only sensible under frequentism—isn't really a frequentist idea, but rather just a peculiar tradition of academic rigour from before it was easier to multiply large numbers in meta-analyses. Or, to put that another way: by "frequentism", the parent poster is referring to those people who believe that one thousand experiments on five people each add up to nothing, because none of them individually had enough power to draw a significant result. And by "Bayesianism", the parent poster is referring to those people who just do the five-person experiments and use whatever data they spit out, however noisy it is, because fractions of a bit of information are still more information than they had before.

ISL 9y ago

Good scientists are conversational in both languages of probability, even if they are fluent and most-comfortable in one. Both approaches have strengths and weaknesses; the application of the appropriate tool at an appropriate time tends to have the greatest success.

derefr 9y ago
The parent isn't talking about paradigm/approach; they're talking about how a certain class of people ("statistics-users in or from academia", let's call them) tend to believe in things like the "statistical power" of an experiment, which—while only sensible under frequentism—isn't really a frequentist idea, but rather just a peculiar tradition of academic rigour from before it was easier to multiply large numbers in meta-analyses. Or, to put that another way: by "frequentism", the parent poster is referring to those people who believe that one thousand experiments on five people each add up to nothing, because none of them individually had enough power to draw a significant result. And by "Bayesianism", the parent poster is referring to those people who just do the five-person experiments and use whatever data they spit out, however noisy it is, because fractions of a bit of information are still more information than they had before.