4 ms·

So just data, but with small sample value, biased (since humans are inherently biased) and non filtered by a scientific method. How is that better than proper r

So just data, but with small sample value, biased (since humans are inherently biased) and non filtered by a scientific method. How is that better than proper research? Of course personal experience is very valuable, precisely because it's a (limited form of) research.

ChadNauseam 3y ago
It's not better than proper research, but when proper research doesn't exist it's all we have
- josephg 3y ago
  Your “gut” instincts can also be a remarkably effective tool in the right situations. You have to feed it enough data and are careful of systematic biases, but your instincts can be brilliant. I had an incredible moment in a job interview: for context, I’ve been programming for about 30 years and 10 of those years have involved a lot of JavaScript. I was given a timed debugging problem to test my skills: “Here is some JS code with failing tests. Go fix the bugs.” Before I even ran the tests I scrolled through the code to get a sense of it. My instincts tweaked on one section. “This code smells. I don’t trust it.” I said. On second glance - “Oh yeah this is totally buggy”. Sure enough, I was right. The interviewer was blown away. There’s no way I could do that consciously. Science is great for a lot of things, but when we don’t have science, we could do a lot worse than trusting our intuition.
PH95VuimJjqBqy 3y ago
some people are better than others, the evidence is there by how some people are more successful than other people. you can attempt to dismiss that as being a small sample value, but that's the reality. The reason people want to be so data driven is because it takes the decision making out of their hands, which takes the responsibility for poor decisions out of their hands. But when it's all said and done, some people are better than others, not all "small sample values" are equal.
- meheleventyone 3y ago
  > some people are better than others, the evidence is there by how some people are more successful than other people Better at what though? Success doesn’t tell you why someone was successful. The point of looking at larger sets of data is to learn something tangible beyond anecdote.
  - PH95VuimJjqBqy 3y ago
    you cannot find causation in data, only correlation with an attempt to identify the causation. it's another version of cargo cult programming. The data suggests if a plane is present, food will come. We want food to come, lets build a plane, with no real understanding of where the food actually comes from. > Better at what though? whatever the fuck it is you're trying to evaluate. Are you evaluating musical aptitude? then the answer to your question is that some people are better at music. are you evaluating the ability to tear down an engine? then the answer to your question is that some people are better at tearing down an engine. Linus Torvalds opinion on kernel development is not equivalent to that 16 year old web developer. Dismissing opinions as "small sample value" _completely_ misses the forest for the trees.
    - meheleventyone 3y ago
      Finding correlation in data is how we find out about the real world though. There are extra steps needed there but let’s not throw out the baby with the bathwater. Otherwise we should just jack in science which AFAIK has been a spectacularly successful project. And it’s actually this study of collected data that will defeat cargo cults when their hypothesis doesn’t stand up to scrutiny. It’s only reliance on presumed experts and anecdotal evidence that shores them up. From the original cargo cults to conspiracy theorists like Alex Jones to software orthodoxy like TDD, SOLID and Scrum. Likewise anecdotal success of an individual is also just correlation but with only a single data point. I’d also agree that expert opinions are great but they’re also not infallible or necessarily generalisable. Nor is it necessarily easy to tell who is an expert and who is a charlatan. Which is why we should use them as the basis for investigating the larger picture. Similarly both expert opinion and singular anecdotes can lead to cargo culting.

ChadNauseam 3y ago

It's not better than proper research, but when proper research doesn't exist it's all we have

josephg 3y ago
Your “gut” instincts can also be a remarkably effective tool in the right situations. You have to feed it enough data and are careful of systematic biases, but your instincts can be brilliant. I had an incredible moment in a job interview: for context, I’ve been programming for about 30 years and 10 of those years have involved a lot of JavaScript. I was given a timed debugging problem to test my skills: “Here is some JS code with failing tests. Go fix the bugs.” Before I even ran the tests I scrolled through the code to get a sense of it. My instincts tweaked on one section. “This code smells. I don’t trust it.” I said. On second glance - “Oh yeah this is totally buggy”. Sure enough, I was right. The interviewer was blown away. There’s no way I could do that consciously. Science is great for a lot of things, but when we don’t have science, we could do a lot worse than trusting our intuition.

PH95VuimJjqBqy 3y ago

some people are better than others, the evidence is there by how some people are more successful than other people. you can attempt to dismiss that as being a small sample value, but that's the reality. The reason people want to be so data driven is because it takes the decision making out of their hands, which takes the responsibility for poor decisions out of their hands. But when it's all said and done, some people are better than others, not all "small sample values" are equal.

meheleventyone 3y ago
> some people are better than others, the evidence is there by how some people are more successful than other people Better at what though? Success doesn’t tell you why someone was successful. The point of looking at larger sets of data is to learn something tangible beyond anecdote.
- PH95VuimJjqBqy 3y ago
  you cannot find causation in data, only correlation with an attempt to identify the causation. it's another version of cargo cult programming. The data suggests if a plane is present, food will come. We want food to come, lets build a plane, with no real understanding of where the food actually comes from. > Better at what though? whatever the fuck it is you're trying to evaluate. Are you evaluating musical aptitude? then the answer to your question is that some people are better at music. are you evaluating the ability to tear down an engine? then the answer to your question is that some people are better at tearing down an engine. Linus Torvalds opinion on kernel development is not equivalent to that 16 year old web developer. Dismissing opinions as "small sample value" _completely_ misses the forest for the trees.
  - meheleventyone 3y ago
    Finding correlation in data is how we find out about the real world though. There are extra steps needed there but let’s not throw out the baby with the bathwater. Otherwise we should just jack in science which AFAIK has been a spectacularly successful project. And it’s actually this study of collected data that will defeat cargo cults when their hypothesis doesn’t stand up to scrutiny. It’s only reliance on presumed experts and anecdotal evidence that shores them up. From the original cargo cults to conspiracy theorists like Alex Jones to software orthodoxy like TDD, SOLID and Scrum. Likewise anecdotal success of an individual is also just correlation but with only a single data point. I’d also agree that expert opinions are great but they’re also not infallible or necessarily generalisable. Nor is it necessarily easy to tell who is an expert and who is a charlatan. Which is why we should use them as the basis for investigating the larger picture. Similarly both expert opinion and singular anecdotes can lead to cargo culting.