3 ms·

It's a big assumption though, yes? Namely, that the accuracy of information, or lack thereof, will somehow be consistent across topic areas? It seems reasonabl

It's a big assumption though, yes? Namely, that the accuracy of information, or lack thereof, will somehow be consistent across topic areas? It seems reasonable to me to expect journalists to have a better grasp of certain things than others (say politics or law instead of physics), and for this to vary across writers (one writer might have a background in chemistry; another in population biology). I also think there's a certain variability of opinion on certain topics when you get to a certain level, and experts aren't always aware of this. That is, some (not all) tend to believe they are infallible and everyone else who disagrees is incorrect, or sometimes, "unscientific." There's probably several ways this could play a role, but I do think it plays some role in this. I think this Gell-Mann amnesia is sort of a real bias that's interesting to think about, but I also think in reality it might not be quite what's being assumed in this essay.

godelski 5y ago
> It's a big assumption though, yes? Namely, that the accuracy of information, or lack thereof, will somehow be consistent across topic areas? Yes and no. It's more a method to rule out, not rule in. If a presenter gives bad information about something I know I can't trust them. If a presenter gives good information about something I know, it builds evidence that I can trust them. These two are different. The latter isn't complete trust and still maintains skepticism. At the end of the day, we have to have some trust. But that also doesn't mean absolute.
- carabiner 5y ago
  What if a presenter does both?
  - godelski 5y ago
    On subjects I'm an expert in? That undoes the evidence for trust, right? On subjects I'm not an expert in? Well I can't know. The problem here is that I'm not an expert in everything. In fact, I'm not an expert in most things. I feel like people are taking this test as a perfect test. Far from it. There is no such test and never will be. But to everyone that is trying to counter this I'll ask you the inverse question: "Why should I trust someone when they've proven to mischaracterize a topic?" That's the heart of this metric. If you've proven that you won't do enough research, I have reason to not believe you. If you've proven that you will do enough research, I have reason to believe you. It is statistical. Everything is. It is imperfect. Everything is. If you got suggestions about how to extend this and improve upon this, I'm listening. But I'm not just going to abandon it and start listening to people that have given me strong evidence to distrust them. That's a bit insane. It is also exactly what the Gell-Mann amnesia effect is making fun of: trusting people in things you don't know about that proved to be untrustworthy in things you do.

godelski 5y ago

> It's a big assumption though, yes? Namely, that the accuracy of information, or lack thereof, will somehow be consistent across topic areas? Yes and no. It's more a method to rule out, not rule in. If a presenter gives bad information about something I know I can't trust them. If a presenter gives good information about something I know, it builds evidence that I can trust them. These two are different. The latter isn't complete trust and still maintains skepticism. At the end of the day, we have to have some trust. But that also doesn't mean absolute.

carabiner 5y ago
What if a presenter does both?
- godelski 5y ago
  On subjects I'm an expert in? That undoes the evidence for trust, right? On subjects I'm not an expert in? Well I can't know. The problem here is that I'm not an expert in everything. In fact, I'm not an expert in most things. I feel like people are taking this test as a perfect test. Far from it. There is no such test and never will be. But to everyone that is trying to counter this I'll ask you the inverse question: "Why should I trust someone when they've proven to mischaracterize a topic?" That's the heart of this metric. If you've proven that you won't do enough research, I have reason to not believe you. If you've proven that you will do enough research, I have reason to believe you. It is statistical. Everything is. It is imperfect. Everything is. If you got suggestions about how to extend this and improve upon this, I'm listening. But I'm not just going to abandon it and start listening to people that have given me strong evidence to distrust them. That's a bit insane. It is also exactly what the Gell-Mann amnesia effect is making fun of: trusting people in things you don't know about that proved to be untrustworthy in things you do.