4 ms·

For a rational agent the expected value of information is never negative. (Of course it can be less than the cost of the test.) If a system is treating it as ne

For a rational agent the expected value of information is never negative. (Of course it can be less than the cost of the test.) If a system is treating it as negative, that's a problem to fix. I'm reminded of a recent visit to a doctor who refused to say anything quantitative about a test. Sure, your average client knows little decision theory, but this doctor either didn't either or pretended not to follow. To make a good decision you need both probabilities and utilities; the doctor presumably best knows the former while the client best knows and cares about the latter, so they need to be able to communicate this way.

arkades 7y ago
Cool, so your serum zorblaxian levels are 300 ng/L. The finding was incidental, so it’s jot tied to existing symptoms / clinical suspicion. It’s a molecule involved in the inflammatory cascade. We don’t know what diseases it’s associated with or not, nor what the change in probability of having the diseases it could be associated with are, not change in prognosis. But, you have a number which your local lab pegs at 1 standard deviation above the mean for the sample they calibrated their measurement technique on. So, should we now do a work-up for every disease known to man with an inflammatory component? If you have one, if, every test - with their attendant complications and costs - for every disease you don’t have will have net negative effect on your health. You baseline have no symptoms or clinical suspicion for any of these (again, tautology or incidental finding), so the baseline probability is you actually have no disease at all, and this is a spurious value. So, what’s the advantage of this incidental finding?
- abecedarius 7y ago
  Framing it as a choice between "have a cookbook decision tree that's been validated by RCTs in exactly this context" and "do a lot of costly (in resources and side effects) followup tests to show we're doing everything we can" is an example of the irrationality that needs fixing. People are capable of actually reasoning with uncertainty when they care about outcomes. When it's you, personally, facing a decision in your life, you don't throw up your hands and say "It's too complicated to decide! There are no actuarial tables about exactly this situation!" Having to decide means having to make a bet. You can be a better or a worse gambler, but we're all gamblers. Medicine as a profession seems to want to pretend they're not, rather like "investors" pretend to be qualitatively different from "speculators".
  - arkades 7y ago
    It’s not “cookbook decision tree”, it’s “data from which to make meaningful inferences about this finding.” It would be nice if we always have that; with incidental findings, we often don’t.
    - abecedarius 7y ago
      Put it this way: my doctor ordered a test for me, got the results, and made a recommendation. I was ignorant of both the costs (of all sorts) beforehand and of the product of probability and utility on which he (implicitly and unavoidably unless irrationally) based the decision. He did make a meaningful inference from this data, as anyone must to make a decision; he just refused to help me, as a consultant instead of a master, to make my own.
- bsder 7y ago
  > Cool, so your serum zorblaxian levels are 300 ng/L. The finding was incidental, so it’s jot tied to existing symptoms / clinical suspicion. It’s a molecule involved in the inflammatory cascade. This isn't that hard. You look at it again in 6 months. And you ask "Did it go up? Did it go down? Is this just your baseline?" And you keep your eyes open a little more strongly for issues that might correlate. And, you know what, I, the patient, am FAR more invested in keeping tabs on whether things are going right or wrong in my own body than any doctor.
  - MegaButts 7y ago
    > And, you know what, I, the patient, am FAR more invested in keeping tabs on whether things are going right or wrong in my own body than any doctor. As someone who has had to figure out his health on his own after multiple doctors have failed or many never even attempted to help, I cannot agree with this more.
mattkrause 7y ago
We could do an autopsy on you right now if you want. How sure are you that you just stubbed your toe and it's not some highly metastasized cancer?
- abecedarius 7y ago
  "Of course it can be less than the cost of the test." includes non-monetary costs. I thought this was clear since the benefit we're talking about is obviously not wholly monetary. (Or even mostly.)
  - mattkrause 7y ago
    I suppose you're right that the expected value of information, not including the acquisition costs, is never negative. That feels like cheating though--this information isn't free. My argument is that the expected value of this information, including acquisition costs, can very easily be negative, since you can't just "ignore" being dead. These costs are extremely large in my snarky autopsy suggestion, but dominate for whole-body MRIs + followups too. The catch is that the scan doesn't have huge benefits and the costs of the followups, while non-trivial, aren't exactly open-heart surgery either, so both are in the neighborhood of zero, except for the monetary expense, which will be crazy high.
    - abecedarius 7y ago
      Yup, agreed! In fact I wouldn't have done the test mentioned above if I'd been better informed (an MRI with gadolinium contrast; I was only told the latter bit the day I showed up for it. There seems to be a bit of controversy about possible low-level health effects of this contrast agent, and considering I was very probably fine a priori, I think it was a mistake to do this. But it's kind of another thing to decide this on the spot after making an appointment.) (To say nothing of the extra $1000 surprise bill...) Anyway, what is raising my hackles doesn't seem rational in the same way. It seems to have the flavor of "when we look and see certain signs, our system too often takes stupid harmful costly actions on that information. Therefore, don't look!" It'd be reassuring to continue "This is a workaround to an admittedly irrational system while we have the following people researching better ways to improve." But that's not the sort of thing I've read.

arkades 7y ago

Cool, so your serum zorblaxian levels are 300 ng/L. The finding was incidental, so it’s jot tied to existing symptoms / clinical suspicion. It’s a molecule involved in the inflammatory cascade. We don’t know what diseases it’s associated with or not, nor what the change in probability of having the diseases it could be associated with are, not change in prognosis. But, you have a number which your local lab pegs at 1 standard deviation above the mean for the sample they calibrated their measurement technique on. So, should we now do a work-up for every disease known to man with an inflammatory component? If you have one, if, every test - with their attendant complications and costs - for every disease you don’t have will have net negative effect on your health. You baseline have no symptoms or clinical suspicion for any of these (again, tautology or incidental finding), so the baseline probability is you actually have no disease at all, and this is a spurious value. So, what’s the advantage of this incidental finding?

abecedarius 7y ago
Framing it as a choice between "have a cookbook decision tree that's been validated by RCTs in exactly this context" and "do a lot of costly (in resources and side effects) followup tests to show we're doing everything we can" is an example of the irrationality that needs fixing. People are capable of actually reasoning with uncertainty when they care about outcomes. When it's you, personally, facing a decision in your life, you don't throw up your hands and say "It's too complicated to decide! There are no actuarial tables about exactly this situation!" Having to decide means having to make a bet. You can be a better or a worse gambler, but we're all gamblers. Medicine as a profession seems to want to pretend they're not, rather like "investors" pretend to be qualitatively different from "speculators".
- arkades 7y ago
  It’s not “cookbook decision tree”, it’s “data from which to make meaningful inferences about this finding.” It would be nice if we always have that; with incidental findings, we often don’t.
  - abecedarius 7y ago
    Put it this way: my doctor ordered a test for me, got the results, and made a recommendation. I was ignorant of both the costs (of all sorts) beforehand and of the product of probability and utility on which he (implicitly and unavoidably unless irrationally) based the decision. He did make a meaningful inference from this data, as anyone must to make a decision; he just refused to help me, as a consultant instead of a master, to make my own.
bsder 7y ago
> Cool, so your serum zorblaxian levels are 300 ng/L. The finding was incidental, so it’s jot tied to existing symptoms / clinical suspicion. It’s a molecule involved in the inflammatory cascade. This isn't that hard. You look at it again in 6 months. And you ask "Did it go up? Did it go down? Is this just your baseline?" And you keep your eyes open a little more strongly for issues that might correlate. And, you know what, I, the patient, am FAR more invested in keeping tabs on whether things are going right or wrong in my own body than any doctor.
- MegaButts 7y ago
  > And, you know what, I, the patient, am FAR more invested in keeping tabs on whether things are going right or wrong in my own body than any doctor. As someone who has had to figure out his health on his own after multiple doctors have failed or many never even attempted to help, I cannot agree with this more.

mattkrause 7y ago

We could do an autopsy on you right now if you want. How sure are you that you just stubbed your toe and it's not some highly metastasized cancer?

abecedarius 7y ago
"Of course it can be less than the cost of the test." includes non-monetary costs. I thought this was clear since the benefit we're talking about is obviously not wholly monetary. (Or even mostly.)
- mattkrause 7y ago
  I suppose you're right that the expected value of information, not including the acquisition costs, is never negative. That feels like cheating though--this information isn't free. My argument is that the expected value of this information, including acquisition costs, can very easily be negative, since you can't just "ignore" being dead. These costs are extremely large in my snarky autopsy suggestion, but dominate for whole-body MRIs + followups too. The catch is that the scan doesn't have huge benefits and the costs of the followups, while non-trivial, aren't exactly open-heart surgery either, so both are in the neighborhood of zero, except for the monetary expense, which will be crazy high.
  - abecedarius 7y ago
    Yup, agreed! In fact I wouldn't have done the test mentioned above if I'd been better informed (an MRI with gadolinium contrast; I was only told the latter bit the day I showed up for it. There seems to be a bit of controversy about possible low-level health effects of this contrast agent, and considering I was very probably fine a priori, I think it was a mistake to do this. But it's kind of another thing to decide this on the spot after making an appointment.) (To say nothing of the extra $1000 surprise bill...) Anyway, what is raising my hackles doesn't seem rational in the same way. It seems to have the flavor of "when we look and see certain signs, our system too often takes stupid harmful costly actions on that information. Therefore, don't look!" It'd be reassuring to continue "This is a workaround to an admittedly irrational system while we have the following people researching better ways to improve." But that's not the sort of thing I've read.