3 ms·

Most patient-reported outcome measures are designed to provide an objective assessment of a patient's health status, for example in urology many of the surveys

Most patient-reported outcome measures are designed to provide an objective assessment of a patient's health status, for example in urology many of the surveys ask specific questions about urinary symptoms, eg. how many times did you have to get up in the night to urinate, or in orthopedics whether you had difficulty performing specific tasks related to your joint. Some aspects are always going to be subjective (eg. impact on quality of life or pain) and IMHO that's OK and we should absolutely attempt to measure them, not least because that information could help inform the treatment itself. Also, by measuring the changes in response over time for a patient, you can attempt to control for individual biases. I agree that patient satisfaction surveys (in the UK, categorized as patient reported experience measures) can be very prone to bias and while important, are not necessarily correlated with outcomes.

nradov 9y ago
The trouble with that is it doesn't establish causality. Did the patient's symptoms improve because of the provider's intervention or in spite of it? Plus you can't force patients to respond to the survey so there's no way to know if you have a representative sample. In my subjective experience, patients that are happy and improving are more likely to answer those questions than patients who are unhappy and ill.
- Retric 9y ago
  X people per hour for over a few years averages out much random noise. Even surgeons see on average multiple people per day over a few years which is still recent enough to be relevant.
  - nradov 9y ago
    No it doesn't average out at all due to persistent differences in patient populations across providers which can't be adequately controlled for using the available data.
    - Retric 9y ago
      You misunderstand. You have a representative sample of the patients seen by that doctor. Individual differences like patient weight may be very important at the individual level, but across a thousands of people that's far less important than the overall differences across populations.
      - nradov 9y ago
        No that's simply not how it works and won't give you an accurate picture of quality differences between providers. I don't know how to make it any more clear.
        Retric 9y ago
        Comparisons between providers is a separate issue. Suppose you assigned people randomly to 101 doctors from 2 populations (A,B). Now suppose A was 10x as likely to die. D(0) get's 0% of A's and 100% B's. D(1) is 1A and 99B. All the way to D(100) that only get's B's. In that admittedly simplified example you could determine that D(0) did a better job than D(100) by only getting 8x as many deaths even if 9.8x may be statistically irrelevant. Yes, the real world is vastly more complex. But, while that may make a strict ordering impossible you can likely find out the best doctor is likely in the top quarter and the worst doctor very likely in the bottom quarter, which can be useful. Picture a score card that said 80% chance in (0% - 20%], 15% chance in the (25%-50%], etc. That's not exactly meaningless information.
        nradov 9y ago
        Nope that doesn't work because there's still no way to reliably control for the confounding factors. We don't even know what all the confounding factors are. Patients aren't randomly assigned to providers.
        Retric 9y ago
        > Patients aren't randomly assigned to providers. Sure, we don't have that information today. But, if we want to collect relevant information we could easily have a subset of doctors with random patient assignment. IMO, that's simply an implementation detail required if we want good information. PS: US healthcare spending is over 3 trillion per year, rationally even minor improvements are worth large investments.
  - bigfudge 9y ago
    The real issue here is the selection bias in the caseloads of good vs poor physicians. In psychological treatment teams it's common for the caseload to be (implicitly or explicitly) allocated based on the perceived skill of team members. Note — this may not necessarily correlate with their actual skill, but it still screws up any estimates of provider performance unless you have very good prognostic indicators of outcome from before treatment (and you likely won't).
imglorp 9y ago
I imagine the time aspect is especially important in some areas. For example patient impressions of orthopedic surgery is guaranteed to be terrible in the short term, but it's the long term outcome--eg mobility--that might really matter, after everything heals and recovers.

nradov 9y ago

The trouble with that is it doesn't establish causality. Did the patient's symptoms improve because of the provider's intervention or in spite of it? Plus you can't force patients to respond to the survey so there's no way to know if you have a representative sample. In my subjective experience, patients that are happy and improving are more likely to answer those questions than patients who are unhappy and ill.

Retric 9y ago
X people per hour for over a few years averages out much random noise. Even surgeons see on average multiple people per day over a few years which is still recent enough to be relevant.
- nradov 9y ago
  No it doesn't average out at all due to persistent differences in patient populations across providers which can't be adequately controlled for using the available data.
  - Retric 9y ago
    You misunderstand. You have a representative sample of the patients seen by that doctor. Individual differences like patient weight may be very important at the individual level, but across a thousands of people that's far less important than the overall differences across populations.
    - nradov 9y ago
      No that's simply not how it works and won't give you an accurate picture of quality differences between providers. I don't know how to make it any more clear.
      - Retric 9y ago
        Comparisons between providers is a separate issue. Suppose you assigned people randomly to 101 doctors from 2 populations (A,B). Now suppose A was 10x as likely to die. D(0) get's 0% of A's and 100% B's. D(1) is 1A and 99B. All the way to D(100) that only get's B's. In that admittedly simplified example you could determine that D(0) did a better job than D(100) by only getting 8x as many deaths even if 9.8x may be statistically irrelevant. Yes, the real world is vastly more complex. But, while that may make a strict ordering impossible you can likely find out the best doctor is likely in the top quarter and the worst doctor very likely in the bottom quarter, which can be useful. Picture a score card that said 80% chance in (0% - 20%], 15% chance in the (25%-50%], etc. That's not exactly meaningless information.
        nradov 9y ago
        Nope that doesn't work because there's still no way to reliably control for the confounding factors. We don't even know what all the confounding factors are. Patients aren't randomly assigned to providers.
        Retric 9y ago
        > Patients aren't randomly assigned to providers. Sure, we don't have that information today. But, if we want to collect relevant information we could easily have a subset of doctors with random patient assignment. IMO, that's simply an implementation detail required if we want good information. PS: US healthcare spending is over 3 trillion per year, rationally even minor improvements are worth large investments.
- bigfudge 9y ago
  The real issue here is the selection bias in the caseloads of good vs poor physicians. In psychological treatment teams it's common for the caseload to be (implicitly or explicitly) allocated based on the perceived skill of team members. Note — this may not necessarily correlate with their actual skill, but it still screws up any estimates of provider performance unless you have very good prognostic indicators of outcome from before treatment (and you likely won't).

imglorp 9y ago

I imagine the time aspect is especially important in some areas. For example patient impressions of orthopedic surgery is guaranteed to be terrible in the short term, but it's the long term outcome--eg mobility--that might really matter, after everything heals and recovers.