3 ms·

Dissing Track Records

17 points by blfr 12y ago

patio11 12y ago
I naively asked a lawyer what his win rate was when I was seven. He explained, fairly patiently, that that was an exceptionally poor metric to judge lawyers by. "Few disputes end up in court. Most are settled out of court, with trials meaning a negotiation broke down or one thought one had a slight edge. If they have a major edge, you beg for a settlement. If you do, typically their lawyer will. If you never lose a case it's because you are being insufficiently aggressive on behalf of clients with strong-but-not-unbeatable claims and are disposing of them pre-trial to the detriment of the client. A lawyer who would do that is a poor lawyer regardless of the stats."
- rwallace 12y ago
  Maybe so. On the other hand, a lawyer who loses too many cases is likely being too aggressive on behalf of clients with poor claims, to the detriment of the client, the other party and society in general.
  - achamayou 12y ago
    A lot like code coverage. A good rate doesn't mean you're doing great, but a poor rate is a good indicator that you have a problem.
topkai22 12y ago
While I'd be interested in the track record data, it is hard to properly metric these professions. How do you control for a priori risk of a patient (IE- one total knee case is a 70 year old smoker so ways 300 lbs and the other is a 28 year old athlete who fell skiing.) The ACA has resulted in the collection of this type of data and we'll probably see it released someday. We'll almost certainly see complication rates reduced and risky v patients refused certain types of care, but I do feel bad for the professions getting metriced like this. After all, I'd be pretty pissed if my value as a software developer was judge entirely by my bugs/KSLOC number.
- topkai22 12y ago
  BTW - these these sort of metrics are much more useful in a internal evaluative scenario- a hospital talking to its doctors, a principal deciding which teaches to retain, etc... Because those decision makers have much more access to the qualitative aspects of the work.
  - tmarthal 12y ago
    I am actually working at a startup to do just this, but using the 'Choosing Wisely' guidelines for care practitioners.
rwallace 12y ago
A very interesting point. Note the pattern in responses: 1. Lots of people criticizing the proposed metrics. 2. Hardly anyone proposing ways to improve them (e.g. augmenting them with information about the difficulty of the cases that were taken). 3. Hardly anyone pointing out that even worse criticisms can be made of the informal criteria customers currently use (the shape of someone's nose, where he went to school, who he knows, the amount of money that was spent on his office). Most of the criticisms made in #1 are true as far as they go, so looking at any one of them in isolation wouldn't tell you anything much. It's the overall pattern that's informative.

17 points by blfr 12y ago

I naively asked a lawyer what his win rate was when I was seven. He explained, fairly patiently, that that was an exceptionally poor metric to judge lawyers by. "Few disputes end up in court. Most are settled out of court, with trials meaning a negotiation broke down or one thought one had a slight edge. If they have a major edge, you beg for a settlement. If you do, typically their lawyer will. If you never lose a case it's because you are being insufficiently aggressive on behalf of clients with strong-but-not-unbeatable claims and are disposing of them pre-trial to the detriment of the client. A lawyer who would do that is a poor lawyer regardless of the stats."

rwallace 12y ago
Maybe so. On the other hand, a lawyer who loses too many cases is likely being too aggressive on behalf of clients with poor claims, to the detriment of the client, the other party and society in general.
- achamayou 12y ago
  A lot like code coverage. A good rate doesn't mean you're doing great, but a poor rate is a good indicator that you have a problem.

topkai22 12y ago

While I'd be interested in the track record data, it is hard to properly metric these professions. How do you control for a priori risk of a patient (IE- one total knee case is a 70 year old smoker so ways 300 lbs and the other is a 28 year old athlete who fell skiing.) The ACA has resulted in the collection of this type of data and we'll probably see it released someday. We'll almost certainly see complication rates reduced and risky v patients refused certain types of care, but I do feel bad for the professions getting metriced like this. After all, I'd be pretty pissed if my value as a software developer was judge entirely by my bugs/KSLOC number.

topkai22 12y ago
BTW - these these sort of metrics are much more useful in a internal evaluative scenario- a hospital talking to its doctors, a principal deciding which teaches to retain, etc... Because those decision makers have much more access to the qualitative aspects of the work.
- tmarthal 12y ago
  I am actually working at a startup to do just this, but using the 'Choosing Wisely' guidelines for care practitioners.

rwallace 12y ago

A very interesting point. Note the pattern in responses: 1. Lots of people criticizing the proposed metrics. 2. Hardly anyone proposing ways to improve them (e.g. augmenting them with information about the difficulty of the cases that were taken). 3. Hardly anyone pointing out that even worse criticisms can be made of the informal criteria customers currently use (the shape of someone's nose, where he went to school, who he knows, the amount of money that was spent on his office). Most of the criticisms made in #1 are true as far as they go, so looking at any one of them in isolation wouldn't tell you anything much. It's the overall pattern that's informative.