3 ms·

Your statistic (good hires / total hires), using the jargon from your link, is precision or positive predictive value (PPV). Using the math from that link, if

by fsk 12y ago

Your statistic (good hires / total hires), using the jargon from your link, is precision or positive predictive value (PPV). Using the math from that link, if you decrease FN, then PPV increases.

deleted 12y ago
[deleted]
sytelus 12y ago
Sorry, I did mixed up precision in my reply. I just got time to think about this whole debate more carefully and I realize you are actually right if we fix up some of the terminology you have used. The mis-statements and confusion on my part has occurred due to this terminology differences. First FP and FN are not probabilities. They are just unbounded numbers. This may feel pedantic but in a moment I'll show you why this is critical. Let me draw the confusion matrix first (G = Good candidates, H = Hired candidate etc): \ H NH \--------- G | TP FN B | FP TN What you are referring to as probabilities is actually False Positive Rate or FPR and TNR respectively which is defined as follows: FPR = FP / (FP + TN) = FP/B FNR = FN / (TP + FN) = FN/G Now the quantity you are after is probability that given you did hiring and ended up with good guy which is, nothing but precision: precision = P(G|H) = TP/H So how do we get TP to calculate precision if we only knew FPR, FNR, G and B? I did little equation gymnastics using above and got below: TP = G - GFNR H = TP + FP = TP + FPRB So now you can plug this in to above equation for precision and find that as you increase FNR, precision goes down while you keep FPR constant. So you are actually correct. Although it might look like unnecessary exercise vs following intuition I think above equation can actually help calculate exact drop in precision and multiply that with cost of FP vs FN to get the operating sweet spot. On my part I need to do some soul searching to figure out why this didn't triggered to me before :).

by fsk 12y ago

Your statistic (good hires / total hires), using the jargon from your link, is precision or positive predictive value (PPV). Using the math from that link, if you decrease FN, then PPV increases.

deleted 12y ago

[deleted]

sytelus 12y ago

Sorry, I did mixed up precision in my reply. I just got time to think about this whole debate more carefully and I realize you are actually right if we fix up some of the terminology you have used. The mis-statements and confusion on my part has occurred due to this terminology differences. First FP and FN are not probabilities. They are just unbounded numbers. This may feel pedantic but in a moment I'll show you why this is critical. Let me draw the confusion matrix first (G = Good candidates, H = Hired candidate etc): \ H NH \--------- G | TP FN B | FP TN What you are referring to as probabilities is actually False Positive Rate or FPR and TNR respectively which is defined as follows: FPR = FP / (FP + TN) = FP/B FNR = FN / (TP + FN) = FN/G Now the quantity you are after is probability that given you did hiring and ended up with good guy which is, nothing but precision: precision = P(G|H) = TP/H So how do we get TP to calculate precision if we only knew FPR, FNR, G and B? I did little equation gymnastics using above and got below: TP = G - GFNR H = TP + FP = TP + FPRB So now you can plug this in to above equation for precision and find that as you increase FNR, precision goes down while you keep FPR constant. So you are actually correct. Although it might look like unnecessary exercise vs following intuition I think above equation can actually help calculate exact drop in precision and multiply that with cost of FP vs FN to get the operating sweet spot. On my part I need to do some soul searching to figure out why this didn't triggered to me before :).