Welcome to the fifth installment of our series on ROC curves, the associated statistics and other related statistics. In this installment, we will discuss the dollar costs of false positive and false negative results and the area under the sensitivity vs. 1-specificity curve.
Review the sketch below. Recall that, as the cut-off increases (left to right), the sensitivity (number of diseased patients with a positive laboratory result divided by the number of patients labeled positive by the clinician(s)) decreases. As the cut-off increases, the specificity (the number of patients who had a 'normal' laboratory result divided by the number of non-diseased per the clinician(s)) increases.
Now, take a few seconds to study the table below. Note that there many more non-diseased patients (122) than diseased patients (32). This is usual. For example, only about 20 percent of patients reporting to the ER with signs and symptoms of DVT will have a final diagnosis of DVT.
At this point, we want to discuss the expense associated with false positives (FP, a positive laboratory result on a patient labeled non-disease by the clinician(s)) and false negatives (FN, a 'normal' laboratory result on a patient the clinician(s) labeled positive). In most, if not all cases, it is difficult to assign a dollar to both FP and FN. For example, in some cases, an FP will be detected with another laboratory test; while, in others, it may require a scan or biopsy. An FN can be life-threatening or detected with another laboratory test. Given this, it might be misleading to put an "average" dollar amount to FP and FN. With this in mind, we have done so -- if only to give you an idea of how such an exercise might turn out. This may have use when discussing cut-offs, or when considering a reflexive laboratory protocol.
Efficiency = per cent correct (lab result matches diagnosis); NPV = negative predictive value or TN/(TN+FN); PPV = positive predictive value or TP/(TP+FP).
Questions regarding the Table and the Figure:
1. As the cut off moves higher what happens to the sensitivity? The specificity?
2. As the cut off moves higher what happens to the efficiency? The number of laboratory tests that agree with the diagnosis (the number correct)? Why do these go in the same direction?
3. Why is the total cost lower at cut off #2? Sketch an x-y plot of sensitivity (x) vs. total cost (y). What does this tell us about the importance of sensitivity?