Features

Let's ROC, Part 7

Welcome back to part 7 of our series on ROC, other related statistics and charts

View Comments (0)Print ArticleEmail Article
Section Sponsored by:
http://fluidsurveys.com/surveys/merionmatters/success-story-contest

Welcome to the last section of ROCs. The two installments in this section discuss statistics related to ROCs in that these statistics help understand the laboratory data and its meaning to the physician and patient. The basis for much of this discussion is the 2 x 2 grid we have used before. The four statistics we consider are: the likelihood ratio (LR), the odds ratio (OR), the relative risk (or risk ratio, RR) and the hazard ratio (HR). These ratios are probably the most difficult concepts we will study. Even without the math, these concepts can be daunting. You may have to read these two sections more than once and work the examples yourself on paper to see how they can make sense. It is important that you understand these concepts so, when you read published data, you can understand and critique it properly. Additionally, you may find in your own work that one of them may be better than another or a better tool than an ROC curve.

The first of a series of four ratios is the likelihood ratio (LR) which is usually separated into the LR+ and the LR-. We begin with these two related metrics because they are derived from other statistics we discussed earlier -- sensitivity and specificity. Here is the grid followed by the formulas for them.

                                                                                                                 Diagnosis

                                                                                 Positive                               Negative

 Variable (elevated lab test) a (TP)           b (FP)

 Variable ('normal' lab test ) c (FN) d (TN)

 

                                                                Sensitivity = a/(a+c)

                                                                Specificity = d/(b+d)

The LRs are basically ratios of the probability that a test result is correct to the probability that the test result is incorrect. We can calculate for both positive (LR+) and negative test (LR-) The calculations are:

                                LR+ = sensitivity / 1- specificity

                                LR- = 1- sensitivity / specificity

Using the letters a - d, here are the formulas for the LRs:

                                LR+ = a/(a+c) / 1-(d/(b+d))

                                LR- = 1-a/(a+c) / (d/(b+d))

The LRs are usually combined with information about the:

  1. prevalence of the disease
  2. characteristics of the patient pool
  3. information about a particular patient

We will use the example of a laboratory test for d-dimer (a test to rule out deep vein thrombosis and pulmonary embolism) to illustrate these two terms. In an emergency room situation only about 1 in 5 (20%) of patients suspected of DVT or PE actually are finally diagnosed with either of these. The 2 x2 grid below is based on this statistic and published data on d-dimer.

 

 

Diagnosis

 

VTE

 

no-VTE

Pos

19

 

39

 

a

 

b

D-dimer

 

 

 

 

 

Neg

1

 

81

c

 

d

The sensitivity = a /(a+c) = 19/20 = 0.95.

The specificity = d/(b+d) = 81/(81+39) = 0.68.

From these data, the LR+ is sensitivity / (1-specificity) = 0.95/(1-0.68) or nearly 3.0. In its simplest expression, LR+ is equivalent to the probability that a person with the disease tested positive for the disease (TP) divided by the probability that a person without the disease tested positive for the disease (FP). Thus in our example the probability is about 3 times higher that a person with VTE gave a positive d-dimer than a person without VTE having a positive test. When comparing two or more diagnostic tests, the best test to use for ruling in a disease is the one with the largest LR+.

The LR- = (1-sensitivity)/ specificity) = (1 - a/(a+c)) / (d/(b+d))

 = (1- .95)/.68 = 0.07

LR- is equivalent to the probability that a person with the disease tested negative for the disease (FN) divided by the probability that a person without the disease tested negative for the disease (TP). In our example the probability is less than 1.0 that a person with VTE would be tested negative compared to a person who does not have VTE. When evaluating tests to use to rule out disease is to use the one with the smallest LR-. [It is noteworthy that a study of 'simply' 1) sensitivity and specificity 2) LRs and 3) a pictogram and physician's ability to evaluate a diagnostic test found no difference among the three tools. 

The next statistic is the odds ratio (OR). The OR may be used to compare whether the probability (chances) of a certain event (e.g. VTE) is the same for two groups of patients with different characteristics (for example: a group on a 'blood thinner' and a group not on such a drug). An odds ratio of 1 implies that the event (e.g. VTE) is equally likely in both groups. An odds ratio greater than 1.0 implies that the event is more likely in the first group (VTE). We will use the same data here as we did with the LRs.

The OR is the ratio of two other ratios: 1) a/b [TP/FP] and 2) c/d [FN/TN], which becomes a*d/b*c = (19*81)/(1*39). In our case that is 49. One way to phrase this is to say that the odds of a person having a high value for d-dimer and VTE is 49 times greater than a person having a high d-dimer and no DVT. [Suggestion: consider these ratios and 'odds' and say in your words the meaning of them and how you would describe the similarities and differences between them.]

To this point, we have studied a number of statistics that are used to evaluate a diagnostic test or to help a clinician understand to an extent what the test means or how the test will help explain the result to a patient. Here is a list of most of these tests with their formula using the 2x2 grid

Sensitivity           = TP/(TP+FN)                     LR+ = sensitivity / (1-specificity)

Specificity            = TN/(TN+FP)                    LR - = (1-sensitivity)/ specificity

PPV                        = TP/(TP+FP)                     OR = (TP*TN)/(FP*FN)

NPV                       = TN/(TN+FP)

We encourage you to put these equations on an Excel sheet and see what happens to each of these if, say, the cut off is moved up. And then moved down. This helps you solidify the meaning these have for selecting a test and for understanding what the various "odds" are for each of these.

Our next and last installment will discuss the last two statistics - the relative risk or risk ratio and the hazard ratio.




     

Email: *

Email, first name, comment and security code are required fields; all other fields are optional. With the exception of email, any information you provide will be displayed with your comment.

First * Last
Name:
Title Field Facility
Work:
City State
Location:

Comments: *
To prevent comment spam, please type the code you see below into the code field before submitting your comment. If you cannot read the numbers in the below image, reload the page to generate a new one.

Captcha
Enter the security code below: *

Fields marked with an * are required.

 
http://www.advanceweb.com/jobs/search/employer/1185/baycare-health-system.html
http://laboratory-manager.advanceweb.com/Web-Extras/Online-Extras/Editorial-Webinars.aspx
http://shop.advanceweb.com/clearance.html
http://www.stlukehealthnet.org
http://www.quickslide.com
http://www.advanceweb.com/jobs/search/jobview/501907/registered-nurses.html?jobb=#forward
http://www.advanceweb.com/jobs/search/employer/1318/dekalb-medical.html
http://www.advanceweb.com/jobs/search/employer/1183/supplemental-health-care.html