Receiver Operating Characteristics (ROC) curve
What is an ROC curve?
ROC curves are made to help decide on a good cutoff point for a test. If the cutoff is very high, there will be too many false negatives but few false positives. Whereas, if the cutoff is too low there will be too many false positives and few false negatives. To do this, the true positive rate (sensitivity) and false positive rate (1 – specificity) of the test are calculated for several possible cutoff points. Then these are plotted to create the ROC curve and the area under the curve can be measured to determine the overall accuracy of a test.
Notice that the ROC curve plots the sensitivity against the 1-specificity which is essentially the positive likelihood ratio (LR+). Remember, LR+ = sensitivity/ (1-specificity). Therefore, the ROC curve can be considered a plot of the LR+.
Let’s create a sample ROC curve…
You want to determine the best cutoff for test A which is used to diagnose disease A. A sample population of 10,000 people is taken and each is given test A. To create the ROC curve, several cutoffs need used and each plotted as a point on a graph.
The first point will be when the cutoff is set to the minimum test result possible, where everyone will be diagnosed as having the disease i.e. sensitivity (true positive rate) will be 100%. However, the false-positive rate (1-specificity) will also be 100%. This point will go in the top right corner of our graph. The next point will be when the cutoff is set to the maximum test result possible, where no will be diagnosed as having the disease, making the sensitivity 0% and the false-positive rate 0%. This point will go in the bottom left corner of our graph. Next, we need to plot the best possible cutoff point, i.e. the one that will allow us to diagnose as many of those with the disease while avoiding false positives. This means getting close to 100% sensitivity and 0% false positive rate, or getting close to top left corner of the graph.
The most accurate or “perfect” ROC curve will have 100% sensitivity and 0% false positives. This curve will go from the bottom left corner then up to the top left corner then over to the top right corner, and the area under this will be 1.0. When the sensitivity and the false positive rate of a test are always equal, we get a diagonal line as our curve. This ROC curve is often called “no benefit”. Most tests, will have an ROC curve between “no benefit” and “perfect”.
Let’s compare the ROC curves of several tests…
When comparing ROC curves, the better or more accurate curve will be closer the the “perfect” ROC curve described above. The LR+ is essentially our ROC curve, therefore, when the curve is closer to perfect, i.e. the cutoff point for the test allows us to get close to 100% sensitivity and 0% false positives, the greater our LR+.
Let’s do some sample problems…
- A series of new tests are developed to diagnose a new flu virus. A receiver operating characteristics curve is made for each test. Which test is the most accurate?
Answer: Curve C is the most accurate because it has the greatest sensitivity and lowest false positive rate, i.e. it is closer to the top left corner (100% sensitivity, 0% false positive rate).
