1887
banner image
No data available.
Please log in to see this content.
You have no subscription access to this content.
No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
Assessing operating characteristics of CAD algorithms in the absence of a gold standard
Rent:
Rent this article for
USD
10.1118/1.3352687
/content/aapm/journal/medphys/37/4/10.1118/1.3352687
http://aip.metastore.ingenta.com/content/aapm/journal/medphys/37/4/10.1118/1.3352687

Figures

Image of FIG. 1.
FIG. 1.

Distribution of test statistics for (a) differences of sensitivity and (b) differences of average false positives per patient between reader panel and CAD readings for LIDC data set. Distributions were calculated under null hypothesis of no difference between reading methods using a randomization test (with 1000 replications). The value of the observed test statistic is indicated by a dotted vertical line.

Image of FIG. 2.
FIG. 2.

Latent class estimates plotted against true lesion status for one replication of simulation experiment ( lesion candidates, 10% true lesions, four readers each in reference and CAD assisted groups). True sensitivity and specificity for individual readers is and in reference group, and and in CAD group. Estimated sensitivity and specificity for individual readers is and in reference group, and and in CAD group. Note that estimated latent class values are probabilities, which can take any value between 0 (not a lesion) and 1 (lesion). Using a decision rule of assigning lesion status if the estimated status value is more than 0.5 (depicted by dotted line), the misclassification error rate is 0.009. Misclassified points are shown as crossed circles.

Image of FIG. 3.
FIG. 3.

Distribution of estimated lesion status values estimated using latent class methodology for LIDC data set with lesion candidates, readers in both reference and CAD groups (0 is definitely not a nodule, 1 is definitely a nodule). Estimated parameters are given in Table III.

Tables

Generic image for table
TABLE I.

Results of low sensitivity, high specificity simulation experiment. Comparison of a) sensitivity and b) specificity estimates of a CAD assisted group from simulation experiment with four readers per group, 1000 lesion candidates. The fraction of nodule candidates that are actually positive is . Operating characteristics of individual readers in the reference panel are given in rows as sensitivity (Se) and in columns as specificity (Sp). The operating characteristics of the CAD assisted group are set as: , . White columns (RP, reader panel) are estimated apparent sensitivities using lesions identified by three or more reference readers as gold standard. Gray shaded columns (LCA) are estimated sensitivities using LCA in the same datasets. Each reported value is the average of 500 replicates. For sensitivity, average standard deviations of the reader panel and LCA based estimates are 0.02 and 0.03, respectively. For specificity, average standard deviations of the reader panel and LCA based estimates are 0.005 and 0.007, respectively.

Generic image for table
TABLE II.

Results of high sensitivity, low specificity simulation experiment. Comparison of a) sensitivity and b) specificity estimates of CAD assisted group from simulation experiment with four readers per group, 1000 lesion candidates. Fraction of nodule candidates that are actually positive is . Operating characteristics of individual readers in the reference panel are given in rows as sensitivity (Se) and in columns as specificity (Sp). The operating characteristics of the CAD assisted group are set as: , . White columns (RP, reader panel) are estimated apparent sensitivities using lesions identified by three or more reference readers as gold standard. Grey shaded columns (LCA) are estimated sensitivities using LCA in the same datasets. Each reported value is the average of 500 replicates. For sensitivity, average standard deviations of the reader panel and LCA based estimates are both 0.01. For specificity, average standard deviations reader panel and LCA based estimates are 0.008 and 0.017, respectively.

Generic image for table
TABLE III.

Estimates based upon LCA for the LIDC dataset, with nodule candidates, readers in both free read and CAD assisted groups. , : Sensitivity of reference (free read) and CAD groups, respectively. , : Average false positives per patient of reference and CAD assisted groups, respectively. : Fraction of TP nodules in data. Naïve estimates of standard error are calculated using a formula for binomial proportions, which assumes conditional independence of diagnoses across readers and nodule candidates (third column). Naïve estimates of standard errors can’t be computed for FPP because the number of true negative nodules is unknown. Robust estimates of standard error, which do not assume independence, are obtained from bootstrap resampling of data (fourth column).

Loading

Article metrics loading...

/content/aapm/journal/medphys/37/4/10.1118/1.3352687
2010-03-29
2014-04-24
Loading

Full text loading...

This is a required field
Please enter a valid email address
752b84549af89a08dbdd7fdb8b9568b5 journal.articlezxybnytfddd
Scitation: Assessing operating characteristics of CAD algorithms in the absence of a gold standard
http://aip.metastore.ingenta.com/content/aapm/journal/medphys/37/4/10.1118/1.3352687
10.1118/1.3352687
SEARCH_EXPAND_ITEM