Two New Parameters Based on Distances in a Receiver Operating Characteristic Chart for the Selection of Classification Models

• Published in: Journal of chemical information and modeling Vol. 51; no. 10; pp. 2746 - 2759
• Main Authors: Pérez-Garrido, Alfonso, Helguera, Aliuska Morales, Borges, Fernanda, Cordeiro, M. Natália D. S, Rivero, Virginia, Escudero, Amalio Garrido
• Format: Journal Article
• Language: English
• Published: Washington, DC American Chemical Society 24-10-2011
• Subjects: Animals; Applied sciences; Biological and medical sciences; Biostatistics; Computational Biology - methods; Computer science; control theory; systems; Correlation analysis; Data processing. List processing. Character string processing; Databases, Factual; Euclidean space; Exact sciences and technology; General pharmacology; Medical sciences; Memory organisation. Data processing; Pharmaceutical Modeling; Pharmaceutical technology. Pharmaceutical industry; Pharmacology. Drug treatments; Quantitative Structure-Activity Relationship; ROC Curve; Software; Toxicity; Correlation coefficient; Capability index; Euclidean geometry; Statistical analysis; Probabilistic approach; Model selection; Virtual screening; Experimental study; Modeling; Linear model; Model matching; Structure activity relation; Genetic algorithm; Classification; Derivative; Receiver operating characteristic curves; Distance
• ISSN: 1549-9596; 1549-960X
• DOI: 10.1021/ci2003076

There are several indices that provide an indication of different types on the performance of QSAR classification models, being the area under a Receiver Operating Characteristic (ROC) curve still the most powerful test to overall assess such performance. All ROC related parameters can be calculated for both the training and test sets, but, nevertheless, neither of them constitutes an absolute indicator of the classification performance by themselves. Moreover, one of the biggest drawbacks is the computing time needed to obtain the area under the ROC curve, which naturally slows down any calculation algorithm. The present study proposes two new parameters based on distances in a ROC curve for the selection of classification models with an appropriate balance in both training and test sets, namely the following: the ROC graph Euclidean distance (ROCED) and the ROC graph Euclidean distance corrected with Fitness Function (FIT(λ)) (ROCFIT). The behavior of these indices was observed through the study on the mutagenicity for four genotoxicity end points of a number of nonaromatic halogenated derivatives. It was found that the ROCED parameter gets a better balance between sensitivity and specificity for both the training and prediction sets than other indices such as the Matthews correlation coefficient, the Wilk’s lambda, or parameters like the area under the ROC curve. However, when the ROCED parameter was used, the follow-on linear discriminant models showed the lower statistical significance. But the other parameter, ROCFIT, maintains the ROCED capabilities while improving the significance of the models due to the inclusion of FIT(λ).