On Fair Performance Comparison between Random Survival Forest and Cox Regression: An Example of Colorectal Cancer Study

• Published in: SciMedicine journal Vol. 3; no. 1; pp. 66 - 76
• Main Authors: Cetin, Sirin, Ulgen, Ayse, Dede, Isa, Li, Wentian
• Format: Journal Article
• Language: English
• Published: Ital Publication 01-03-2021
• Subjects: brier’s score; colorectal cancer; cox regression; discordance index; machine learning; random survival forest
• ISSN: 2704-9833; 2704-9833
• DOI: 10.28991/SciMedJ-2021-0301-9

Random Forest (RF), a mostly model-free and robust machine learning method, has been successfully applied to right-censored survival data, under the name of Random Survival Forest (RSF). However, RF/RSF has its distinct strategies in classification and prediction. First, it is an ensemble classifier and its performance is an average of multiple rounds of data fitting. Second, the training set is a bootstrap (sampling with replacement) generated set with repeated used of roughly 2/3 of all samples and testing set consists of those not used (out of bag samples). Both features are not intrinsic to Cox regression or other single classifiers. Not considering these two features could potentially lead to a partial comparison between the performance of the two methods. By using a colorectal survival dataset, we illustrate the problems of using k-fold cross-validation, using only one resampling without an ensemble average, and using the whole dataset for both fitting and testing, in Cox regression, when comparing with RSF. We provide a more accessible R code for simple calculation of discordance index (D-index) and unweighted integrated Brier score (IBS) for Cox regression, and unweighted IBS for RSF. Doi: 10.28991/SciMedJ-2021-0301-9 Full Text: PDF