Does combining numerous data types in multi-omics data improve or hinder performance in survival prediction? Insights from a large-scale benchmark study

• Published in: BMC medical informatics and decision making Vol. 24; no. 1; pp. 244 - 14
• Main Authors: Li, Yingxia, Herold, Tobias, Mansmann, Ulrich, Hornung, Roman
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 02-09-2024; BioMed Central; BMC
• Subjects: Benchmark; Benchmarking; Biological analysis; Cancer; Copy number; Cost control; Data acquisition; Data entry; Datasets; Deep learning; Evaluation; Feature selection; Genomics; Humans; Information management; Methylation; miRNA; Modelling; mRNA; Multi-omics data; Multiomics; Neoplasms - genetics; Neoplasms - mortality; Oncology, Experimental; Performance enhancement; Performance evaluation; Performance prediction; Prediction; Prediction models; Predictions; Prognosis; Public health administration; Statistical analysis; Survival; Survival Analysis; TCGA; Variables; Germany; Benchmark; Survival analysis; Multi-omics data; TCGA; Prediction; Cancer
• ISSN: 1472-6947; 1472-6947
• DOI: 10.1186/s12911-024-02642-9

Predictive modeling based on multi-omics data, which incorporates several types of omics data for the same patients, has shown potential to outperform single-omics predictive modeling. Most research in this domain focuses on incorporating numerous data types, despite the complexity and cost of acquiring them. The prevailing assumption is that increasing the number of data types necessarily improves predictive performance. However, the integration of less informative or redundant data types could potentially hinder this performance. Therefore, identifying the most effective combinations of omics data types that enhance predictive performance is critical for cost-effective and accurate predictions.
In this study, we systematically evaluated the predictive performance of all 31 possible combinations including at least one of five genomic data types (mRNA, miRNA, methylation, DNAseq, and copy number variation) using 14 cancer datasets with right-censored survival outcomes, publicly available from the TCGA database. We employed various prediction methods and up-weighted clinical data in every model to leverage their predictive importance. Harrell's C-index and the integrated Brier Score were used as performance measures. To assess the robustness of our findings, we performed a bootstrap analysis at the level of the included datasets. Statistical testing was conducted for key results, limiting the number of tests to ensure a low risk of false positives.
Contrary to expectations, we found that using only mRNA data or a combination of mRNA and miRNA data was sufficient for most cancer types. For some cancer types, the additional inclusion of methylation data led to improved prediction results. Far from enhancing performance, the introduction of more data types most often resulted in a decline in performance, which varied between the two performance measures.
Our findings challenge the prevailing notion that combining multiple omics data types in multi-omics survival prediction improves predictive performance. Thus, the widespread approach in multi-omics prediction of incorporating as many data types as possible should be reconsidered to avoid suboptimal prediction results and unnecessary expenditure.