OUTRIDER: A Statistical Method for Detecting Aberrantly Expressed Genes in RNA Sequencing Data

• Published in: American journal of human genetics Vol. 103; no. 6; pp. 907 - 917
• Main Authors: Brechtmann, Felix, Mertes, Christian, Matusevičiūtė, Agnė, Yépez, Vicente A., Avsec, Žiga, Herzog, Maximilian, Bader, Daniel M., Prokisch, Holger, Gagneur, Julien
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 06-12-2018; Elsevier
• Subjects: aberrant gene expression; Algorithms; Gene Expression - genetics; Gene Expression Profiling - methods; Genetic Variation - genetics; High-Throughput Nucleotide Sequencing - methods; Humans; normalization; outlier detection; rare disease; RNA - metabolism; RNA sequencing; Sequence Analysis, RNA - methods; RNA sequencing; aberrant gene expression; rare disease; normalization; outlier detection
• ISSN: 0002-9297; 1537-6605
• DOI: 10.1016/j.ajhg.2018.10.025

RNA sequencing (RNA-seq) is gaining popularity as a complementary assay to genome sequencing for precisely identifying the molecular causes of rare disorders. A powerful approach is to identify aberrant gene expression levels as potential pathogenic events. However, existing methods for detecting aberrant read counts in RNA-seq data either lack assessments of statistical significance, so that establishing cutoffs is arbitrary, or rely on subjective manual corrections for confounders. Here, we describe OUTRIDER (Outlier in RNA-Seq Finder), an algorithm developed to address these issues. The algorithm uses an autoencoder to model read-count expectations according to the gene covariation resulting from technical, environmental, or common genetic variations. Given these expectations, the RNA-seq read counts are assumed to follow a negative binomial distribution with a gene-specific dispersion. Outliers are then identified as read counts that significantly deviate from this distribution. The model is automatically fitted to achieve the best recall of artificially corrupted data. Precision-recall analyses using simulated outlier read counts demonstrated the importance of controlling for covariation and significance-based thresholds. OUTRIDER is open source and includes functions for filtering out genes not expressed in a dataset, for identifying outlier samples with too many aberrantly expressed genes, and for detecting aberrant gene expression on the basis of false-discovery-rate-adjusted p values. Overall, OUTRIDER provides an end-to-end solution for identifying aberrantly expressed genes and is suitable for use by rare-disease diagnostic platforms.