PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions

• Published in: Cell genomics Vol. 2; no. 5; p. 100129
• Main Authors: Olson, Nathan D., Wagner, Justin, McDaniel, Jennifer, Stephens, Sarah H., Westreich, Samuel T., Prasanna, Anish G., Johanson, Elaine, Boja, Emily, Maier, Ezekiel J., Serang, Omar, Jáspez, David, Lorenzo-Salazar, José M., Muñoz-Barrera, Adrián, Rubio-Rodríguez, Luis A., Flores, Carlos, Kyriakidis, Konstantinos, Malousi, Andigoni, Shafin, Kishwar, Pesout, Trevor, Jain, Miten, Paten, Benedict, Chang, Pi-Chuan, Kolesnikov, Alexey, Nattestad, Maria, Baid, Gunjan, Goel, Sidharth, Yang, Howard, Carroll, Andrew, Eveleigh, Robert, Bourgey, Mathieu, Bourque, Guillaume, Li, Gen, Ma, ChouXian, Tang, LinQi, Du, YuanPing, Zhang, ShaoWei, Morata, Jordi, Tonda, Raúl, Parra, Genís, Trotta, Jean-Rémi, Brueffer, Christian, Demirkaya-Budak, Sinem, Kabakci-Zorlu, Duygu, Turgut, Deniz, Kalay, Özem, Budak, Gungor, Narcı, Kübra, Arslan, Elif, Brown, Richard, Johnson, Ivan J., Dolgoborodov, Alexey, Semenyuk, Vladimir, Jain, Amit, Tetikol, H. Serhat, Jain, Varun, Ruehle, Mike, Lajoie, Bryan, Roddey, Cooper, Catreux, Severine, Mehio, Rami, Ahsan, Mian Umair, Liu, Qian, Wang, Kai, Ebrahim Sahraeian, Sayed Mohammad, Fang, Li Tai, Mohiyuddin, Marghoob, Hung, Calvin, Jain, Chirag, Feng, Hanying, Li, Zhipan, Chen, Luoqi, Sedlazeck, Fritz J., Zook, Justin M.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 11-05-2022
• Subjects: Basic Medicine; benchmark; benchmarking; bioinformatics; Bioinformatics (Computational Biology); Bioinformatics and Systems Biology; Bioinformatik (beräkningsbiologi); Bioinformatik och systembiologi; Biologi; Biological Sciences; Computer and Information Science; Data- och informationsvetenskap (Datateknik); DNA; long-read sequencing; Medical and Health Sciences; Medical Genetics; Medicin och hälsovetenskap; Medicinsk genetik; Medicinska och farmaceutiska grundvetenskaper; Natural Sciences; Naturvetenskap; precisionFDA; shot-read sequencing; variant; variant calling; whole-genome sequencing; precisionFDA; variant calling; benchmarking; whole-genome sequencing; bioinformatics
• ISSN: 2666-979X; 2666-979X
• DOI: 10.1016/j.xgen.2022.100129

The precisionFDA Truth Challenge V2 aimed to assess the state of the art of variant calling in challenging genomic regions. Starting with FASTQs, 20 challenge participants applied their variant-calling pipelines and submitted 64 variant call sets for one or more sequencing technologies (Illumina, PacBio HiFi, and Oxford Nanopore Technologies). Submissions were evaluated following best practices for benchmarking small variants with updated Genome in a Bottle benchmark sets and genome stratifications. Challenge submissions included numerous innovative methods, with graph-based and machine learning methods scoring best for short-read and long-read datasets, respectively. With machine learning approaches, combining multiple sequencing technologies performed particularly well. Recent developments in sequencing and variant calling have enabled benchmarking variants in challenging genomic regions, paving the way for the identification of previously unknown clinically relevant variants.
[Display omitted]
•Sixty-four submissions employing innovative variant-calling methods for three technologies•Submissions evaluated with new GIAB benchmark sets and new genome stratifications•Submissions differ in performance overall and in challenging genomic regions•Challenge data are available at https://doi.org/10.18434/mds2-2336
Olson et al. report on the results of precisionFDA Truth Challenge V2 for variant-calling pipelines. The challenge focused on small-variant accuracy of innovative deep learning and graph-based methods, utilizing a new benchmark and with new stratifications to demonstrate strengths and weaknesses of different methods.