TED: A Tolerant Edit Distance for segmentation evaluation

• Published in: Methods (San Diego, Calif.) Vol. 115; pp. 119 - 127
• Main Authors: Funke, Jan, Klein, Jonas, Moreno-Noguer, Francesc, Cardona, Albert, Cook, Matthew
• Format: Journal Article Publication
• Language: English
• Published: United States Elsevier Inc 15-02-2017
• Subjects: Analysis of Variance; Animals; Automàtica i control; Cerebral Cortex - anatomy & histology; Cerebral Cortex - ultrastructure; Classificació INSPEC; Computer vision; Drosophila melanogaster - cytology; Drosophila melanogaster - ultrastructure; Electron microscopy; Evaluation; Humans; Image Processing, Computer-Assisted - methods; Image Processing, Computer-Assisted - statistics & numerical data; Informàtica; Learning; Machine Learning; Mice; Microscopy, Electron - statistics & numerical data; Neuron segmentation; Neurons - cytology; Neurons - ultrastructure; Pattern recognition; Pattern Recognition, Automated - statistics & numerical data; Segmentation; Àrees temàtiques de la UPC; Learning; Computer vision; Evaluation; Segmentation; Electron microscopy; Neuron segmentation
• ISSN: 1046-2023; 1095-9130
• DOI: 10.1016/j.ymeth.2016.12.013

•A Tolerant Edit Distance (TED) for segmentation comparison is proposed.•The TED reflects the effort or time needed to fix a segmentation compared to ground truth.•Small, tolerable, errors are ignored.•It is shown that minimizing the TED during training leads to improved accuracy of segmentation algorithms.
In this paper, we present a novel error measure to compare a computer-generated segmentation of images or volumes against ground truth. This measure, which we call Tolerant Edit Distance (TED), is motivated by two observations that we usually encounter in biomedical image processing: (1) Some errors, like small boundary shifts, are tolerable in practice. Which errors are tolerable is application dependent and should be explicitly expressible in the measure. (2) Non-tolerable errors have to be corrected manually. The effort needed to do so should be reflected by the error measure. Our measure is the minimal weighted sum of split and merge operations to apply to one segmentation such that it resembles another segmentation within specified tolerance bounds. This is in contrast to other commonly used measures like Rand index or variation of information, which integrate small, but tolerable, differences. Additionally, the TED provides intuitive numbers and allows the localization and classification of errors in images or volumes. We demonstrate the applicability of the TED on 3D segmentations of neurons in electron microscopy images where topological correctness is arguable more important than exact boundary locations. Furthermore, we show that the TED is not just limited to evaluation tasks. We use it as the loss function in a max-margin learning framework to find parameters of an automatic neuron segmentation algorithm. We show that training to minimize the TED, i.e., to minimize crucial errors, leads to higher segmentation accuracy compared to other learning methods.