Clinical application of machine learning and computer vision to indocyanine green quantification for dynamic intraoperative tissue characterisation: how to do it

• Published in: Surgical endoscopy Vol. 37; no. 8; pp. 6361 - 6370
• Main Authors: Hardy, Niall P., MacAonghusa, Pol, Dalli, Jeffrey, Gallagher, Gareth, Epperlein, Jonathan P., Shields, Conor, Mulsow, Jurgen, Rogers, Ailín C., Brannigan, Ann E., Conneely, John B., Neary, Peter M., Cahill, Ronan A.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-08-2023; Springer Nature B.V
• Subjects: 2022 SAGES Oral; Abdominal Surgery; Artificial intelligence; Cameras; Colorectal cancer; Colorectal Neoplasms - pathology; Colorectal Neoplasms - surgery; Colorectal surgery; Computer vision; Computers; Endoscopy; Gastroenterology; Gastrointestinal surgery; General anesthesia; Gynecology; Hepatology; Hospitals; Humans; Indocyanine Green; Laparoscopy; Liver; Machine learning; Medicine; Medicine & Public Health; Metastasis; Pathology; Patients; Proctology; Prospective Studies; Reproducibility of Results; Surgeons; Surgery; Tumors; Indocyanine green quantification; Artificial intelligence; Fluorescence-guided surgery; Colorectal cancer; Machine learning
• ISSN: 0930-2794; 1432-2218
• DOI: 10.1007/s00464-023-09963-2

Introduction
Indocyanine green (ICG) quantification and assessment by machine learning (ML) could discriminate tissue types through perfusion characterisation, including delineation of malignancy. Here, we detail the important challenges overcome before effective clinical validation of such capability in a prospective patient series of quantitative fluorescence angiograms regarding primary and secondary colorectal neoplasia.
Methods
ICG perfusion videos from 50 patients (37 with benign (13) and malignant (24) rectal tumours and 13 with colorectal liver metastases) of between 2- and 15-min duration following intravenously administered ICG were formally studied (clinicaltrials.gov: NCT04220242). Video quality with respect to interpretative ML reliability was studied observing practical, technical and technological aspects of fluorescence signal acquisition. Investigated parameters included ICG dosing and administration, distance–intensity fluorescent signal variation, tissue and camera movement (including real-time camera tracking) as well as sampling issues with user-selected digital tissue biopsy. Attenuating strategies for the identified problems were developed, applied and evaluated. ML methods to classify extracted data, including datasets with interrupted time-series lengths with inference simulated data were also evaluated.
Results
Definable, remediable challenges arose across both rectal and liver cohorts. Varying ICG dose by tissue type was identified as an important feature of real-time fluorescence quantification. Multi-region sampling within a lesion mitigated representation issues whilst distance–intensity relationships, as well as movement-instability issues, were demonstrated and ameliorated with post-processing techniques including normalisation and smoothing of extracted time–fluorescence curves. ML methods (automated feature extraction and classification) enabled ML algorithms glean excellent pathological categorisation results (AUC-ROC > 0.9, 37 rectal lesions) with imputation proving a robust method of compensation for interrupted time-series data with duration discrepancies.
Conclusion
Purposeful clinical and data-processing protocols enable powerful pathological characterisation with existing clinical systems. Video analysis as shown can inform iterative and definitive clinical validation studies on how to close the translation gap between research applications and real-world, real-time clinical utility.