Signal Processing Platform for Long-Range Multi-Spectral Electro-Optical Systems

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 3; p. 1294
• Main Authors: Latinović, Nikola, Popadić, Ilija, Tomić, Branko, Simić, Aleksandar, Milanović, Petar, Nijemčević, Srećko, Perić, Miroslav, Veinović, Mladen
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 08-02-2022; MDPI
• Subjects: Algorithms; Artificial intelligence; Cameras; Communication; Computer vision; Design; Digital integrated circuits; Equipment and supplies; Ethernet; Field programmable gate arrays; Global Positioning System; Global positioning systems; GPS; Graphics processing units; Image enhancement; Image processing; Infrared cameras; Laser range finders; Lasers; multi sensor electro-optical systems; Multisensor fusion; Semiconductor industry; Sensors; Short wave radiation; Signal processing; smart sensor; Software; video processing platform; United States; multi sensor electro-optical systems; smart sensor; video processing platform
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22031294

In this paper, we present a hardware and software platform for signal processing (SPP) in long-range, multi-spectral, electro-optical systems (MSEOS). Such systems integrate various cameras such as lowlight color, medium or long-wave-infrared thermal and short-wave-infrared cameras together with other sensors such as laser range finders, radars, GPS receivers, etc. on rotational pan-tilt positioner platforms. An SPP is designed with the main goal to control all components of an MSEOS and execute complex signal processing algorithms such as video stabilization, artificial intelligence-based target detection, target tracking, video enhancement, target illumination, multi-sensory image fusion, etc. Such algorithms might be very computationally demanding, so an SPP enables them to run by splitting processing tasks between a field-programmable gate array (FPGA) unit, a multicore microprocessor (MCuP) and a graphic processing unit (GPU). Additionally, multiple SPPs can be linked together via an internal Gbps Ethernet-based network to balance the processing load. A detailed description of the SPP system and experimental results of workloads for typical algorithms on demonstrational MSEOS are given. Finally, we give remarks regarding upgrading SPPs as novel FPGAs, MCuPs and GPUs become available.