Autonomous Scanning and Cleanliness Classification of Pharmaceutical Bins Through Artificial Intelligence and Robotics

• Published in: IEEE access Vol. 12; pp. 117256 - 117270
• Main Authors: Comari, Simone, Carricato, Marco
• Format: Journal Article
• Language: English
• Published: IEEE 2024
• Subjects: Artificial intelligence; Cameras; Cleaning; Computer vision; object scanning; Pharmaceuticals; quality inspection; robotics; Robots; Solid modeling; Surface cleaning; surface cleanliness; Three-dimensional displays; Trajectory; trajectory planning
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3447158

In the pharmaceutical industry, bins need to be cleaned up to a critical level because the products that they contain are often incompatible with each other, and their mixture can facilitate the formation of bacterial fauna. In this work, a strategy is presented to fully automatize the procedure of cleanliness quality inspection of a pharmaceutical bin through a robotic arm and the use of both traditional and artificial-intelligence-based computer-vision techniques. An autonomous mobile robot is used to mimic the approach of a manipulator to a bin inserted in a washing cabin with an uncertain position. The manipulator is equipped with an eye-on-hand color camera and it carries out the binary classification of the bin surface status (e.g. clean vs dirty) through a convolutional neural network based on ResNet. The viewpoints from which the images are taken are the result of an optimization that, starting from the digital three-dimensional model of the bin and exploiting a virtual-twin-based planning scene, minimizes their number while maximizing the visible area of the bin from the current location of the robot. The results of this optimization are used to set up a pipeline that is entirely bin-independent. The same procedure may also be employed to generate the best washing trajectories to be performed by the cleaning robot, by simply replacing the inspection camera mounted on the robot end-effector with a washing nozzle. Though a complete tuning session is still required, preliminary experimental results are very promising, reaching a classifier accuracy (namely a capability of distinguishing clean and dirty surfaces) of 98% on conditioned data, showing that this work has the potential of becoming an effective and versatile industrial product.