Computational Geometry for Machine Learning and Computer Vision: Practical Provable Approximations

• Main Authors: Jubran, Ibrahim, ןארבו׳ג, םיהרבא
• Format: Dissertation
• Language: English
• Published: ProQuest Dissertations & Theses 01-01-2022
• Subjects: Algorithms; Artificial intelligence; Autonomous vehicles; Cameras; Computer science; Construction; Decision trees; Drones; Drownings; Geometry; Information Technology; Internet of Things; Linear algebra; Localization; Machine learning; Mathematics; Medical equipment; Medicine; Optimization; Registration; Surveillance; Transportation; Web Studies
• ISBN: 9798374486292

Machine learning and computer vision are arguably two of the most popular and active fields of research in modern times. However, algorithms developed in those fields are either computationally intensive, memory demanding, numerically unstable, or are simply heuristics which do not provide any optimality guarantees. Algorithms which suffer from such issues can be difficult, or even impossible, to deploy onto, for example, IoT devices which usually have very limited resources, real-time systems, or systems which crucially require high levels of accuracy as in autonomous vehicles or medical devices.To this end, in this thesis we: (i) develop theoretically-grounded compression schemes which reduce the computational burden of machine learning algorithms, and (ii) provide provable and efficient approximation algorithm for cornerstone problems in computer vision. Most of those results are achieved via novel connections which we forge between the task at hand, and theorems and techniques from computational geometry.We start by investigating the idea of accurate coresets, where a coreset is usually a small weighted subset of an input set of items, that provably approximates, up to some small constant, their loss function for a given set of queries (models, classifiers, hypothesis). We provide an in depth overview of lossless coresets which do not introduce any approximation errors. We provide full proofs, intuitions, illustrations, and code for the tricks and techniques involved in such coreset constructions.We then develop a novel accurate coreset for the family of least-mean-squares solvers, such as ridge regression. This coreset was empirically shown to reduce the computational time and space of such algorithms by up to x100, while also enhancing their numerical stability. At the heart of this work lies an old theorem from computational geometry that dates back to 1907.We then proceed to construct a lossy coreset for the more complex task of computing a decision tree for a given low-dimensional input dataset. The construction of this coreset required 3 main components: (i) a novel connection to partition trees common in computational geometry, (ii) an efficient, rough, but provable approximation algorithm, and (iii) our previous accurate coreset. Our suggested coreset can help reduce the construction time and hyperparameter tuning of decision trees and forests by up to x10.An approximation algorithm is crucial during the construction of f provable coresets. As no such provable and efficient algorithms were suggested for cornerstone computer vision problems, we started to investigate along this direction. Throughout this research, we provided the first provable constant factor approximation algorithm for the difficult task of point cloud registration under various loss functions, and then provided a provable (1+ )-approximation algorithm for the famous Perspective- -point problem (PnP). Extensive experimental results indicate that our methods outperform existing provable algorithms as well as heuristics.To contribute to the community, we provide open source code for all the algorithms given in this thesis. We hope our algorithms and code, as well as the tricks, techniques, and links to the field of computational geometry will open up new avenues for future research on compression schemes and approximations for modern tasks.