Networks for approximation and learning

• Published in: Proceedings of the IEEE Vol. 78; no. 9; pp. 1481 - 1497
• Main Authors: Poggio, T., Girosi, F.
• Format: Journal Article
• Language: English
• Published: IEEE 01-09-1990
• Subjects: Approximation methods; Artificial intelligence; Associative memory; Backpropagation algorithms; Contracts; Network synthesis; Network topology; Neural networks; Prototypes; System identification
• ISSN: 0018-9219
• DOI: 10.1109/5.58326

The problem of the approximation of nonlinear mapping, (especially continuous mappings) is considered. Regularization theory and a theoretical framework for approximation (based on regularization techniques) that leads to a class of three-layer networks called regularization networks are discussed. Regularization networks are mathematically related to the radial basis functions, mainly used for strict interpolation tasks. Learning as approximation and learning as hypersurface reconstruction are discussed. Two extensions of the regularization approach are presented, along with the approach's corrections to splines, regularization, Bayes formulation, and clustering. The theory of regularization networks is generalized to a formulation that includes task-dependent clustering and dimensionality reduction. Applications of regularization networks are discussed.< >