Some comparisons between linear approximation and approximation by neural networks

Some comparisons between linear approximation and approximation by neural networks

Abstract

We present some comparisons between the approximation rates relevant to linear approximators and the rates relevant to neural networks, i.e. nonlinear approximators represented by sets of parametrized functions corresponding to a type of computational unit. Our analysis uses the concept of variation of a function with respect to a set. The comparison is made in terms of Kolmogorov n-width for linear spaces and a proper nonlinear n-width for the nonlinear context represented by neural networks. The results of this paper contribute to the theoretical understanding of the superiority of neural networks with respect to linear approximators in complex tasks, as is confirmed by a wide variety of applications (recognition of handwritten characters and spoken numerals, approximate solution of functional optimization problems from control theory, etc.).

Grafik Top
Authors
  • Sanguineti, Marcello
  • Hlavackova-Schindler, Katerina
Grafik Top
Editors
  • Dobnikar, Andrej
  • Steele, Nigel
  • Pearson, David
  • Albrecht, Rudolf
Grafik Top
Shortfacts
Category
Paper in Conference Proceedings or in Workshop Proceedings (Paper)
Event Title
International Conference onArtificial Neural Nets and Genetic Algorithms, Slovenia, 1999
Divisions
Data Mining and Machine Learning
Subjects
Kuenstliche Intelligenz
Event Location
Slovenia
Event Type
Conference
Event Dates
1999
Series Name
Artificial Neural Nets and Genetic Algorithms
ISSN/ISBN
978-3-7091-6384-9
Page Range
pp. 172-178
Date
1999
Export
Grafik Top