An algorithm of incremental approximation of functions in a normed linearspace by feedforward neural networks is presented. The concept of variation of a function with respect to a set is used to estimate the approximation error together with the weight decay method, for optimizing the size and weights of a network in each iteration step of the algorithm. Two alternatives, recursively incremental and generally incremental, are proposed. In the generally incremental case, the algorithm optimizes parameters of all units in the hidden layer at each step. In the recursively incremental case, the algorithm optimizes the parameterscorresponding to only one unit in the hidden layer at each step. In thiscase, an optimization problem with a smaller number of parameters is being solved at each step.

Top