Файл:Auto-association by multilayer perceptrons and singular value decomposition 1548720901c89b3b7481f7500d7cd64e99bd.pdf

H. Bourlard and Y. Kamp Philips Research Laboratory, Avenue Van Becelaere 2, Box 8, B-1170 Brussels, Belgium

Abstract

The multilayer perceptron, when working in auto-association mode, is sometimes considered as an interesting candidate to perform data compression or dimensionality reduction of the feature space in in- formation processing applications. The present paper shows that, for auto-association, the nonlinearities of the hidden units are useless and that the optimal parameter values can be derived directly by purely linear techniques relying on singular value decompo- sition and low rank matrix approximation, similar in spirit to the well-known Karhunen-Lo6ve transform. This approach appears thus as an efficient alternative to the general error back-propagation algorithm com- monly used for training multilayer perceptrons. Moreover, it also gives a clear interpretation of the r61e of the different parameters.