A variational method for learning sparse and overcomplete representations

Authors
Citation
M. Girolami, A variational method for learning sparse and overcomplete representations, NEURAL COMP, 13(11), 2001, pp. 2517-2532
Citations number
19
Categorie Soggetti
Neurosciences & Behavoir","AI Robotics and Automatic Control
Journal title
NEURAL COMPUTATION
ISSN journal
08997667 → ACNP
Volume
13
Issue
11
Year of publication
2001
Pages
2517 - 2532
Database
ISI
SICI code
0899-7667(200111)13:11<2517:AVMFLS>2.0.ZU;2-6
Abstract
An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variat ional approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectatio n-maximization algorithm for learning the overcomplete basis vectors and in ferring the most probable basis coefficients.