Statistical mechanics of lossy compression for nonmonotonic multilayer perceptrons

Abstract

References (26)

Citing Articles

Download: PDF (326 kB) or Buy this Article (US$25) (Use Article Pack)

Florent Cousseau
Graduate School of Frontier Sciences, University of Tokyo, Chiba 277-5861, Japan

Kazushi Mimura
Faculty of Information Sciences, Hiroshima City University, Hiroshima 731-3194, Japan

Toshiaki Omori
Brain Science Institute, RIKEN, Saitama 351-0198, Japan

Masato Okada
Graduate School of Frontier Sciences, University of Tokyo, Chiba 277-5861, Japan and Brain Science Institute, RIKEN, Saitama 351-0198, Japan

Received 20 March 2008; published 19 August 2008

Alossy data compression scheme for uniformly biased Boolean messages isinvestigated via statistical mechanics techniques. We utilize a treelike committeemachine (committee tree) and a treelike parity machine (parity tree)whose transfer functions are nonmonotonic. The scheme performance at theinfinite code length limit is analyzed using the replica method.Both committee and parity treelike networks are shown to saturatethe Shannon bound. The Almeida-Thouless stability of the replica symmetricsolution is analyzed, and the tuning of the nonmonotonic transferfunction is also discussed.

URL:	http://link.aps.org/doi/10.1103/PhysRevE.78.021124
DOI:	10.1103/PhysRevE.78.021124
PACS:	02.50.-r; 05.50.+q; 89.70.-a 02.50.-r Probability theory, stochastic processes, and statistics 05.50.+q Lattice theory and statistics 89.70.-a Information and communication theory YEAR: 2008
KEYWORDS:	data compression, information theory, parity, replica techniques, statistical mechanics