Information theoretic limits for sublinear-rank symmetric matrix factorization
A talk in the Seminar Zufallsmatrizen series by
Anas Rahman
| Abstract: | Let X be an $N \times M$ signal matrix with i.i.d. real entries drawn from some
distribution. We consider a statistical model for the measurement Y of XX*
through an additive Gaussian channel in the high-dimensional regime where
M scales with N as $M=o(N^{1/4})$. Working in the Bayes-optimal setting, we
show that the limiting free entropy of the model, equivalently the mutual
information between the measurement Y and signal X, is given by a
variational formula involving a replica symmetric potential corresponding to
an M-dimensional vector channel. In fact, we show that in many cases, we
can reduce further to the replica symmetric potential of a scalar channel ($M
= 1$). Our arguments draw on an application of the cavity method allowing
for growing rank M, a surprisingly simple result on overlap concentration,
and some information-theoretic identities concerning concavity properties
of the distribution on the entries of X. Please contact Lucas Hackl (Lucas.Hackl@unimelb.edu.au) for details regarding access Within the CRC this talk is associated to the project(s): C6 |