Information Theory and Statistical Physics

R. Meir and N. Merhav,
``On the stochastic complexity
of learning realizable and unrealizable rules,''
Machine Learning, vol. 19, no. 3, pp. 241261, 1995.

N. Merhav,
``An identity of Chernoff bounds with an interpretation in statistical
physics and applications in information theory,''
IEEE Trans. Inform. Theory, vol. 54, no. 8, pp. 37103721,
August 2008.

N. Merhav,
``Relations between random coding exponents and the statistical
physics of random codes,''
IEEE Trans. Inform. Theory, , vol. 55, no. 1, pp. 8392, January 2009.

N. Merhav,
``The generalized random energy model of spin glasses and its
application to the statistical physics of code ensembles with
hierarchical structures,''
IEEE Trans. Inform. Theory , vol. 55, no. 3, pp. 12501268, March 2009.

N. Merhav,
``The random energy model in
a magnetic field and joint sourcechannel coding,''
Physica A: Statistical
Mechanics and its Applications , vol. 387, issue 22,
pp. 56625674, September 15, 2008.

N. Merhav,
``Joint sourcechannel coding via statistical mechanics: thermal
equilibrium between the source and the channel,''
IEEE Trans. Inform. Theory, vol. 55, no. 12, pp. 53825393, December 2009.

N. Merhav,
``On the statistical physics of directed polymers in a random medium
and their relation to tree codes,''
IEEE Trans. Inform. Theory, vol. 56, no. 3, pp. 13451350, March 2010.

N. Merhav, D. Guo, and S. Shamai (Shitz),
``Statistical physics of signal estimation in Gaussian noise:
theory and examples of phase transitions,''
IEEE Trans. Inform. Theory, vol. 56, no. 3, pp. 14001416, March 2010.

N. Merhav,
``Physics of the Shannon limits,''
IEEE Trans. Inform. Theory, vol. 56, no. 9, pp. 42744285,
September 2010.
Short version 
Proc. 2009 IEEE Workshop on Information
Theory (ITW 2009),
Taormina, Sicily, Italy, October 1116, 2009.

N. Merhav,
``Statistical physics and information theory,''
(invited paper) Foundations and Trends in
Communications and Information Theory, vol. 6, nos. 12, pp. 1212, 2009.

N. Merhav,
``Another look at the physics of large deviations with application to
ratedistortion theory,'' Technical Report, CCIT Pub. no. 742,
EE Pub. no. 1699, August 2009. Also, available in
arXiv and
here is
the
conference version, which appears in Proc. ISIT 2010, Austin, Texas,
U.S.A., June 2010.

N. Merhav,
``Optimum estimation via gradients of partition functions and
information measures: a statisticalmechanical perspective,''
IEEE Trans. on Inform. Theory, vol. 57, no. 6, pp. 38873898, June
2011.

N. Merhav and Y. Kafri,
``BoseEinstein condensation in the large deviations regime with
applications to information system models,''
Journal of Statistical Mechanics: Theory and
Experiment, P02011, February 2010.

N. Merhav,
``Threshold effects in parameter estimation as phase transitions in
statistical mechanics,''
IEEE Trans. on Inform. Theory,
vol. 57, no. 10, pp. 70007010, October 2011.

N. Merhav,
``Data processing theorems and the second law of thermodynamics,''
IEEE Trans. on Inform. Theory, vol. 57, no. 8, pp. 49264939, August
2011.

N. Merhav and Y. Kafri,
``Statistical properties of entropy production derived from fluctuation
theorems,''
Journal of Statistical Mechanics: Theory and
Experiment, P12022, December 2010. doi: 10.1088/17425468/2010/12/P12022
 N. Merhav,
``A statisticalmechanical view on source coding: physical compression and
data compression,''
Journal of Statistical Mechanics: Theory and
Experiment, P01029, January 2011,
doi: 10.1088/17425468/2011/01/P01029.
[With a certain overlap to no. 11, but with a
different emphasis and some other results.]

N. Merhav,
``On optimum strategies for minimizing exponential moments of a loss
function,'' Communications in Information
and Systems,
vol. 11, no. 4, pp. 343368, 2011.

N. Merhav,
``Relations between redundancy patterns of the Shannon code and
wave diffraction patterns of partially disordered media,''
IEEE Trans. Inform. Theory, vol. 58, no. 6,
pp. 34023406, January 2012.

N. Merhav,
``Subsetsum phase transitions and data compression,''
Journal of Statistical Mechanics: Theory and Experiment,
P09017, September 2011,
doi: 10.1088/17425468/2011/09/P01029

N. Merhav,
``Another look at expurgated bounds and their statisticalmechanical
interpretation,'' unpublished.

N. Merhav,
``Erasure/list exponents for SlepianWolf decoding,''
IEEE Trans. Inform. Theory, vol. 60, no. 8, pp. 44634471, August
2014.

W. Huleihel and N. Merhav,
``Analysis of mismatched estimation errors using gradients of partition
functions,''
IEEE Trans. Inform. Theory,, vol. 60, no. 4, pp. 21902216,
April 2014.

J. Scarlett, L. Peng, N. Merhav, A. Martinez, and A. G. i Fabregas,
``Expurgated randomcoding ensembles: exponents, refinements and
connections,'' IEEE Trans. Inform. Theory,
vol. 60, no. 8, pp. 44494462, August 2014.

W. Huleihel and N. Merhav,
``Asymptotic MMSE analysis under sparse representation modeling,''
IEEE Trans. Inform. Theory,, Signal Processing, vol. 131,
pp. 320332, February 2017.

D. Vinkler, H. Permuter, and N. Merhav,
``Analogy between gambling and measurementbased work extraction,''
Proc. ISIT 2014, pp. 11111115, Honolulu, Hawaii,
JuneJuly 2014. Full version is
here.

N. Merhav,
``Statistical physics of random binning,''
IEEE Trans. Inform. Theory,
vol. 61, no. 5, pp. 24542464, May 2015.

N. Merhav,
``Sequence complexity and work extraction,''
Journal of Statistical Mechanics: Theory and Experiment,
P06037, June 2015. doi:10.1088/17425468/2015/06/P06037

N. Merhav,
``Relations between work and entropy production for general
informationdriven, finitestate engines,''
Journal of Statistical Mechanics: Theory and Experiment,
023207, February 2017. https://doi.org/10.1088/17425468/aa58f3

N. Merhav,
``Lower bounds on exponential moments of the quadratic error in parameter
estimation,''
IEEE Trans. Inform. Theory, vol. 64, no. 12, pp. 76367648,
December 2018.

L. Touzo, M. Marsili, N. Merhav, and E. Roldan,
``Optimal work extraction and the minimum description length principle,''
Journal of Statistical Mechanics: Theory and Experiment (JSTAT)},
094403, September 2020.
http://dx.doi.org/10.1088/17425468/abacb3