A Toolbox for Refined Information-Theoretic Analyses
By Neri Merhav, Technion – Israel Institute of Technology, Israel, merhav@ee.technion.ac.il | Nir Weinberger, Technion – Israel Institute of Technology, Israel, nirwein@technion.ac.il
Abstract
This monograph offers a toolbox of mathematical techniques that have been effective and widely applicable in informationtheoretic analyses. The first tool is a generalization of the method of types to Gaussian settings, and then to general exponential families. The second tool is Laplace and saddlepoint integration, which allow to refine the results of the method of types, and can obtain various precise asymptotic results. The third is the type class enumeration method, a principled method to evaluate the exact random-coding exponent of coded systems, which results in the best known exponent in various problems. The fourth is a subset of tools aimed at evaluating the expectation of non-linear functions of random variables, either via integral representations, by a refinement of Jensen’s inequality via change-of-measure, by complementing Jensen’s inequality with a reversed inequality, or by a class of generalized Jensen’s inequalities that are applicable for functions beyond convex/concave. Various examples of all these tools are provided throughout the monograph.
A Toolbox for Refined Information-Theoretic Analyses
This monograph offers a toolbox of mathematical techniques that have been effective and widely applicable in information-theoretic analyses. The first tool is a generalization of the method of types to Gaussian settings, and then to general exponential families. The second tool is Laplace and saddle-point integration, which allow to refine the results of the method of types, and is capable of obtaining various precise asymptotic results.
The third is the type class enumeration method, a principled method to evaluate the exact random-coding exponent of coded systems, which results in the best known exponent in various problem settings. The fourth is a subset of tools aimed at evaluating the expectation of non-linear functions of random variables, either via integral representations, by a refinement of Jensen’s inequality via change-of-measure, by complementing Jensen’s inequality with a reversed inequality, or by a class of generalized Jensen’s inequalities that are applicable for functions beyond convex/concave. Various examples of all these tools are provided throughout the monograph.
