IAM-PIMS Distinguished Colloquium: Tsachy Weissman

The first part will consist of a tour through a sparse sample of the information theory literature - both classical and recent - on relations between information and estimation. Beyond aesthetic value, these relations underlie some of the main tools in Shannon theory. They also give considerable insight into and a quantitative understanding of several estimation theoretic objects, such as the costs of causality and of mismatch, as well as the performance and structure of minimax estimators. Further, they enable the transfer of analytic tools and algorithmic know-how from one domain to another. Examples will be given to illustrate these points. The second will begin by introducing Directed Information and making the case for caring about how to estimate it. I will then describe some approaches to this estimation problem which are based on universal sequential probability assignments. The estimators inherit much of the algorithmic simplicity and convergence properties of the latter. I will conclude by showcasing some experimental results illustrating the efficacy of directed information estimation as a tool for detecting and measuring causality. Much of the talk is based on collaborations with Rami Atar, Jiantao Jiao, Young-Han Kim, Albert No, Haim Permuter, Kartik Venkat, and Lei Zhao.