2014 Poster Sessions : Universal Estimation of Directed Information

Student Name : Jiantao Jiao
Advisor : Tsachy Weissman
Research Areas: Information Systems
Directed Information is an information-theoretic quantity defined for a pair of jointly distributed sequences, which is often a natural measure of the extent to which one sequence is relevant for causal inference on the other. It first appeared in the context of feedback communications, and was subsequently found useful in identifying and measuring causal relevance in neurological, biological and financial data. The well-known measure --Granger Causality-- is one special case, as it is the manifestation of directed information under certain (Gaussian and linear) model assumptions.

We propose four estimators of the directed information rate between a pair of stochastic processes based on universal probability assignment. We show these estimators converge almost surely, without requiring knowledge of the structure of corresponding stochastic processes a priori. Guided by these theoretical results, the proposed estimators are implemented using the context tree weighting (CTW) as the universal probability assignment. Experiments on synthetic and real data are presented, in particular, we show there is significant causal influence from the US stock market to the Chinese stock market, but relatively low causal influence in the reverse direction from 1990 to 2011.

Jiantao Jiao received the B.Eng. degree with the highest honor in Electronic Engineering from Tsinghua University, Beijing, China, in 2012. He is currently working towards the Ph.D. degree in the Department of Electrical Engineering, Stanford University. His research interests include information theory and statistical signal processing, with applications in communication,control, computation, networking, data compression, and learning. Mr. Jiao's study is partially supported by a Stanford Graduate Fellowship (SGF).