论文标题
用内核方法的信息理论
Information Theory with Kernel Methods
论文作者
论文摘要
我们考虑通过复制内核希尔伯特空间的相关协方差操作员对概率分布进行分析。我们表明，冯·诺伊曼（Von Neumann）的熵和这些操作员的相对熵与香农熵和相对熵的通常概念密切相关，并具有许多特性。它们与来自概率分布的各种口径的有效估计算法结合在一起。我们还考虑产品空间，并表明对于张量产品内核，我们可以定义互信息和联合熵的概念，然后可以完美地表征独立性，但只能部分条件独立。我们最终展示了这些新的相对熵概念如何导致对数分区函数的新上限，这些概念可以与变异推理方法中的凸优化一起使用，从而提供了新的概率推理方法。
We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are intimately related to the usual notions of Shannon entropy and relative entropy, and share many of their properties. They come together with efficient estimation algorithms from various oracles on the probability distributions. We also consider product spaces and show that for tensor product kernels, we can define notions of mutual information and joint entropies, which can then characterize independence perfectly, but only partially conditional independence. We finally show how these new notions of relative entropy lead to new upper-bounds on log partition functions, that can be used together with convex optimization within variational inference methods, providing a new family of probabilistic inference methods.
