论文标题
在线贝叶斯推断，主动学习和主动抽样的边际和联合跨凝结和预测
Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling
论文作者
论文摘要
当我们只专注于边际预测分布（边际预测性）时，原则上的贝叶斯深度学习（BDL）并没有掌握其潜力。最近的工作强调了（贝叶斯）顺序决策的联合预测的重要性，从理论和合成的角度进行。我们提供基于现实世界应用程序的其他实用论点，以关注共同的预测：我们讨论在线贝叶斯推论，这将使我们能够在不进行重新培训的情况下考虑其他数据时进行预测，我们建议使用主动学习和主动抽样进行新的挑战性评估设置。这些环境是由对边际和联合预测的检查，各自的跨凝管以及它们在离线和在线学习中的位置的动机。在Wen等人的工作基础上，它们比以前建议的更现实。 （2021）和Osband等。 （2022），专注于在线监督环境中评估近似BNN的性能。然而，最初的实验提出了有关这些思想在具有当前BDL推理技术的高维参数空间中这些想法的可行性的问题，我们建议实验可能有助于进一步阐明当前研究的实用性。重要的是，我们的工作重点介绍了当前研究中以前未知的差距，并且需要更好地近似联合预测。
Principled Bayesian deep learning (BDL) does not live up to its potential when we only focus on marginal predictive distributions (marginal predictives). Recent works have highlighted the importance of joint predictives for (Bayesian) sequential decision making from a theoretical and synthetic perspective. We provide additional practical arguments grounded in real-world applications for focusing on joint predictives: we discuss online Bayesian inference, which would allow us to make predictions while taking into account additional data without retraining, and we propose new challenging evaluation settings using active learning and active sampling. These settings are motivated by an examination of marginal and joint predictives, their respective cross-entropies, and their place in offline and online learning. They are more realistic than previously suggested ones, building on work by Wen et al. (2021) and Osband et al. (2022), and focus on evaluating the performance of approximate BNNs in an online supervised setting. Initial experiments, however, raise questions on the feasibility of these ideas in high-dimensional parameter spaces with current BDL inference techniques, and we suggest experiments that might help shed further light on the practicality of current research for these problems. Importantly, our work highlights previously unidentified gaps in current research and the need for better approximate joint predictives.
