论文标题
神经神经纹理使SIM2Real保持一致
Neural Neural Textures Make Sim2Real Consistent
论文作者
论文摘要
未配对的图像翻译算法可用于SIM2REAL任务，但许多人无法生成时间一致的结果。我们提出了一种新的方法，该方法将可区分的渲染与图像翻译结合在一起，以使用表面一致性损失和\ emph {神经神经纹理}实现无限时间尺度上的时间一致性。我们称此算法Triton（纹理恢复图像翻译网络）：一种无监督的，端到端的，无状态的Sim2Real算法，通过生成真实的可学习神经纹理来利用输入场景的基础3D几何。通过在场景中的对象上安顿特定的纹理，我们确保框架之间的一致性无效。与以前的算法不同，Triton不仅限于相机的运动 - 它也可以处理对象的运动，使其可用于诸如机器人操纵之类的下游任务。
Unpaired image translation algorithms can be used for sim2real tasks, but many fail to generate temporally consistent results. We present a new approach that combines differentiable rendering with image translation to achieve temporal consistency over indefinite timescales, using surface consistency losses and \emph{neural neural textures}. We call this algorithm TRITON (Texture Recovering Image Translation Network): an unsupervised, end-to-end, stateless sim2real algorithm that leverages the underlying 3D geometry of input scenes by generating realistic-looking learnable neural textures. By settling on a particular texture for the objects in a scene, we ensure consistency between frames statelessly. Unlike previous algorithms, TRITON is not limited to camera movements -- it can handle the movement of objects as well, making it useful for downstream tasks such as robotic manipulation.
