论文标题
扩展现实的隐私反射渲染
Privacy-preserving Reflection Rendering for Augmented Reality
论文作者
论文摘要
许多增强现实（AR）应用依赖于全向环境照明来呈现影片虚拟对象。当虚拟对象由反射材料（例如金属球形）组成时，呈现此类对象所需的照明信息可以包含对当前摄像机视图之外的隐私敏感信息。在本文中，我们首次展示了准确驱动的多视图环境照明可以揭示相机外的场景信息并妥协隐私。我们提出了一种简单而有效的隐私攻击，该攻击在许多应用程序场景下提取敏感的场景信息，例如人脸和渲染对象的文本信息。 为了防止这种攻击，我们开发了一种新颖的$ ipc^{2} s $防御和有条件的$ r^2 $防御。我们的$ ipc^{2} s $防御与通用照明重建方法一起使用，在混淆对隐私敏感的信息时保留了场景几何形状。作为概念验证，我们利用现有的OCR和面部检测模型来识别过去的相机观测值的文本和人脸，并模糊与检测区域相关的颜色像素。我们通过将渲染的虚拟对象与使用通用多光的重建技术，ARKIT和$ r^2 $防御的渲染的对象进行比较来评估防御的视觉质量影响。我们的视觉和定量结果表明，我们的防御在各种渲染场景中导致结构相似的反射，而在各种渲染场景中得分高达0.98，同时通过将自动提取成功率降低到最多8.8％，从而保留敏感信息。
Many augmented reality (AR) applications rely on omnidirectional environment lighting to render photorealistic virtual objects. When the virtual objects consist of reflective materials, such as a metallic sphere, the required lighting information to render such objects can consist of privacy-sensitive information that is outside the current camera view. In this paper, we show, for the first time, that accuracy-driven multi-view environment lighting can reveal out-of-camera scene information and compromise privacy. We present a simple yet effective privacy attack that extracts sensitive scene information such as human face and text information from the rendered objects, under a number of application scenarios. To defend against such attacks, we develop a novel $IPC^{2}S$ defense and a conditional $R^2$ defense. Our $IPC^{2}S$ defense, used in conjunction with a generic lighting reconstruction method, preserves the scene geometry while obfuscating the privacy-sensitive information. As a proof-of-concept, we leverage existing OCR and face detection models to identify text and human faces from past camera observations and blur the color pixels associated with detected regions. We evaluate the visual quality impact of our defense by comparing rendered virtual objects to ones rendered with a generic multi-lighting reconstruction technique, ARKit, and $R^2$ defense. Our visual and quantitative results demonstrate that our defense leads to structurally similar reflections with up to 0.98 SSIM score across a variety of rendering scenarios while preserving sensitive information by reducing the automatic extraction success rate to at most 8.8%.
