Privacy in Visual Localization
Doktorsavhandling, 2025
Visual localization is a very important piece in the puzzle for many computer vision applications. It is essential for seamlessly running applications that rely on knowing the pose of the camera with respect to the 3D surroundings. Many such applications run on user devices such as smartphones, VR headsets, \etc, that have limited storage and compute. In such cases, it is practically more feasible to off-load the computations to a powerful cloud-based server. As visual information is stored on a remote server, or sent to it for obtaining the camera pose, it becomes important to ensure that private user content is not accessed in an unauthorized manner.
This thesis contributes towards increased protection of cloud-based visual localization systems against threats to user privacy. One of the popular ways to represent scenes and images for localization is through a sparse set of 3D or 2D points respectively. However, the point-based representation can reveal highly detailed images of the user scene, prompting research in obfuscating the geometry of these points. Papers A and B of this thesis highlight an important vulnerability of such geometry obfuscation methods that claim to preserve user privacy while enabling visual localization. This urges future methods to include clear guarantees about their claims of privacy preservation.
Paper C introduces a novel attack vector in a scenario where an adversary gains access to query the localization server of another user's scene with its own set of images. We show that an attacker can gain unauthorized information about presence and positions of objects in a user's 3D space. Based on the insight that recovering details from a very sparse geometric signal is difficult, we explore representing a scene in the form of only its outline. Paper D presents an efficient and accurate method to reconstruct the edges of a scene from images.
This thesis contributes towards increased protection of cloud-based visual localization systems against threats to user privacy. One of the popular ways to represent scenes and images for localization is through a sparse set of 3D or 2D points respectively. However, the point-based representation can reveal highly detailed images of the user scene, prompting research in obfuscating the geometry of these points. Papers A and B of this thesis highlight an important vulnerability of such geometry obfuscation methods that claim to preserve user privacy while enabling visual localization. This urges future methods to include clear guarantees about their claims of privacy preservation.
Paper C introduces a novel attack vector in a scenario where an adversary gains access to query the localization server of another user's scene with its own set of images. We show that an attacker can gain unauthorized information about presence and positions of objects in a user's 3D space. Based on the insight that recovering details from a very sparse geometric signal is difficult, we explore representing a scene in the form of only its outline. Paper D presents an efficient and accurate method to reconstruct the edges of a scene from images.
Ethical Computer Vision
Visual Localization
Privacy-Preserving Localization
ES 51, Hörsalsvägen
Opponent: Stefan Leutenegger, Technical University of Munich, Germany
Författare
Kunal Chelani
Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik
Privacy-Preserving Representations are not Enough: Recovering Scene Content from Camera Poses
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,;Vol. 2023-June(2023)p. 13132-13141
Paper i proceeding
How privacy-preserving are line clouds? Recovering scene details from 3D lines
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,;(2021)p. 15663-15673
Paper i proceeding
Chelani K, Benbihi A, Sattler T, Kahl F EdgeGaussians - 3D Edge Mapping via Gaussian Splatting
Chelani K, Benbihi A, Kahl F, Sattler T, Kukelova Z. Obfuscation Based Privacy Preserving Representations are Recoverable Using Neighborhood Information
Spatially intelligent devices, such as autonomous vehicles and robotic vacuum cleaners, are already widespread, with many more such devices expected to become mainstream. Additionally, specialized headsets and smartphone applications are increasingly leveraging Augmented Reality (AR) for various business applications. The core technology enabling the spatial awareness of these devices is visual localization, a computer vision algorithm that estimates a device’s position and orientation based on images captured by its camera.
Visual localization algorithms require substantial storage, computational power, and energy, which many devices lack. To overcome these limitations, computations are offloaded to cloud-based servers. While this approach enhances efficiency, it also introduces privacy concerns, as sensitive visual data must be transmitted and stored remotely, increasing the risk of unauthorized access or misuse.
Firstly, this thesis exposes critical vulnerabilities in several data representations that have been purported to be privacy-preserving. It highlights the need for future methods to establish clear guarantees and define the conditions under which their privacy-preserving capabilities remain valid. Secondly, it shows a novel attack mechanism that is possible irrespective of the data representation used, underlining the need for a holistic analysis of potential privacy-losses in the localization process. Finally, based on the observation that sparse data storage and transfer is a key to not revealing detailed visual information about user scenes, it presents an efficient and accurate method to represent the scene in the form of only its geometric boundary.
Visual localization algorithms require substantial storage, computational power, and energy, which many devices lack. To overcome these limitations, computations are offloaded to cloud-based servers. While this approach enhances efficiency, it also introduces privacy concerns, as sensitive visual data must be transmitted and stored remotely, increasing the risk of unauthorized access or misuse.
Firstly, this thesis exposes critical vulnerabilities in several data representations that have been purported to be privacy-preserving. It highlights the need for future methods to establish clear guarantees and define the conditions under which their privacy-preserving capabilities remain valid. Secondly, it shows a novel attack mechanism that is possible irrespective of the data representation used, underlining the need for a holistic analysis of potential privacy-losses in the localization process. Finally, based on the observation that sparse data storage and transfer is a key to not revealing detailed visual information about user scenes, it presents an efficient and accurate method to represent the scene in the form of only its geometric boundary.
Styrkeområden
Informations- och kommunikationsteknik
Ämneskategorier (SSIF 2025)
Datorseende och lärande system
Robotik och automation
Drivkrafter
Innovation och entreprenörskap
ISBN
978-91-8103-191-1
Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 5649
Utgivare
Chalmers
ES 51, Hörsalsvägen
Opponent: Stefan Leutenegger, Technical University of Munich, Germany