IWCG Privacy Update 26.10.2018 TPAC, Lyon
Agenda 15m - High-level overview of documented threat vectors and potential mitigations … based on issues and explainer at Privacy & Security repo https://github.com/immersive-web/privacy-and-security 15m - Questions and Discussion
What is WebXR? API that allows sites to build virtual and augmented reality experiences. Provides an API for determining device orientation for rendering purposes, and exposes new types of real-world data for augmented reality. For VR, allows scenes to be rendered in a headset (+smartphone +desktop) For AR, allows rendering of virtual objects into the real world ● On screen-based displays, manages camera rendering… ● … and supports non-screen displays such as transparent HMDs
Privacy Approach The privacy & security repo documents patterns for web-based XR threat vectors. Specifically - What are the threat vectors if a site has access to a data type - What are considerations and potential mitigations Focus of this presentation: Mitigations unique to the threat vector / data type. E.g. User consent is always a potential mitigation
Threat vectors Considered thus far.. ● Perception of camera access ● Access to real-world geometry ● Object identification ● Permissions ● Pose (device orientation) and frame of reference data
Perception of Camera Access
Why does it matter? In AR, on screen-based devices the user may think the site has access to the camera… even if the site doesn’t.
Perception of Camera Access Threat Vector #1… without actual camera access 1. A malicious site could create an AR session where the user agent displays the camera (and nothing else); 2. A smartphone user could visit that site, be presented with a front-camera view, potentially capturing a compromising situation; 3. The user would likely close the camera view, quickly; 4. The site could then say “Thank you for that picture! We’ll add to the public gallery. Please pay our $10 membership fee to control who can see the picture.” Threat Vector #2… without actual camera access 1. The user is using an optically see-through device; 2. A malicious 2D web page - without access to camera data - recognizes that it is on an optically see-through device (e.g. detecting through CSS that it is on an additive display); 3. The page renders a camcorder viewfinder with a blinking REC indicator in the corner (e.g. on an additive display, the viewfinder could be filled with black pixels, and would thus show the real world). This would make it appear that the web page can record real world imagery, even though it can’t. Mitigation: Clear communication to the user about site access to camera data
Real-World Geometry Data
Why does it matter? New web APIs allow sites to access 3D geometry that represents the user’s physical environment.
Threat Vectors ● Personally Identifying Information ○ Face analysis ○ Embossed credit card # ● Fingerprinting ○ Room geometry ● Location ○ If near a known location (e.g. geometry of eiffel tower) ○ Independent of GPS sensors (and thus possibly without permission for geolocation) ● Physical security ○ Map out inside of private, secure area; get geometry of a door key
Threat Vectors ● Profiling ○ Income estimation (size of house, valuable items) ○ User height (from ground plane) ○ Determine previously visited places based on what geometry is available ○ Even bounding boxes can be used for profiling, context ● Historical geometry / Session Data Privacy ○ Some AR systems have geometry from before the user’s current session ■ May allow analysis of where the user has previously been (e.g. historical path analysis) ○ Others merge geometry from the session into larger planes
Potential Mitigations Throttling & Precision: Not clear what value this will bring, because ● Lower precision hurts the user experience ● Multiple sessions can merge inaccurate data to infer ground truth ● Low-resolution data still has threat vectors ○ But: Users can visualize low-resolution data; that may be a useful policy tool Filtering: User agent can limit geometry access: ● E.g. Only allow access to what is seen in the current session ● But: May need AR sub-system to cooperate (e.g. geometry behind wall) Clearing data: Where supported can prevent historical geometry being shared
Object Identification
Why does it matter? It’s useful in augmented reality for the site to be able to identify 2D images or 3D objects. But: The user may not know what’s being looked for. Bottle of water
Threat Vectors (Object Identification) ● Location ○ Storefronts, QR codes, street signs, configurations of objects ● Profiling ○ Income estimation (particularly dangerous if location is known) ● Compromising info ○ E.g. Embarrassing imagery or objects ● User Safety / Obscuring ○ Put user in a dangerous situation, e.g. covering a stop sign or a hole in the ground ○ Put others in a dangerous situation, e.g. make it look like someone else holds a gun ○ Embarrassing situations, e.g. insert inflammatory images during a negotiation
Object Identification Possible Mitigations ● Throttling ○ Limit # of objects or images the site can identify ● User visibility ○ Allow the user to see what the site is looking for ● No access to object position ○ Prevent site from actually knowing where in the user’s space an object was located ○ However, even the existence of an object may be enough for a bad actor ● Composition rules : UA prevents obscuring important objects ○ E.g. a “safe field” where the user always sees the real world
Permissions
Permissions AR uses a lot of sensors and data! Considerations for web permissions: ● Permission fatigue ● Ambiguity and complexity: What is the site doing exactly? ● Persistent or long-running permissions beyond scope of session Possible mitigations ● Time-limited permissions ● Session-scoped ‘modes’ (e.g. AR Mode) ○ Flexibility: Could either ask for consent, or give notification, or both depending on data access ○ Clear scope: User agent has more UX flexibility to inform the user clearly ○ Enforcement: Only the data permitted is allowed, and only during the session
Pose and Frame of Reference Data (This is still a WIP…)
Access to Pose and Frame of Reference Data Sites can access: ● Pose of viewer or device: Position and Orientation ○ Note: Position is real-world scale, but is not a real-world location ● Floor height, space bounds On a variety of devices and in various contexts ● Immersive (HMDs) ● Inline ○ On phone ○ On desktop/laptop ○ In HMD while 2D browsing
Threat Vectors ● Gaze Tracking ○ What is user looking at? ● Input sniffing ○ When: Input ⇒ Device Orientation / Motion ⇒ Pose ○ This has been demonstrated to read PINs on mobile devices at 20Hz ○ May be possible on desktop/laptop (keyboard) or HMDs (gaze inputs, virtual keyboards) ● Profiling ○ Using: Bounds, Floor Height, Action Analysis (e.g. walking) ● Fingerprinting ○ Using: Bounds, Floor Height, Gait Analysis ● Location ○ May be possible to identify specific location based on trajectory or path ○ Example: Recognizing pattern of major roadways, recognizing pathways of a campus
WebXR Modes, Implied Threats Mode (>> Submode) Available Data Implied Threat Vectors frameOfRef = NULL None None Stationary >> position-disabled Orientation Input sniffing Stationary >> eye-level Orientation, Position Input sniffing, Location, Profiling, Gaze, Fingerprinting Stationary >> floor-level Orientation, Position, Floor Level Input sniffing, Location, Profiling, Gaze, Fingerprinting Bounded Orientation, Position, Region Input sniffing, Location, Profiling, bounds, Floor Level Gaze , Fingerprinting Unbounded Orientation, Position Input sniffing, Location, Profiling, Gaze , Fingerprinting
Potential Mitigations Input Sniffing ● Focused & Visible ● Restrict pose data to: ○ Same origin only (i.e. pose data only provided to same-origin elements) ○ Single-origin only (i.e. only one origin can have pose data at a time) ● Secure context ● Feature policy ● User consent
… more generally... Potential mitigations similar to Generic Sensor API ● Focused & Visible ● Same origin only ● Secure context only ● Feature policy ● User consent (as appropriate)
Potential Mitigations Location ● Limit position ranges based on frame of reference (e.g. bounded, stationary) ● User consent (e.g. unbounded) Non-solutions ● Data throttling ● Errors or rounding ● Inline vs. exclusive
Potential Mitigations Gaze Tracking ● No position (orientation only) ● Only give pose when user is looking at inline element ● … or only when user is looking at page (when inline) ● Blur when a frame is not focused
Potential Mitigations Fingerprinting ● Data errors / rounding (for bounds) ○ E.g. Restrict bounds to rectangle only ● Fixed position (for gait) Profiling ● Restrict bounds data to rectangle ○ Harder to determine room type
Join the conversation! https://github.com/immersive-web/privacy-and-security
Q&A
Recommend
More recommend
