User-Assisted Creation of Semantic Indoor Models for Smarter - PowerPoint PPT Presentation

Fakultät für Informatik Technische Universität München User-Assisted Creation of Semantic Indoor Models for Smarter Applications Matthias Mögerle Advisors: Dr. Marc-Oliver Pahl, Stefan Liebald Supervisor: Prof. Dr.-Ing. Georg Carle Chair of Network Architectures and Services Department of Informatics Technical University of Munich (TUM) 27.11.2017

Motivation: Location in Smart Environments Imagine: Using the microphone of your smartphone to turn on the light in a smart environment. Questions: - All lights? - Which light? - Location of the user? According [1] [WiFi] can deliver room-level accuracy , which is good enough for many applications, including asset-tracking , location-based advertising , location-based information for users , etc. ”People are going to want everyday applications to have location-awareness that goes beyond simple numerical latitude and longitude [..] - places like ’my home’ or ’Ed’s office’ which are within room-level granularity ” [2] [1] T. Gallagher, B. Li, A. G. Dempster, and C. Rizos, “Database updating through user feedback in fingerprint-based Wi-Fi location systems,” UPINLBS 2010, 2010. [2] D. H. Kim, K. Han, and D. Estrin, “Employing user feedback for semantic location services,” in Proceedings of UbiComp ’11, 2011, p. 217. 27.11.2017 Matthias Mögerle 2

Motivation: Location in Smart Environments Sensor Input Clustering Labelling API Home - Lisa’s Room Parasitic use of - Lisa‘s Max‘s Max’s Room Kitchen Sensor Data Room Room - Kitchen - Master Bedroom - Bath Bath Master Living - Closet Bedroom Room - Living Room Closet - Deck Deck Sensors Clustered Locations via Labelled Locations Provide Data via API ML - - - No installation of Detected Rooms Third Parties - - - additional infra- Location recognition Human natural identifiers Smart Environments - - - structure Continuous adaptions User feedback mechanism History about the location - - - - Using available Robust against changes Minimal intrusive and Current Location - - mobile phone sensors Cryptic Identifier maximal accurate Available Locations 27.11.2017 Matthias Mögerle 3

Outline Motivation − Research Goal − Related Work for Indoor Positioning Systems − Analysis and Requirements − State of The Art (Sensors, Clustering, Labelling) − Design and Implementation − Evaluation − Project Status − 27.11.2017 Matthias Mögerle 4

Research Goal and Related Work Research Goal The goal of this thesis is to enhance state of the art research on indoor positioning and combine it with an application for real world use. a) provide accurate indoor location in private smart environments without adding hardware b) focus on measurable results on usability, feasibility and accuracy Related Work Sensors Clustering Labelling Comments Gaussian Fit [3] WiFi Bayesian Inference & Signal Initial Expert Labelling Room-Level Accuracy Intensity Distr. Topological Map Joint Clustering [4] WiFi Pre-processing Initial Expert Labelling Focus on performance gains k-Strongest Fingerprints Individual room-labels No floor plan required IPIN Tracking WiFi, GPS, IMU Sensor fusion, Extreme gradient Floor Plan, Training on one explicit case Competition [5] Sensors, Cameras boost, dead reckoning Hierarchical Map RedPin [6] WiFi Least square error to identify No initial training Labelling by user error between old and new Labelling by user during use locations Individual labels PILS [7] WiFi, Signal Intensity Distr. Individual labels Identifies user movements for Accelerometer Probabilistic model No floor plan required feedback requests Ekahau Real-Time WiFi, na Initial Expert Labelling Industrialized solution Location Systems [8] RFID (active) RFID increase accuracy Floor plan required [3] Andreas Haeberlen, Eliot Flannery, Andrew M. Ladd, Algis Rudys, Dan S. Wallach, and Lydia E. Kavraki. Practical robust localization over large-scale 802.11 wireless networks. Mobile computing and networking - MobiCom ’04 [4] Moustafa A Youssef, Ashok Agrawala, A Udaya Shankar, A Udaya Shankar, and A Udaya Shankar. WLAN location determination via clustering and probability distributions. Pervasive Computing and Communications, 2003.(PerCom 2003) [5] V. C. Ta, D. Vaufreydaz, T. K. Dao, and E. Castelli, “Smartphone-based user location tracking in indoor environment,” 2016 Int. Conf. Indoor Position. Indoor Navig. IPIN 2016, no. October, pp. 4–7, 2016. [6] Philipp Bolliger. Redpin - Adaptive, Zero-Con guration Indoor Localization through User Collaboration. Melt’08, pages 55–60, 2008. [7] Philipp Bolliger, Kurt Partridge, Maurice Chu, and Marc Langheinrich. Improving Location Fingerprinting through motion detection and asynchronous interval labeling. Lecture Notes in Computer, 5561 LNCS:37–51, 2009. [8] https://www.ekahau.com/ 27.11.2017 Matthias Mögerle 5

Analysis and Requirements Research Goal The goal of this thesis is to enhance state of the art research on indoor positioning and combine it with an application for real world use. a) provide accurate indoor location in private smart environments without adding hardware b) focus on measurable results on usability, feasibility and accuracy Requirements <R.1> Sensors to gather location data <R.3> Labelling with user-feedback - No additional infrastructure - User intrusiveness - Using smartphone sensors - Accuracy & Robustness <R.2> Clustering to recognize relevant locations - Environment representation - Continuous learning - Hierarchical (e.g. Home/Living - Room-level accuracy Room) - Accuracy & Robustness <R.4> API for other applications Results − Labelled indoor positioning information for home environments. − Quantifiable results must be comparable to further research. 27.11.2017 Matthias Mögerle 6

Sensors – State of The Art Indoor Available Technology Transmission Range Reliability Pos. Use Infrastructure GPS Low Globally Not required Relies on GPS satellites GSM Medium Max. 35 km per cell Not required Depending on GSM cells Depending on WiFi WiFi High 50-100 meter High infrastructure Depending on Bluetooth Bluetooth (LE) High 10 – 15 meter Medium infrastructure 1m (passive) and up to Depending on RFID RFID Medium Low 100m (active) infrastructure Depending on camera Camera dependent Vision Based Medium Low infrastructure Depending on light Requires LOS Optically Medium Low infrastructure Depending on audio Range of sound Auditive Medium Low infrastructure Low (IMU Unlimited with increasing Depending on sensors IMU Not required only) error 27.11.2017 Matthias Mögerle 7

Clustering – Semi-Supervised Learning – State of The Art Localization - Methods - Cell Proximity (Cell ID) (simple, but inaccurate) - Triangulation (difficult, can be accurate) - Fingerprinting (passive, room-level accurate) Clustering - Methods - K-Nearest Neighbours - SVM (Support Vector Machine) • As default • Supervised Learning • Label of fingerprints with least error • Expensive Training Phase • Quick - DBSCAN/OPTICS - K-Means • Cluster separation • Quick • Cluster density variation - Neural Networks • Expensive Training Phase • Expensive Training Phase 27.11.2017 Matthias Mögerle 8

Labelling – State of The Art Environment Representation - Geocoordinates (Longitude, Latitude) - Semantic Labelling - Hierarchical Representation State of the Art - (Initial) expert labelling - User labelling on map - User labelling on movement status (still/moving) User Feedback - Active Feedback requests - Passive Feedback  As accurate as required and as minimal intrusive as possible. 27.11.2017 Matthias Mögerle 9

My Android Positioning App – Design Sensor Inputs Indoor Positioning (Clustering) Semantic Labelling API to Smart Env. Quantified Self Smart Environment 27.11.2017 Matthias Mögerle 10

My Android Positioning App – Implementation Current Status: Framework to test different clustering and labelling algorithms Sensors My Android Positioning App - Accelerometer - WiFi Fingerprinting Database Clustering - Raw Inputs - - k-Nearest Neighbours Movements - - Planned: Aggregated Fingerprints - - Identified Locations k-Means - - Labels of Locations Support Vector Machines - OPTICS - For Evaluation: Evaluation Unit Event Handler and Listener - Handling Sensor Events Labelling - Handling Location Detections - Hierarchical environment Representation - Handling Location Labelling - Notification for User Feedback - Notification, Vibration,… Background Process - UI to administrate locations - - For Evaluation: Evaluation Unit Android Process API User Interface API to 3 rd Party Apps - UI Elements API to Smart Environments Done To be done 27.11.2017 Matthias Mögerle 11

Evaluation Clustering – Quality of different clustering algorithms Accuracy − Number of Samples vs. Accuracy − Calculation Time/Calculation Cycles − vs. Number of Measurements/Number of Locations Robustness against changes − (reducing available networks, adding additional networks) Labelling – Quality of labelling method Number of notifications vs. accuracy − User-Feedback (SUS Test, Interviews) − Average duration per room labelling − Feedback requests per time and location − Ratio of labelled clusters (dependent on identified clusters) − How motivated has the user been to provide feedback? − 27.11.2017 Matthias Mögerle 12