From Context Awareness to Socially Aware and Interactive Systems - PowerPoint PPT Presentation

Overview • Part 1: Introduction of key concepts • Part 2: Basic technologies – Sensing – Reasoning • Part 3: – phenomenology of complex socio-technical systems – applications and outlook

Overview 1. General Background 2. Examples

What is Context Recognition ? Artificial Intelligence: imitating human cognition - often video based - includes interpretation Context Recognition: mapping signals from simple sensors onto a set of predefined, environment related states Embedded Controllers: feedback control loop

Context Aware Systems 2000 acceleration, sound, light... • Gellersen, Schmidt, Beigl 1999, from the SmartIts project • also Day, Abowd, Pentland, Starner, Schiele,... around 2000

Context Aware Systems 2000 modes of locomotion acceleration, sound, light... • Gellersen, Schmidt, Beigl 1999, from the SmartIts project • also Day, Abowd, Pentland, Starner, Schiele,... around 2000

....and 2012 GPS Compass Microphone Bluetooth Camera Wireless LAN Gyroscope Accelerometer

Context Recognition IEEE PAMI 06, PMC 07, Pattern Rec 08, ACM TIST 11, Springer PAA 11 ISWC, Pervasive,..

Basic Principle acc gyro

Basic Principle acc ??????? gyro class membership inferred from coordinates only

Basic Principle acc gyro class membership inferred from coordinates only

Separation Plane acc gyro recognition given through separation plane

complex separation plane acc gyro often complex and impossible to represent in an analytic way

Recognition Issues 1. representation of the acc separation plane – memory requirements ! 2. locating a point with respect to the separation place gyro 3. Identifying the separation plane – Generation from a sample – cost minimazation

Learning Separation Planes acc gyro

Learning Separation Planes acc separation plane is not perfect • ambivalent sensor data gyro • size of the sample • sensor noise

Begrüssung +1g 0g +2g /+2g 0g /0g -2g /-2g time

Analyzing Recordings Torso acceleration [g] Sitting Standing Standing Sitting Sitting up down Handshake Wrist Arm Arm Arm on table Arm resting on table moving moving time [s]

Time Series Process P„ Process P„„ t z„ 1 z„„ 1 z„ 2 z„„ 2 z„ 3 z„„ 3 … … z„ N z„„ N

Time Series Process P„ Process P„„ ????? t z* 1 z* 2 z* 3 … z* N

Context Rec. Challenges 1. Ambiguous sensors – e.g. phone in a pocket tracking patient care 2. Lack of representative training data – huge inter and intra person variations – training with real users often not practicable 3. Inconsistent state space – e.g walking, having a meal and taking a pill – different granularities, temporal scales, and levels of abstraction 4. Dynamically changing sensor configurations – phone moves around, different phones, new sensors

Context Problem Types situation state action parameters

Context Problem Types situation state action parameters - physical characterization - ‘ classical ’ signal processing ?

Context Problem Types window size Signal stream situation shift state action parameters - separable in time invariant feature space - sliding window algorithms