Semantic Reasoning and Verification for Ambient Assisted Living - PowerPoint PPT Presentation

Semantic Reasoning and Verification for Ambient Assisted Living Thibaut Tiberghien, Mounir Mokhtari, Liu Yan & Dong Jin Song � � FSFMA 2014 2nd French Singaporean Workshop on Formal Methods and Applications 13 May 2014, Singapore

Outline I. Introduction II. Context reasoning - Reasoning strategies - Reasoning architecture III. Validation & Verification IV.Conclusion � 2

Population ageing - Major demographic changes * 2010 2050 Age Age Males Females Males Females over 85 over 85 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 6% 4% 2% 0% 2% 4% 6% 6% 4% 2% 0% 2% 4% 6% � 3 * United Nations, World Population Prospects: The 2010 Revision. http://esa.un.org/unpd/ wpp/index.htm

Population ageing - Elderly support ratio is shrinking - e.g. in Singapore * From 13.5 in 1970 to 2.6 in 2015! � 4 * Resident Old-Age Support Ratio. Department of Statistics Singapore. http://www.singstat.gov.sg

Progression of dementia - Projections of frail elderly population in Singapore * 45000 Number of frail elderly 35000 25000 15000 5000 2000 2010 2020 2030 7 000 beds available in nursing homes Need in 2015: 22 000 beds � 5 * Dr. Philip Yap, Khoo Teck Puat Hospital, Singapore

What is AAL? Sensing Context-aware framework Service provision � 6

Scientific challenges « Rules verification » System verification « Information consistency » « Context awareness » « Sensor prototyping » « User interface plasticity » « Modelling » « Service platform » « Micro context acquisition » « Human factors » « Reasoning techniques » Sensing Context-aware framework Service provision Communication layer « Privacy » « Discovery protocols » « Wireless communication » � 7

Scientific challenges « Rules verification » System verification « Information consistency » « Context awareness » « Sensor prototyping » « User interface plasticity » « Modelling » « Service platform » « Micro context acquisition » « Human factors » « Reasoning techniques » Sensing Context-aware framework Service provision Communication layer « Privacy » « Discovery protocols » « Wireless communication » our research work collaboration with � 8 Liu Yan, Dong Jin Song, NUS (PAT)

Outline I. Introduction II. Context reasoning - Reasoning strategies - Reasoning architecture III. Validation & Verification IV.Conclusion � 9

Related work Rule-based Pattern matching (Semantic Web) (HMM, DBN…) Sensor fusion Detection of patterns and in sequence of binding to context preprocessed sensor data + + leverage prior knowledge no prior knowledge no learning data driven � � - - no implicit patterns labelled dataset for learning sensitive to noise fine grain sensing � 10

Semantic Web Technologies - Metadata-data graph model - Make statements about resources - Triple-based representation « subject predicate object » � - URI and namespaces - RDF, RDF-S, OWL (TBox) - Own namespaces (TBox + Abox) - Serializations: XML, N3, ... � 11

Functional context model Person name Livingroom ... usedBy runningFor snoozeTime Device Kitchen Bedroom name Toilet Bathroom onDevice Resident Caregiver stageForAlert busy Outside isAlone Service Room inRoomSince liveIn/detectedIn/cameFrom watchesAfter inRoomFor name House doesActivitySince hasAckService model doesActivityFor repeat id Environment timeSent helpsWith partOf stage motionMeasured believedToDo useNow locatedIn deployedIn Reminder Object attachedTo Notification Context lastUsed ackHandled Sensor name escalateTo acknowledgement needHands type getConfidenceScore hasValue lastUpdate SensorType Furniture solvedBy hasLastUpdate SensorState deployedIn Deviance Door Activity hasPossibleState/ indicateLocation indicateUse hasCurrentState solved Location riskLevel Class datatypeProperty objectProperty subClassOf � 12 Legend

Rule design: epistemology - Type I: Rules of reasoning - Irrevocable knowledge - Scientific models, common sense - Type II: Rules of fact - Domain knowledge - Experts’ experience, intuition and insight - Possible contradiction (scoring) - Type III: Arbitration rules � 13

Type I - Rules of reasoning ∀ Sensor se; SensorState st; Room r, r2; House h; Resident u; Timestamp t; � {se hasCurrentState st } ⋀ { se hasLastUpdate true } ⋀ { st indicateLocation true } ⋀ { se deployedIn r } ⋀ { r partOf h } ⋀ { u liveIn h } ⋀ { u detectedIn r2 } ⋀ { r2 ≠ r } ⋀ { se lastUpdate t} ⟹ {u detectedIn r } ⋀ { u cameFrom r2 } ⋀ { u inRoomSince t} � 14

Type II - Rules of fact � 15

Type III - Arbitration rules Compute the Rule-Based Confidence Score � 16

Outline I. Introduction II. Context reasoning - Reasoning strategies - Reasoning architecture III. Validation & Verification IV.Conclusion � 17

« La Pensée » A Cognitively Inspired Hybrid Reasoning Architecture Cerebral cortex Cogitation (conscious mind) Cerebration (unconscious mind) Knowledge formalisation S t i m u l u s ( ~ e v e n t ) � 18

Live event processing 1.live event Decision Stimulistener Cortex 3.think 5. decision Interpreter iv.think i.think xi. decision 2.update(live event) Cerebration* Cogitation ii.query iii.update v.dump x.update ix.inferred triples vii.think NTriplestore EYE viii.loaded in memory 0.loaded once at each inference when ubismart starts vi.ontological dump Legend at each inference exchange data invoke service model.n3 home.n3 rules.n3 dump.n3 � 19

Hybrid reasoning "X" Gateway* - flow - in the semsea (ontology) - fused information SmartSpaceComposer (S2C): configuration - no meaning - semantic (meaning) - domain knowledge - low level - mostly useless - useful & calculable - higher level - protocol dependent (code is specific) * this module can have several instances D I K W? - normalization of data (e.g. HEX2DEC) "X" this module is semantic-aware, i.e. linked with NTriplestore - signal forwarding SensApp dev. INFERENCE > introduce libs in > process is not necessary External Module SensApp API semantic but I/O goes > restructure data in a through semantic sea Internal Module more meaningful way handleEvent using processes Abstract Class Legend SensApp* - process data - produce information InfoConsumer* Service Delivery Modules - platform or application using the same data processEvent AtomicService* CoupledService* subscribe Reasoning Modules SensorFlowManager (Routing) Stimulistener publish - merge all "X" signals UbiService* - match sensor/state with semsea bindings start/stop & feedback - registration of SensApps subscribe - write context information in semsea DecisionInterpreter - forward data to relevant SensApp EventBridge* - forward information to InfoConsumers - interpret thoughts (reasoning results) - discovery of new sensors - update triplestore FrontEndService* BackEndService* think - configuration gate for Binder - start/stop service processThoughts - sensors to SensApps bindings XMPPBridge - etc. DPWSBridge Reminder* CallCenter* start/stop/update Sensing Modules Cortex HTTPBridge Notification* ActivityLog* register - handle reasoning cycle (until stable cogitative state) OSGiBridge - alternate between cogitation pass & cerebration pass ServiceGateway display(<ServiceSkin>) & ack think - registration of services - control over services subscribe - status update in triplestore UbiDevice* Cogitation Cerebration* - high level rule-based reasoning register send & ack - decision making IPhone* - threaded asynchronous inference - conscious process Configuration Modules - low level, specific, computational task IPTV* Binder - I/O in seamsea - subconscious process - configuration tool (build, declarative) - choice, download & installation of SensApps Classification DeviceGateway ComLayer* - parametrization of SensApps - bind semantics (to sensor + its events) Statistical Analysis - registration & management of devices - semsea I/O Numeric Computation XMPP REST + Event Bus APNs ServiceSkin* CoreEngine* REST + download MobileGUI* BinderUI* EYE TextToSpeech* SensAppRepository* SVM ... NTriplestore - list SensApps per category of sensor - API to semantic sea (semsea) - provide SensApp for download - provide CRUD access to the triplestore � 20 - implemented as a service

Ontology evolution breakdown - initial ontology 7 2012-06-27 13:07:58 Legend A6 motion pir6_on hasLastUpdate livingroom object occupied id hasCurrentState deployedIn partOf object property detectedIn believedToDo pir6 object johndoe house motion 7 liveIn datatype property pir2 partOf litteral deployedIn hasCurrentState id toilet query hasLastUpdate pir2_off motion A2 update 0 2012-06-27 08:42:27 � 21