Implementations Jean-Paul Calbimonte jean-paul.calbimonte@hevs.ch - PowerPoint PPT Presentation

R2RML Mappings :ObsValueMap URI of subject rr:subjectMap [ rr:template "http://opensense.epfl.ch/data/ObsResult_NO2_{sensor}_{time}"]; rr:predicateObjectMap [ URI of predicate rr:predicate qu:numericalValue; rr:objectMap [ rr:column "no2"; rr:datatype xsd:float; ]]; Object: colum name rr:predicateObjectMap [ rr:predicate obs:uom; rr:objectMap [ rr:parentTriplesMap :UnitMap; ]]. :ObservationMap rr:subjectMap [ rr:template "http://opensense.epfl.ch/data/Obs_NO2_{sensor}_{time}"]; rr:predicateObjectMap [ Column names in a template rr:predicate ssn:observedProperty; rr:objectMap [ rr:constant opensense:NO2]]; Can be used for mapping both databases, CSVs, JSON, etc 21 http://streamreasoning.org/events/rsp2016 21

An example: TripleWave Sources Running modes http://streamreasoning.org/events/rsp2016 22

RDF Streams in W3C RSP https://www.w3.org/community/rsp/ http://streamreasoning.github.io/RSP-QL/RSP_Requirements_Design_Document/ Graph-based RDF-friendly :g1 {:axel :isIn :RedRoom. :darko :isIn :RedRoom} {:g1, prov:generatedAtTime, "2015-06-18T12:00:00Z"^^xsd:dateTime} :g2 {:axel :isIn :BlueRoom. } {:g2, prov:generatedAtTime, "2015-06-18T12:00:35"^^xsd:dateTime} :g3 {:minh :isIn :RedRoom. } {:g3, prov:generatedAtTime, "2015-06-18T12:02:07"^^xsd:dateTime} ... Flexible Flexible time metadata property :g_1 :startsAt "2015-06-18T12:00:00"^^xsd:dateTime :g_1 :endsAt "2015-06-18T13:00:00"^^xsd:dateTime Intervals :g_2 :validBetween [:startsAt "2015-06-18T12:00:00"^^xsd:dateTime; :endsAt "2015-06-18T13:00:00"^^xsd:dateTime] http://streamreasoning.org/events/rsp2016 23

RSP Engine Implementations http://streamreasoning.org/events/rsp2016 24

Existing RSP systems (oversimplified!) C-SPARQL: RDF Store + Stream processor  • Combined architecture RDF Store C-SPARQL continuous translator query results Stream processor CQELS: Implemented from scratch. Focus on  performance • Native + adaptive joins for static-data and streaming data continuous CQELS Native RSP results query http://streamreasoning.org/events/rsp2016 25

Existing RSP systems (oversimplified!) SPARQL stream : Ontology-based stream query answering  • Virtual RDF views, using R2RML mappings • SPARQL stream queries over the original data streams. continuous SPARQLStream rewriter DSMS/CEP results query R2RML mappings EP-SPARQL: Complex-event detection  • SEQ, EQUALS operators Prolog continuous EP-SPARQL translator engine results query Instans: RETE-based evaluation  http://streamreasoning.org/events/rsp2016 26

Classification of existing systems Triple window Co-ocurrence Union, Join, R2S operator Time window Continuous Aggregates Sequence, Optional, execution Model Filter ✗ ✔ ✗ ✗ ✗ ✗ TA- TA-RDF Limited SPARQL ✗ ✔ ✗ ✗ ✗ ✗ ✗ tRDF tSPARQL ✔ ✔ ✗ ✔ ✔ ✗ ✗ RDF Streaming Stream SPARQL ✔ ✔ ✔ ✔ ✔ RDF Rstream time C-SPARQL Stream only function ✔ ✔ ✔ ✔ ✔ ✗ RDF Istream CQELS Stream only ✔ ✔ ✔ ✔ ✗ ✔ ✗ (Virtual) SPARQLStr RDF eam Stream ✔ ✔ ✔ ✗ ✗ ✗ ✔ EP- RDF SPARQL Stream ✔ ✔ ✔ ✗ ✗ ✗ ✗ RDF Instans Disclaimer: other features may be missing http://streamreasoning.org/events/rsp2016 27

C-SPARQL http://streamreasoning.org/events/rsp2016 28

A Reminder of SPARQL http://streamreasoning.org/events/rsp2016 29

Where C-SPARQL Extends SPARQL http://streamreasoning.org/events/rsp2016 30

C-SPARQL Language Query and Stream Registration http://streamreasoning.org/events/rsp2016 31

C-SPARQL Language Query and Stream Registration All C-SPARQL queries over RDF streams are continuous  • Registered through the REGISTER statement The output of queries is in the form of  • Instantaneous tables of variable bindings • Instantaneous RDF graphs • RDF stream Only queries in the CONSTRUCT form can be registered as  generators of RDF streams Composability:  • Query results registered as streams can feed other registered queries just like every other RDF stream http://streamreasoning.org/events/rsp2016 32 32

C-SPARQL Language Query registration - Example Using the social stream fb, Who is where?  REGISTER QUERY QWhoIsWhereOnFb AS PREFIX : <http://… /sr4ld2014-onto#> SELECT ?room ?person FROM STREAM <http://…/ fb> [RANGE 1m STEP 10s] WHERE { ?person1 :posts [ :who ?person ; :where ?room ] . } The resulting variable bindings has to be interpreted as an  instantaneous. It expires as soon as the query is recomputed http://streamreasoning.org/events/rsp2016 33

C-SPARQL Language Stream registration - Example Results of a C-SPARQL query can be stream out for down  stream queries REGISTER STREAM SWhoIsWhereOnFb AS PREFIX : <http://…/sr4ld2014 -onto#> CONSTRUCT { ?person :isIn ?room } FROM STREAM <http://…/ fb> [RANGE 1m STEP 10s] WHERE { ?person1 :posts [ :who ?person ; :where ?room ] . } The resulting RDF triples are streamed out on an RDF  stream • More details in the C-SPARQL Engine hands-on session http://streamreasoning.org/events/rsp2016 34

C-SPARQL Language Stream Registration - Notes The output is constructed in the format of an RDF stream.  Every query execution may produce from a minimum of  zero triples to a maximum of an entire graph. The timestamp is always dependent on the query  execution time only, and is not taken from the triples that match the patterns in the WHERE clause. http://streamreasoning.org/events/rsp2016 35

C-SPARQL Language FROM STREAM Clause http://streamreasoning.org/events/rsp2016 36

C-SPARQL Language FROM STREAM Clause FROM STREAM clauses are similar to SPARQL datasets  • They identify RDF stream data sources • They represent windows over a RDF stream They define the RDF triples available for querying and  filtering. http://streamreasoning.org/events/rsp2016 37

C-SPARQL Language FROM STREAM Clause - windows physical: a given number of triples  logical: a variable number of triples which occur during a  given time interval (e.g., 1 hour) • Sliding: they are progressively advanced of a given STEP (e.g., 5 minutes) • Tumbling: they are advanced of exactly their time interval http://streamreasoning.org/events/rsp2016 38

C-SPARQL Language FROM STREAM Clause - Example Using the social stream fb, how many people are in the  same room? Count on a window of 1 minute that slides every 10 seconds REGISTER QUERY HowManyPoepleAreInTheSameRoom AS PREFIX : <http://…/sr4ld2014 -onto#> SELECT ?room (COUNT(DISTINCT ?s) as ?person) FROM STREAM <http://…/ fb> [RANGE 1m STEP 10s] WHERE { ?person1 :posts [ :who ?person ; :where ?room ] . } GROUP BY ?room http://streamreasoning.org/events/rsp2016 40

C-SPARQL Language C-SPARQL reports only snapshots Incoming Time window [RANGE 40s STEP 10s] Window timestamped content t+40 t+50 t+60 t+70 t+80 t RDF triples t+10 t+20 t+30 d 1 d 1 d 1 d 1 d 1 t+40 d 1 d 1 d 2 d 2 d 2 d 2 t+50 d 1 , d 2 d 2 t+60 d 1 , d 2 d 3 d 3 d 3 t+70 d 1 , d 2 , d 3 t+80 d 2 , d 3 http://streamreasoning.org/events/rsp2016 41

C-SPARQL Language Multiple FROM STREAM Clause - Example Using the social stream fb and fs , how many people are  in the same room? Count on a window of 1 minute that slides every 10 seconds REGISTER QUERY HowManyPoepleAreInTheSameRoom AS PREFIX : <http://… /rsp2014-onto#> SELECT ?room (COUNT(DISTINCT ?s) as ?person) FROM STREAM <http://…/ fb > [RANGE 1m STEP 10s] FROM STREAM <http://…/ fs > [RANGE 1m STEP 10s] WHERE { ?person1 :posts [ :who ?person ; :where ?room ] . } GROUP BY ?room http://streamreasoning.org/events/rsp2016 42

C-SPARQL Language Query Chaining A C-SPARQL query Q 1 registered using the STREAM clause  streams results on an RDF stream A down stream C-SPARQL query Q 2 can open a window on  the RDF stream of Q 1 using the FROM STREAM clause E.g.,  :Bob :posts [ :who :Bob ; :where :BlueRoom ] . :Bob :isIn :BlueRoom . 4 Is in on 4 Stream Stream query Is In across f and 4 query f Is with on f Stream Stream query :Carl :isIn :BlueRoom . :Carl :posts [ :who :Carl , :Bob ] . :Carl :isWith :Bob . http://streamreasoning.org/events/rsp2016 43

C-SPARQL Language TimeStamp Function http://streamreasoning.org/events/rsp2016 44

C-SPARQL Language TimeStamp Function – Syntax and Semantics The timestamp of a triple can be bound to a variable using  a timestamp() function Syntax  • timestamp(variable|IRI|bn, variable|IRI, variable|IRI|bn|literal) Semantics  Triple Result of evalutaion It is not in the window Type Error Timestamp of triple It appears once in the window It appears multiple times in The timestamp of the most recent the window triple http://streamreasoning.org/events/rsp2016 45

C-SPARQL Language TimeStamp Function - Example Who is “following” whom?  REGISTER QUERY FindFollowers AS PREFIX f: <http://larkc.eu/csparql/sparql/jena/ext#> PREFIX : <http://…/sr4ld2014 -onto#> SELECT ?someOne ?someOneElse ?room FROM STREAM <http://… /isIn> [RANGE 1m STEP 10s] WHERE { ?someOne :isIn ?room . ?someOneElse :isIn ?room . FILTER(?someOne!=?someOneElse ) FILTER ( f:timestamp (?someOne :isIn ?room) < f:timestamp (?someOneElse :isIn ?room) } http://streamreasoning.org/events/rsp2016 46

C-SPARQL Language Accessing background Information C-SPARQL allows for asking the engine to issue the query  also against RDF graphs using the FROM clauses. E.g., Where else can Alice go?  REGISTER QUERY WhereElseCanAliceGo AS PREFIX : <http://…/sr4ld2014 -onto#> SELECT ?room FROM STREAM <http://… /isIn> [RANGE 10m STEP 10m] FROM < http://… /bgInfo> WHERE { IRI identifying the graph ?:Alice :isIn ?someRoom . containing the ?someRoom :isConnectedTo ?room . background information } http://streamreasoning.org/events/rsp2016 47

C-SPARQL Language C-SPARQL queries and reasoning - example Memo  • posts is a sub property of observes Data  RDF graph Time-stamp Stream :RedSensor :observes [ :who :Alice; :where :RedRoom ] . t 1 sensors :Bob :posts [ :who :Bob ; :where :RedRoom] . t 2 fs Query under RDFS entailment regime  REGISTER QUERY QueryUnderRDFSEntailmentRegime AS PREFIX : <http://…/sr4ld2014 -onto#> SELECT ?x ?room ?person FROM STREAM <http://…/ fs> [RANGE 1m STEP 10s] FROM STREAM <http://… /sensors> [RANGE 1m STEP 10s] WHERE { ?x :observes [ :who ?person ; :where ?room ] .} Results at t 2 + 10s  ?x ?room ?person :RedSensor :RedRoom :Alice :Bob :RedRoom :Bob http://streamreasoning.org/events/rsp2016 48

Introduction C-SPARQL Engine Architecture  Simple, modular architecture  It relies entirely on existing technologies  Integration of • DSMSs (Esper) and • SPARQL engines (Jena- ARQ) http://streamreasoning.org/events/rsp2016 49

Introduction C-SPARQL Engine Features at a glance 1/3 In-memory RDF stream Processing  • Continuous queries, filtering, aggregations, joins, sub-queries via C-SPARQL • Push based • Reactive C-SPARQL Engine 0.9.5 supports  • SPARQL 1.1 (tested with http://www.w3.org/wiki/SRBench) • query chaining • background RDF graph access and update (via SPARQL 1.1 Update) • naïve stream reasoning (via Jena Generic Rule Reasoner) • time aware matching via timestamp function  http://streamreasoning.org/events/rsp2016 50

Introduction C-SPARQL Engine Features at a glance 2/3 Extensible Middleware  • Runtime management of – RDF streams – C-SPARQL query – Result listerners • API driven Quick start available  • C-SPARQL Engine – http://streamreasoning.org/download/csparqlreadytogopack Source code are released open source under Apache 2.0  • C-SPARQL Engine – https://github.com/streamreasoning/CSPARQL-engine – https://github.com/streamreasoning/CSPARQL-ReadyToGoPack http://streamreasoning.org/events/rsp2016 51

Introduction C-SPARQL Engine Features at a glance 3/3 Known limitations  • large background data and timestamp function can spoil performance • no support for named graphs and named streams • no support for multiple windows on the same stream • triple based windows are buggy http://streamreasoning.org/events/rsp2016 52

Introduction C-SPARQL Engine as general RSP RSP-services proposes a unified interface for the RDF  stream processors and offers Rest services to interact with them. RSP-services-csparql represents the specific  implementation of the RSP-services for the C-SPARQL engine (more detailed information in the hands-on session) Quick start available  • RDF Stream Processging RESTful Interface (RSP-service) for C-SPARQL Engine – http://streamreasoning.org/download/rsp-service4csparql Source code are released open source under Apache 2.0  • RSP-services – https://github.com/streamreasoning/rsp-services-csparql – https://github.com/streamreasoning/rsp-services-api – https://github.com/streamreasoning/rsp-services-client-example http://streamreasoning.org/events/rsp2016 53

Resources Read out more  • C-SPARQL semantics – Davide Francesco Barbieri, Daniele Braga, Stefano Ceri, Emanuele Della Valle, Michael Grossniklaus: C-SPARQL: a Continuous Query Language for RDF Data Streams. Int. J. Semantic Computing 4(1): 3-25 (2010) • Most recent syntax – D. F. Barbieri, D. Braga, S. Ceri, E. Della Valle, M. Grossniklaus, Querying RDF streams with C-SPARQL, SIGMOD Record 39 (1) (2010) 20 – 26. • RSP-services – M Balduini,E Della Valle: A Restful Interface for RDF Stream Processors. International Semantic Web Conference (Posters & Demos) 2013: 209-212 Downloads  • http://streamreasoning.org/download/csparqlreadytogopack • http://streamreasoning.org/download/rsp-service4csparql Contact points  • marco.balduini@polimi.it • emanuele.dellavalle@polimi.it http://streamreasoning.org/events/rsp2016 54

SPARQL Stream & Morph-streams http://streamreasoning.org/events/rsp2016 55

Morph-streams: Overview SELECT ?proximity π timed,prox FROM STREAM <http://streamreasoning.org/SensorReadings.srdf> [NOW – 5 S] σ prox>10 WHERE { ?obs a ssn:ObservationValue; ω qudt:numericalValue ?proximity; 5 Seconds FILTER (?proximity>10) } sens SELECT prox FROM sens.win:time(5 sec) Query WHERE prox >10 Algebra rewriting expression SPARQL Stream SNEE Esper Client Query R2RML Mappings Processing GSN Cosm pull/push Data [tuples] translation [triples/bin Other dings] Morph-streams procesing SPARQL Stream queries https://github.com/jpcik/morph-streams http://streamreasoning.org/events/rsp2016 56

SPARQLStream Language ISTREAM DSTREAM RSTREAM WINDOW FROM NAMED STREAM Underlying data source restrictions http://streamreasoning.org/events/rsp2016 57

SPARQLStream Language  ‘FROM’ [‘NAMED’] ‘STREAM’ StreamIRI ‘[’ Window NamedStream  ‘]’  ‘NOW - ’ Integer TimeUnit [ UpperBound ] [ Slide ] Window   ‘TO NOW - ’ Integer TimeUnit UpperBound  Slide  ‘SLIDE’ Integer TimeUnit   ‘MS’ | ‘S’ | ‘MINUTES’| ‘HOURS’ | ‘DAY’ TimeUnit  SELECT ISTREAM ?room FROM NAMED STREAM <http://www.streamreasoning.org/streams/socialsensor.srdf> [NOW-10 S] WHERE {… Select  ‘SELECT’ [ Xstream ] [ Distinct | Reduced ] …   ‘RSTREAM’ | ‘ISTREAM’ | ‘DSTREAM’ Xstream  http://streamreasoning.org/events/rsp2016 58

SPARQLStream: examples SPARQL Stream PREFIX sr4ld: <http://www.streamreasoning.org/ontologies/socialsensor,owl#> SELECT ?room FROM NAMED STREAM <http://www.streamreasoning.org/streams/socialsensor.srdf> [NOW-10 S] WHERE { ?obs sr4ld:observedBy ?sensor. ?obs sr4ld:where ?room. } All rooms where something was observed in the last 10s PREFIX sr4ld: <http://www.streamreasoning.org/ontologies/socialsensor,owl#> SELECT (COUNT(?person) AS ?nmb) ?room FROM NAMED STREAM <http://www.streamreasoning.org/streams/socialsensor.srdf> [NOW-10 S] WHERE { ?obs sr4ld:who ?pers. ?obs sr4ld:where ?room. } GROUP BY ?room Number of persons observed in each room in the last 10s http://streamreasoning.org/events/rsp2016 59

Underlying Query Processors Esper SNEE • CEP/DSMS • DSMS/Sensor Network Query Evaluator • EPL language • Compile queries to sensor code SELECT prox FROM sensors.win:time(5 minute) SELECT prox FROM sensors [FROM NOW-5 WHERE prox >10 MINUTES TO NOW] WHERE prox >10 GSN Cosm/Xively • Sensor middleware • Sensor middleware • REST API • Open platform • REST API http://montblanc.slf.ch:22001/multidata?vs[0]=sens ors& http://api.cosm.com/v2/feeds/14321/datastreams/ field[0]=proximity_field&c_min[0]=10& 4?start=2012-05-15T05:00:00Z&end=2012-05- from=15/05/2012+05:00:00&to=15/05/2012+10:00: 15T10:00:00Z 00 http://streamreasoning.org/events/rsp2016 60

Morph-streams: Overview SELECT ?proximity π timed,prox FROM STREAM <http://streamreasoning.org/SensorReadings.srdf> [NOW – 5 σ prox>10 S] WHERE { ω ?obs a ssn:ObservationValue; 5 Seconds qudt:numericalValue ?proximity; sens FILTER (?proximity>10) } SELECT prox FROM sens.win:time(5 sec) Query WHERE prox >10 Algebra rewriting expression SPARQL Stream SNEE Esper Client Query R2RML Mappings Processing GSN Cosm pull/push Data [tuples] translation [triples/bin Other dings] Morph-streams procesing SPARQL Stream queries https://github.com/jpcik/morph-streams http://streamreasoning.org/events/rsp2016 61

3rd: Mapping the two models Observation where Room observes detections (person, room,…) Sensor who subClassOf Define mappings Person http://streamreasoning.org/events/rsp2016 62

R2RML – There is a recommendation! We can use the W3C recommendation http://streamreasoning.org/events/rsp2016 63

R2RML - Overview http://streamreasoning.org/events/rsp2016 64

Encoding in R2RML the stream Mapping definition name stream :triplesMap a rr:TriplesMap; attributes rr:logicalTable [ rr:tableName ” sensors "; ] subject URI rr:subjectMap [ rr:template "http://streamreasoning.org/data/Observation/{ person }{ timed }"; rr:class sr4ld:Observation ; rr:graph sr4ld:socialstream.srdf ]; triple predicate + object rr:predicateObjectMap [ rr:predicate sr4ld:who ; rr:objectMap [ rr:template “http ://streamreasoning.org/data/Person/{ person }” ]];. the object (a URI in this case) http://streamreasoning.org/events/rsp2016 65

Underlying Query Processors SPARQLStream R2RML SELECT ?proximity π FROM STREAM <http://streamreasoning.org/SensorReadings.srdf> timed, [NOW – 5 S] prox WHERE { σ prox>10 ?obs a ssn:ObservationValue; Query qudt:numericalValue ?proximity; rewriting FILTER (?proximity>10) } ω 5 Seconds sensors SELECT prox FROM sensors [FROM NOW-5 MINUTES TO NOW] WHERE prox >10 SNEE (DSMS) SELECT prox FROM sensors.win:time(5 minute) WHERE prox >10 Esper (CEP) http://montblanc.slf.ch:22001/multidata?vs[0]=sensors&field[0]=proximi ty_field&c_min[0]=10& from=15/05/2012+05:00:00&to=15/05/2012+10:00:00 GSN (middlwr) http://api.cosm.com/v2/feeds/14321/datastreams/4?start=2012-05- 15T05:00:00Z&end=2012-05-15T10:00:00Z Cosm Xively http://streamreasoning.org/events/rsp2016 66

Underlying query processors Features Esper SNEE GSN Cosm/Xively ✔ ✔ ✔ Projection Fixed ✔ ✔ ✖ ✖ Proj expression ✔ ✔✖ only window ✖ ✖ Joins ✖ ✔✖ not windows ✔ ✖ Union ✔ ✔ ✔ ✖✔ limited Selection ✔ ✔ ✔✖ ✖ Aggregates ✔ ✔ ✔ ✔ Time window ✔ ✔ ✔ ✖ Tuple window ✔ ✔ ✖ ✖ R2S ✔ ✖ ✖ ✖ Conjunction, Disj ✔ ✖ ✖ ✖ Repetition pattern ✔ ✖ ✖ ✖ Sequence http://streamreasoning.org/events/rsp2016 67

Morph-streams: With reasoning! Rewrite taking into account Ontology the ontology TBox TBox Query Algebra rewriting expression SPARQL Stream SNEE Esper Client Query R2RML Mappings Processing GSN Cosm pull/push Data [tuples] translation [triples/bin Other dings] Morph-streams procesing SPARQL Stream queries https://github.com/jpcik/morph-streams http://streamreasoning.org/events/rsp2016 68

Reasoning with Morph-streams Rewriting the SPARQLStream queries:  SELECT ?x FROM NAMED STREAM <http://linkeddata.es/s/obs.srdf> [NOW - 5 MINUTES] WHERE { ?x ssn:observedBy ?y } SELECT ?x FROM NAMED STREAM <http://linkeddata.es/s/obs.srdf> [NOW - 5 MINUTES] WHERE { {?x ssn:observedBy ?y} UNION {?x a ssn:Observation} UNION {?x a aws:TemperatureObservation} UNION {?x a aws:HumidityObservation} } http://streamreasoning.org/events/rsp2016 69

Morph-streams: With reasoning! Rewrite only happens once Ontology TBox Then continuous query is registered Query rewriting Query Algebra + rewriting expression SPARQL Stream SNEE translation Esper Client Query R2RML Mappings Processing GSN Cosm pull/push Data [tuples] translation [triples/bin Other dings] https://github.com/jpcik/morph-streams http://streamreasoning.org/events/rsp2016 70

Now some code Morph-streams: Coded in Scala  JAR bundle, use it from Scala or Java code  Maven, Sbt  Examples  • One off query • Register continuous query • Pull data • Push • Basic REST https://github.com/jpcik/morph-streams  https://github.com/jpcik/morph-web  http://streamreasoning.org/events/rsp2016 71

Code examples Parse SPARQLStream  val query = “PREFIX sr4ld: <…>. SELECT ?a …” val syntax= StreamQueryFactory.create(query); Mapping Execute One-off query  val query = “PREFIX sr4ld: <…>. SELECT ?a …” mapping=Mapping(new URI(mappings/social.ttl)) val adapter:QueryEvaluator=Application.adapter(system) val results= adapter.executeQuery(query,mapping) Bindings http://streamreasoning.org/events/rsp2016 72

Code examples Register and Pull  val queryid= adapter.registerQuery(query,mapping) val results1=adapter.pull(queryid) val results2=adapter.pull(queryid) Query identifier Implement receiver Register and Push  class ExampleReceiver extends StreamReceiver{ override def receiveData(s:SparqlResults):Unit= Logger.debug("got: "+res) } val receiver=new ExampleReceiver val queryid= adapter.listenToQuery(query,mapping,receiver) For Java users : Exactly the same interface! http://streamreasoning.org/events/rsp2016 73

Querying RSPs in Practice http://streamreasoning.org/events/rsp2016 74

RSP Querying Example with CQELS (code.google.com/p/cqels) CQELS continuous query: SELECT ?person ?loc WHERE { STREAM <http://deri.org/streams/rfid> [RANGE 3s] {?person :detectedAt ?loc} Tightly coupled listeners } Results delivery: push & pull? ExecContext context=new ExecContext(HOME, false); register query String queryString =" SELECT ?person ?loc … ContinuousSelect selQuery=context.registerSelect(queryString); adding selQuery.register(new ContinuousListener() listener { public void update(Mapping mapping){ get result updates String result=""; for(Iterator<Var> vars=mapping.vars();vars.hasNext();) result+=" "+ context.engine().decode(mapping.get(vars.next())); System.out.println(result); } }); http://streamreasoning.org/events/rsp2016 75

CQELS fed by a TripleWave WebSocket val conf = ConfigFactory.load.getConfig("experiments.rsp") val qq="""CONSTRUCT { ?s ?p ?o } WHERE { STREAM <ws://localhost:4040/primus> [RANGE 0ms] {?s ?p ?o} }""" val sys=new RspSystem("wstreams") val cqels=new CqelsEngine sys.startStream(Props( new WebSocketStream(cqels,"ws://localhost:4040/primus",conf))) cqels.registerQuery(qq, cqels.createListener(lissy)) def lissy(triples:TripleList):Unit={ println("tikki: "+triples) } http://streamreasoning.org/events/rsp2016 76

Similar models, similar (not equals!) query languages SELECT ?sensor FROM NAMED STREAM <http://www.cwi.nl/SRBench/observations> [NOW-3 HOURS SLIDE 10 MINUTES] SPARQL Stream WHERE { ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; om-owl:result [ om-owl:floatValue ?value ] . } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) SELECT ?sensor FROM STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 1h STEP 10m] WHERE { C-SPARQL ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; om-owl:result [ om-owl:floatValue ?value ] . } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) SELECT ?sensor WHERE { STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 10800s SLIDE 600s] { ?observation om-owl:procedure ?sensor ; om-owl:observedProperty weather:WindSpeed ; CQELS om-owl:result [ om-owl:floatValue ?value ] .} } GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float ) http://streamreasoning.org/events/rsp2016 77

Query using SQL on Streams Powerful languages for Union, Join, R2S operator Time window Continuous Sequence, Aggregates Optional, ocurrence execution function continuous query processing window Triple Model Time Filter Co- W3C RSP  review features in existing ✗ ✔ ✗ ✗ ✗ ✗ ✗ TA-SPARQL TA-RDF Limite d systems ✗ ✔ ✗ ✗ ✗ ✗ ✗ ✗ tSPARQL tRDF  agree on fundamental ✔ ✔ ✗ ✔ ✔ ✗ ✗ ✗ Streaming RDF operators SPARQL Stream  discuss on possible ✔ ✔ ✔ ✔ ✔ ✗ ✗ ✔ C-SPARQL RDF Stream semantics ✔ ✔ ✔ ✔ ✔ ✗ ✗ ✗ CQELS RDF https://www.w3.org/community/rsp/wiki/RSP_Query_Features Stream ✔ ✔ ✔ ✔ ✗ ✔ ✗ ✗ RSP is not always/only SPARQL- SPARQLStrea (Virtual) m RDF like querying Stream ✔ ✔ ✔ ✗ ✗ ✗ ✔ ✗ SPARQL protocol is not enough EP-SPARQL RDF Stream RSP RESTful interfaces? ✔ ✔ ✔ ✗ ✗ ✗ ✗ ✗ Instans RDF http://streamreasoning.org/events/rsp2016 78

W3C RSP-CG: RSP-QL PREFIX e: <http://somevocabulary.org/> PREFIX s: <http://someinvasivesensornetwork.org/streams#> PREFIX g: <http://somesocialnetwork.org/graphs#> PREFIX : <http://acrasycompany.org/rsp> Register REGISTER STREAM :GallehaultWasTheBar UNDER ENTAILMENT REGIME <http://www.w3.org/ns/entailment/RIF> AS CONSTRUCT ISTREAM { Continuously look for stream ?poi rdf:type :Gallehault ; :count ?howmanycouples ; Reasoning bars where people are :for (?somebody ?someoneelse) } falling in love (because of FROM NAMED WINDOW :veryLongWindow ON s:1 [RANGE PT4H STEP PT1H] FROM NAMED WINDOW :longWindow ON s:1 [FROM NOW-PT35M TO NOW-PT5M STEP PT5M] a book ) FROM NAMED WINDOW :shortWindow ON s:1 [RANGE PT10M STEP PT5M] FROM NAMED GRAPH g:SocialGraph Time windows FROM GRAPH g:POIs WHERE { ?poi rdf:type e:bar . WINDOW :veryLongWindow { Access to time {?somebody e:enters ?poi} BEGIN AT ?t3 They entered the same bar {?someoneelse e:enters ?poi} BEGIN AT ?t4 Sequencing FILTER(?t3>?t4) } WINDOW :longWindow { { ?somebody e:isCloseTo ?someoneelse MINUS { ?somebody e:isCloseTo ?yetanotherone . They are close to each other, FILTER (?yetanotherone != ?someoneelse) } } WITH DURATION ?duration with no-one else Duration FILTER (?duration>="PT30M"^^xsd:duration) } WINDOW :shortWindow { { ?somebody e:exits ?bar} BEGIN AT ?t1 { ?someoneelse e:exits ?bar } BEGIN AT ?t2 They get out together FILTER (abs(?t2-?t1)<"PT1M"^^xsd:duration ) } GRAPH g:SocialGraph { Stored Graphs FILTER NOT EXIST { ?somebody e:knows ?someoneelse } } Didn’t know each other FILTER (?somebody != ?someoneelse) } Aggregates AGGREGATE { GROUP BY ?poi COUNT(?somebody) AS ?howmanycouples } http://streamreasoning.org/events/rsp2016 79

RSP Communication http://streamreasoning.org/events/rsp2016 80

RDF Stream Processing continuous queries notify push results stream consumers stream producers producer input RDF streams streams of results cont. query consumer RSP RSP Engine subscribe subscribe Engine continuous queries RDF background knowledge graphs RSP Implementations http://streamreasoning.org/events/rsp2016 81

Reactive Systems Events: Event-Driven react to Load: Scalable Failure: Resilient Users: Responsive Jonas Boner. Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems. 2013. http://streamreasoning.org/events/rsp2016 82

Actor Model communicate send: fire-forget No shared mutable state m through messages Avoid blocking operators Actor Lightweight objects Actor 2 mailbox Loose coupling 1 state behavior non-blocking response Implementations: e.g. Akka for Java/Sca m Parent Actor Actor 2 1 Actor Actor Supervisi m Actor 1 2 on 3 hierarchy X Acto Actor Actor r4 m 2 4 Supervision Remoting http://streamreasoning.org/events/rsp2016 83

RDF Streams: Actors class RdfConsumer extends Actor{ RDF consumer def receive= { Actor receive method case t:Triple => Implements behavior if (t.predicateMatches( RDF.‘type‘)) Message-passing model println(s"received triple $t") } RDF producer Async message passing val sys=ActorSystem.create("system") val consumer=sys.actorOf(Props[RdfConsumer]) class Streamer extends StreamRDF{ override def triple(triple:Triple){ consumer ! triple } } http://streamreasoning.org/events/rsp2016 84

RSP Producer & Consumer Prod ucer RDF streams Cons umer Prod ucer Cons Proce umer ssor Prod ucer Cons umer Prod unresponsive ucer overload faster producers >> slower processor/consumer Overload of the processor/receiver Unresponsiveness in stream processor http://streamreasoning.org/events/rsp2016 85

Dynamic Push-Pull m m m m m m m push m Producer Consumer m m demand flow Push when consumer is faster Consumer Pull when producer is faster Producer m Dynamically switch modes data flow Communication is dynamic depending on demand vs supply http://streamreasoning.org/events/rsp2016 87

Evaluation: throughput Basic dynamic pull push On top of CQELS Limitations of Thread model Not yet fully async http://streamreasoning.org/events/rsp2016 88

Reactive RSP workflows s Etali s CSPARQL Morph Streams Event-driven message passing s Async communication Dyn s Immutable streams amit Transparent Remoting e Parallel and distributed s CQELS Supervised Failure Handling TrOWL Responsive processing Minimal agreements: standards, serialization, interfaces Formal models for RSPs and reasoning Working prototypes/systems! Reactive RSPs http://streamreasoning.org/events/rsp2016 89

RSPs and the Linked Data Principles http://streamreasoning.org/events/rsp2016 90

URIs as Names of Things http://mysensorweb.me/mytemperature http://mysensorweb.me/mytemperature/20151110Z10:00: 00 http://mysensorweb.me/mytemperature/latest http://mysensorweb.me/mytemperature/lastMinute http://mysensorweb.me/mytemperature/lastMonth http://mysensorweb.me/mytemperature/avgLastMonth Different concepts Different granularities Different uses http://streamreasoning.org/events/rsp2016 91

HTTP URIs http://mysensorweb.me/mytemperature/latest Internet of Things How about XMPP, CoAP, MQTT? Websockets? http://streamreasoning.org/events/rsp2016 92

De-referenceable URIs GET http://mysensorweb.me/mytemperature/latest :Obs1 a my:TemperatureObservation; my:hasValue 33.5 ; my:hasUnit u:Celsius; my:atTime “20151110Z10:00:00”. GET http://mysensorweb.me/mytemperature Get the whole stream? GET http://mysensorweb.me/mytemperature/lastMonth Get continuous updates? http://streamreasoning.org/events/rsp2016 93

Link to other URIs • Broken links? http://mysensorweb.me/mytemperature/20151110Z10:00:00 • Mix streaming and stored data • Persist or not persist? • Volatile links? http://streamreasoning.org/events/rsp2016 94

For Java/Scala developers http://streamreasoning.org/events/rsp2016 95

Semantic Web Devs Have fun Love challenges Need cool tools RDF4J http://streamreasoning.org/events/rsp2016 96

Scala: Functions and Objects JVM language Both object and functional oriented Easy Java-interop Reuse Java libraries Growing community http://streamreasoning.org/events/rsp2016 97

RDF in Jena: in Scala sw:John vcard:FN “John Smith” Smith Not too useful ; and () String personURI = "http://somewhere/JohnSmith"; Type Model model = ModelFactory.createDefaultModel(); inference model.createResource(personURI).addProperty(VCARD.FN,"John Smith"); val personURI = "http://somewhere/JohnSmith" val model = ModelFactory.createDefaultModel model.createResource(personURI).addProperty(VCARD.FN,"John Smith") boilerplate val personURI = "http://somewhere/JohnSmith" Terser & compact code implicit val model = createDefaultModel Type-safe DSL add(personURI,VCARD.FN->"John Smith") Compiler takes care String converted to Resource http://streamreasoning.org/events/rsp2016 98

Some more RDF vcard:FN “John Smith” sw:John “John” vcard:Giv Smith _:n en “Smith” vcard:N vcard:Fam ily String personURI = "http://somewhere/JohnSmith"; String givenName = "John"; String familyName = "Smith"; String fullName = givenName + " " + familyName; Model model = ModelFactory.createDefaultModel(); model.createResource(personURI) Blank node .addProperty(VCARD.FN,fullName) .addProperty(VCARD.N,model.createResource() .addProperty(VCARD.Given,givenName) .addProperty(VCARD.Family,familyName)); val personURI = "http://somewhere/JohnSmith" val givenName = "John" Scala DSLs customizable val familyName = "Smith" val fullName = s"$givenName $familyName" implicit val model = createDefaultModel add(personURI,VCARD.FN->fullName, VCARD.N ->add(bnode,VCARD.Given -> givenName, Predicate-objects are pairs VCARD.Family->familyName)) 99 http://streamreasoning.org/events/rsp2016 99

Some more RDF in Jena implicit val m=createDefaultModel val ex="http://example.org/" val alice=iri(ex+"alice") val bob=iri(ex+"bob") val charlie=iri(ex+"charlie") alice+(RDF.`type`->FOAF.Person, FOAF.name->"Alice", FOAF.mbox->iri("mailto:alice@example.org"), FOAF.knows->bob, FOAF.knows->charlie, FOAF.knows->bnode) bob+ (FOAF.name->"Bob", FOAF.knows->charlie) charlie+(FOAF.name->"Charlie", FOAF.knows->alice) Still valid Jena RDF You can do it even nicer http://streamreasoning.org/events/rsp2016 100

Exploring an RDF Graph ArrayList<String> names=new ArrayList<String>(); NodeIterator iter=model.listObjectsOfProperty(VCARD.N); while (iter.hasNext()){ Imperative iteration of collections RDFNode obj=iter.next(); if (obj.isResource()) names.add(obj.asResource() .getProperty(VCARD.Family).getObject().toString()); else if (obj.isLiteral()) names.add(obj.asLiteral().getString()); Type-based conditional execution } Type casting val names=model.listObjectsOfProperty(VCARD.N).map{ case r:Resource=> r.getProperty(VCARD.Family).obj.toString Case type case l:Literal=> Map applied to operators l.getString } http://streamreasoning.org/events/rsp2016 101

http://streamreasoning.org/events/rsp2016 102