Graph Databases for Polyglot Persistence with NotaQL 2017-03-08 - PowerPoint PPT Presentation

Transformations on Graph Databases for Polyglot Persistence with NotaQL 2017-03-08 Johannes Schildgen schildgen@cs.uni-kl.de Yannick Krück Stefan Deßloch

Polyglot Persistence 3

Polyglot Persistence db.product.insert ({…}) db.category.find() 4

Polyglot Persistence OUT._id <- IN._k.split (‘_‘)[0], OUT._id <- IN._k.split (‘_‘)[0], OUT.clicks <- SUM(IN._v) OUT.clicks <- SUM(IN._v) Data Transformation db.product.insert ({…}) INCR dvd_174_cnt db.category.find() 5

6

Graph 8

𝐻 = (𝑊, 𝐹) 9

𝑤 1 𝑤 2 10

𝑤 1 𝑤 2 11

𝑤 1 𝑤 2 12

0.2 𝑤 1 𝑤 2 0.6 13

Property Graphs Friend 𝑤 2 𝑤 1 since:2015-11-11 firstname:Kai, firstname:Ute, nastname:Li lastname:Li 14

1 0 0 0 1 1 1 1 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 15

𝑤 1 : 𝑤 2 , 𝑤 5 𝑤 2 : 𝑤 1 , 𝑤 2 , 𝑤 3 𝑤 3 : 𝑤 3 , 𝑤 4 𝑤 4 : 𝑤 1 𝑤 5 : ∅ 16

SELECT xV.value, xN.value SELECT xV.value, xN.value vid attribute value vid label FROM knoten kai, kanten, knoten FROM knoten kai, kanten, knoten xV, knoten xN, knotenlabels xV, knoten xN, knotenlabels v1 vorname Kai v1 person WHERE kai.attribute = ‘ vorname ’ WHERE kai.attribute = ‘ vorname ’ v1 nachname Li v2 person AND kai.wert = ‘Kai’ AND AND kai.wert = ‘Kai’ AND kanten.start=kai.vid AND kanten.start=kai.vid AND v2 vorname Ute v2 student kanten.ziel = xV.vid AND xV.vid kanten.ziel = xV.vid AND xV.vid = knotenlabels.vid AND = knotenlabels.vid AND v2 nachname Li v3 person xN.vid=xV.vid AND xN.vid=xV.vid AND v2 geboren 1985-01-01 v4 person xV.attribute =‘ vorname ’ AND xV.attribute =‘ vorname ’ AND xN.attribute =‘ nachname ’ AND xN.attribute =‘ nachname ’ AND … knotenlabels.label = ‘student’ knotenlabels.label = ‘student’ eid start ziel label eid attribute value e1 v1 v3 folgt e1 seit 2015 e2 v2 v3 Bruder e3 seit 2014 e3 v2 v4 folgt e4 seit 2015 e4 v3 v4 folgt e4 priorität 5 17

vid label vid properties v1 person v1 {vorname :“Kai“, nachname :“Li“} v2 person v2 {vorname :“Ute“, nachname :“Li“, v2 student geboren:Date(1985-01-01)} v3 person v3 … v4 person v4 … eid start ziel label properties e1 v1 v3 folgt { seit : 2015 } e2 v2 v3 Bruder { } e3 v2 v4 folgt { seit: 2014 } e4 v3 v4 folgt { seit : 2015, Priorität:5 } 18

Row-id graph properties edges v1 label vorname nachname folgt_v3 person Kai Li 2015 v2 label vorname nachname geboren Bruder_v3 person Ute Li 1985-01-01 folgt_v4 - 2015 v3 label … … person v4 label … … person 19

ergebnis = []; ergebnis = []; { _id:“v1“, kai = db.personen.find( kai = db.personen.find( {vorname :”Kai”},{folgt:1}) {vorname :”Kai”},{folgt:1}) label :“ person “, while(kai.hasNext()) { while(kai.hasNext()) { vorname :“Kai“, p = folgt.next(); p = folgt.next(); nachname :“Li“, for(i in p.folgt) { for(i in p.folgt) { folgt:[{_id:“v2“, seit:2015}] id = p.folgt[i]._id; id = p.folgt[i]._id; } s = db.personen.findOne({_id:id, s = db.personen.findOne({_id:id, { label:”student”, label:”student”, _id:“v2“, {vorname:1,nachname:1}) {vorname:1,nachname:1}) label :[“ person “, “ student “], .toArray(); .toArray(); if(s!=null){ergebnis.concat(s);} if(s!=null){ergebnis.concat(s);} vorname :“Ute“, } } nachname :“Li“, } } geboren:1985 folgt:[{_id:“v4“, seit:2014, prioritaet:5}], Bruder:[“v3“] } ... 20

subjekt prädikat objekt http://dbpedia.org/resource/ rdf:type http://schema.org/Place Krefeld_Hauptbahnhof http://dbpedia.org/resource/ foaf:name Krefeld Hauptbahnhof Krefeld_Hauptbahnhof http://dbpedia.org/resource/ georss:point 51.325833333333335 Krefeld_Hauptbahnhof 6.569444444444445 http://dbpedia.org/resource/ rdf:comment Krefeld Hauptbahnhof ist der größte Krefeld_Hauptbahnhof Bahnhof der Stadt Krefeld. Dort … http://dbpedia.org/resource/ country http://dbpedia.org/resource/Germany Krefeld_Hauptbahnhof http://dbpedia.org/resource/ foaf:name Germany Germany 21

• Storage • Index Support (+Apache Lucene) • Graph Query Languages • ACID • REST API • Replication • … 22

{ { _id: 19, _id: 77, firstname : “Jane“, firstname : “Kate“, age: 36, city : „Bern“ age: 38, city : „ Rome “ } } OUT._id <- IN._id, OUT._id <- IN._id, OUT.type <- ‘ person ‘, OUT.type <- ‘ person ‘, OUT.firstname <- IN.firstname, OUT.firstname <- IN.firstname, OUT.age <- IN.age, OUT.age <- IN.age, OUT.city <- IN.city OUT.city <- IN.city type : person type : person _id : 77 _id : 19 name : Kate name : Jane age : 37 age : 35 city : Rome city : Bern 23

{ { _id: 19, _id: 77, firstname : “Jane“, firstname : “Kate“, age: 36, city : „Bern“ age: 38, city : „ Rome “ } } OUT._id <- IN._id, OUT._id <- IN._id, OUT.type <- ‘ person ‘, OUT.type <- ‘ person ‘, OUT.$(IN.*.name()) <- IN.@ OUT.$(IN.*.name()) <- IN.@ type : person type : person _id : 77 _id : 19 name : Kate name : Jane age : 37 age : 35 city : Rome city : Bern 24

{ { _id: 19, _id: 77, firstname : “Jane“, firstname : “Kate“, age: 36, city : „Bern“ age: 38, city : „ Rome “ } } IN- FILTER: type=‘ person ‘, IN- FILTER: type=‘ person ‘, OUT._id <- IN._id, OUT._id <- IN._id, OUT.$(IN.name()) <- IN.@ OUT.$(IN.name()) <- IN.@ type : person type : person _id : 77 _id : 19 name : Kate name : Jane age : 37 age : 35 city : Rome city : Bern 25

Accessing & Traversing Edges 26

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN.age IN.age 37 35 27

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._>e IN._>e 28

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._>e.since IN._>e.since 2016-01-01 29

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._<e.since IN._<e.since 2016-01-01 30

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._e.since IN._e.since 2016-01-01 2016-01-01 31

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._e_ IN._e_ type : person type : person _id : 77 _id : 19 name : Kate name : Jane age : 37 age : 35 city : Rome city : Bern 32

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._e_.name IN._e_.name Jane Kate 33

type : person type : person _id : 77 _id : 19 friend name : Kate name : Jane since:2016-01-01 age : 37 age : 35 city : Rome city : Bern IN._e ?(‘ friend ‘)_.name IN._e ?(‘ friend ‘)_.name Jane Kate 34

35

Creating Edges 36

type : person type : person type : person _id : 26 _id : 77 _id : 25 mother father name : Carla name : Kate name : Carl age : 77 age : 37 age : 57 city : Rome city : Rome city : Rome grandmother via: ‘ father ‘ create an edge to every persons grandmother … OUT._>e OUT._>e _id= IN._>e?(‘ mother ‘||‘ father ‘)_._>e?(‘ mother ‘)_._ id ?( ) <- EDGE( ) ‘ grandmother ‘ , via <- IN._>e[@]._l 37

Iterative Computations 𝑄𝑆(𝑞) 𝑄𝑏𝑕𝑓𝑠𝑏𝑜𝑙: 𝑄𝑆 𝑟 = ෍ |𝑝𝑣𝑢 𝑞 | 𝑞∈𝑗𝑜 𝑟 OUT._id <- IN._>e_._id, OUT._id <- IN._>e_._id, OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) 38

Iterative Computations 𝑄𝑆(𝑞) 𝑄𝑏𝑕𝑓𝑠𝑏𝑜𝑙: 𝑄𝑆 𝑟 = ෍ |𝑝𝑣𝑢 𝑞 | 𝑞∈𝑗𝑜 𝑟 REPEAT: 10, REPEAT: 10, OUT._id <- IN._>e_._id, OUT._id <- IN._>e_._id, OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) 39

Iterative Computations 𝑄𝑆(𝑞) 𝑄𝑏𝑕𝑓𝑠𝑏𝑜𝑙: 𝑄𝑆 𝑟 = ෍ |𝑝𝑣𝑢 𝑞 | 𝑞∈𝑗𝑜 𝑟 REPEAT: 99999999, REPEAT: 99999999, OUT._id <- IN._>e_._id, OUT._id <- IN._>e_._id, OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) 40

Iterative Computations 𝑄𝑆(𝑞) 𝑄𝑏𝑕𝑓𝑠𝑏𝑜𝑙: 𝑄𝑆 𝑟 = ෍ |𝑝𝑣𝑢 𝑞 | 𝑞∈𝑗𝑜 𝑟 REPEAT: -1, REPEAT: -1, OUT._id <- IN._>e_._id, OUT._id <- IN._>e_._id, OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) OUT.pr <- SUM(IN.pr/COUNT(IN._>e._id)) 41