Extended Property Graphs and Cypher on Gradoop Martin Junghanns - PowerPoint PPT Presentation

1st openCypher Implementers Meeting 8 February 2017 Walldorf, Germany Extended Property Graphs and Cypher on Gradoop Martin Junghanns University of Leipzig – Database Research Group

Grado Grado doop doop op op Extende ended d Proper perty ty Graph phs Cypher her on Grado doop op Conc nclusio lusion „An open -source graph dataflow system for declarative analytics of heterogeneous graph data .“ 2 2 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado Grado doop doop op op Extende ended d Proper perty ty Graph phs Cypher her on Grado doop op Conc nclusio lusion Graph Dataflow Operators E xtended P roperty G raph M odel (EPGM) I/O Apache Flink Operator Implementation Distributed Operator Execution (Apache Flink) Distributed Graph Storage (Apache HDFS) 3 3 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende Extende ended ended d Proper d Proper perty perty ty Graph ty Graph phs phs Cypher her on Grado doop op Conc nclusio lusion 2 | Area |title:Shire • Vertices and directed Edges Hobbit name : Frodo • Logical Graphs yob : 2968 hates 4 • since : 2301 Identifiers 3 • Type Labels 1 | Area |title:Mordor • Properties leaderOf memberOf knows 5 1 since : 2790 since : 2013 2 3 Orc since : 2990 1 2 name : Bolg Orc Clan name : Azog name : Tribes of Moria founded : 1981 4 hates 5 Hobbit name : Samwise 4 4 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende Extende ended ended d Proper d Proper perty perty ty Graph ty Graph phs phs Cypher her on Grado doop op Conc nclusio lusion Graph Operators/Transformations Unary Binary Aggregation Combination Logical Graph Overlap Pattern Matching Transformation Exclusion Grouping Equality Subgraph Fusion Call Selection Union Pattern Matching Intersection Graph Collection Distinct Difference Limit Equality Apply Reduce Call 5 5 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende Extende ended ended d Proper d Proper perty perty ty Graph ty Graph phs phs Cypher her on Grado doop op Conc nclusio lusion Subgraph 4 3 4 1 2 3 1 2 3 UDF 5 LogicalGraph graph3 = readFromHDFS(); LogicalGraph graph4 = graph3. subgraph ( (vertex => vertex.getLabel().equals( ‘Green’ )), (edge => edge.getLabel().equals( ‘orange’ ))); 6 6 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende Extende ended ended d Proper d Proper perty perty ty Graph ty Graph phs phs Cypher her on Grado doop op Conc nclusio lusion Pattern Matching (Single Graph Input) 4 3 5 4 4 1 2 3 1 2 3 Pattern 5 Graph Collection GraphCollection collection = graph3. match ( ‘(:Green) -[:orange]- >(:Orange)’ ); 7 7 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion „Which two clan leaders hate each other and one of them knows Frodo over one to ten hops?“ 8 8 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion 9 9 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion PlanTableEntry | type: GRAPH | all-vars: [...] | proc-vars: [...] | attr-vars: [] | est-card: 23 | prediates: () | Plan : |-FilterEmbeddingsNode{filterPredicate=((c1 != c2) AND (o1 != o2))} |.|-JoinEmbeddingsNode{joinVariables=[o2], vertexMorphism=H, edgeMorphism=I} |.|.|-JoinEmbeddingsNode{joinVariables=[o1], vertexMorphism=H, edgeMorphism=I} |.|.|.|-JoinEmbeddingsNode{joinVariables=[c1], vertexMorphism=H, edgeMorphism=I} |.|.|.|.|-FilterAndProjectVerticesNode{vertexVar=c1, filterPredicate=((c1.label = Clan)), projectionKeys=[]} |.|.|.|.|-FilterAndProjectEdgesNode{sourceVar='o1', edgeVar='_e0', targetVar='c1', filterPredicate=((_e0.label = leaderOf)), projectionKeys=[]} |.|.|.|-JoinEmbeddingsNode{joinVariables=[o1], vertexMorphism=H, edgeMorphism=I} |.|.|.|.|-FilterAndProjectVerticesNode{vertexVar=o1, filterPredicate=((o1.label = Orc)), projectionKeys=[]} |.|.|.|.|-FilterAndProjectEdgesNode{sourceVar='o1', edgeVar='_e1', targetVar='o2', filterPredicate=((_e1.label = hates)), projectionKeys=[]} |.|.|-JoinEmbeddingsNode{joinVariables=[o2], vertexMorphism=H, edgeMorphism=I} |.|.|.|-JoinEmbeddingsNode{joinVariables=[h], vertexMorphism=H, edgeMorphism=I} |.|.|.|.|-FilterAndProjectVerticesNode{vertexVar=h, filterPredicate=((h.label = Hobbit) AND (h.name = Frodo Baggins)), projectionKeys=[]} |.|.|.|.|-ExpandEmbeddingsNode={startVar='o2', pathVar='_e3', endVar='h', lb=1, ub=10, direction=OUT, vertexMorphism=H, edgeMorphism=I} |.|.|.|.|.|-FilterAndProjectVerticesNode{vertexVar=o2, filterPredicate=((o2.label = Orc)), projectionKeys=[]} |.|.|.|.|.|-FilterAndProjectEdgesNode{sourceVar='o2', edgeVar='_e3', targetVar='h', filterPredicate=((_e3.label = knows)), projectionKeys=[]} |.|.|.|-JoinEmbeddingsNode{joinVariables=[c2], vertexMorphism=H, edgeMorphism=I} |.|.|.|.|-FilterAndProjectVerticesNode{vertexVar=c2, filterPredicate=((c2.label = Clan)), projectionKeys=[]} |.|.|.|.|-FilterAndProjectEdgesNode{sourceVar='o2', edgeVar='_e2', targetVar='c2', filterPredicate=((_e2.label = leaderOf)), projectionKeys=[]} 10 10 10 10 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion Embedding - Data structure used for intermediate results 0 1 2 3 4 5 6 7 8 9 Identifiers 1 37 5 3 7 8 45 99 12 3 0 1 2 Properties Embedding Frodo Baggins 1.22 Saruman 0 Paths 45: [4,1,33] EmbeddingMetaData – Stores information about the embedding content Mapping : Variable -> ID Column { h : 0, e1 : 1, o2 : 5, ...} Mapping : Variable.Property -> Property Column { h.name : 0, h.height : 1, c1.name : 2, ...} 11 11 11 11 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion Project Filter name: Frodo Baggins height: 1.22m [ ] Hobbit(name=Frodo Baggins) gender : male city: Bag End id Properties 𝜏 𝑀𝑏𝑐𝑓𝑚=𝐼𝑝𝑐𝑐𝑗𝑢 (𝑊) 𝜌 ℎ.𝐽𝑒 (𝑊 ′ ) 1 {…} ∧𝑜𝑏𝑛𝑓=𝐺𝑠𝑝𝑒𝑝 h.id h.name h.height … h.id 2 {…} 31 Frodo 1.22 … 32 3 {…} … … DataSet<Vertex> DataSet<Embedding> FlatMap (Vertex -> Embedding) 12 12 12 12 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion JoinEmbeddings Left: (c1:Clan)<-[:hasLeader]-(o1:Orc) Right: (o1:Orc)-[:hates]->(o2.Orc) c.id _e1.id o1.id 51 11 2 Combine 52 12 3 c.id _e1.id o1.id _e2.id o2.id Check for vertex/edge isomorphism, … … … 51 11 2 13 5 Remove duplicate entries 𝑀 ⋈ 𝑝1.𝑗𝑒 𝑆 52 12 3 14 3 o1.id _e2.id o2.id … … … 2 13 5 3 14 3 … … … DataSet<Embedding > DataSet<Embedding> FlatJoin (lhs, rhs -> combine(lhs, rhs)) DataSet<Embedding> 13 13 13 13 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion ExpandEmbeddings Left: (o2:Orc) Edge: (o2)-[:knows*1..10]->(h) 𝐹′ ⋈ 𝑓.𝑢𝑗𝑒=_𝑓3.𝑡𝑗𝑒 𝐹 o2.id o2.id _e3.id h.id 5 3 [26] 31 _e3.sid _e3.id _e3.tid 𝑀 ⋈ 𝑝2.𝑗𝑒=_𝑓3.𝑡𝑗𝑒 𝐹 3 [26,31,27] 32 5 26 31 3 [26,31,27,32,28] 33 31 27 32 Combine 32 28 33 Check for vertex/edge isomorphism BulkIteration DataSet<Embedding> FlatJoin (lhs, rhs -> DataSet<Embedding> combine(lhs, rhs)) DataSet<Embedding> 14 14 14 14 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion 15 15 15 15 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns

Grado doop op Extende ended d Proper perty ty Graph phs Cypher Cypher her on Grado her on Grado doop doop op op Conc nclusio lusion 16 16 16 16 Extended Property Graphs and Cypher on Gradoop – 1st openCypher Implementers Meeting – Martin Junghanns