datafun
play

Datafun a functional query language Michael Arntzenius - PowerPoint PPT Presentation

Sep 28, 2023 •592 likes •1.3k views

Datafun a functional query language Michael Arntzenius daekharel@gmail.com http://www.rntz.net/datafun Strange Loop, September 2017 Recurse Center, March 2018 Early stage work What if programming languages were more like query languages? 1.

  1. Datafun a functional query language Michael Arntzenius daekharel@gmail.com http://www.rntz.net/datafun Strange Loop, September 2017 Recurse Center, March 2018

  2. Early stage work

  3. What if programming languages were more like query languages?

  4. 1. What’s a functional query language? 2. From Datalog to Datafun 3. Incremental Datafun

  5. SQL Parent Child Arathorn Aragorn SELECT parent Drogo Frodo FROM parentage E¨ arwen Galadriel WHERE child = "Galadriel" Finarfin Galadriel . . . . . .

  6. Tables as sets Parent Child // set of (parent, child) pairs Arathorn Aragorn { (Arathorn, Aragorn) Drogo Frodo , (Drogo, Frodo) = E¨ arwen Galadriel , (E¨ arwen, Galadriel) Finarfin Galadriel , (Finarfin, Galadriel) . . . . . . ... }

  7. Tuples and sets are just datatypes!

  8. Tuples and sets are just datatypes! If tables are sets, what are queries?

  9. Queries as set comprehensions SELECT parent FROM parentage WHERE child = "Galadriel"

  10. Queries as set comprehensions SELECT parent FROM parentage WHERE child = "Galadriel" ⇒ = { parent | (parent, child) in parentage , child = "Galadriel" }

  11. Queries as set comprehensions: finding siblings SELECT DISTINCT A.child, B.child FROM parentage A INNER JOIN parentage B ON A.parent = B.parent WHERE A.child <> B.child ⇒ = { (a,b) | (parent, a) in parentage , (parent, b) in parentage , not (a = b) }

  12. Queries as set comprehensions: finding siblings SELECT DISTINCT A.child, B.child FROM parentage A INNER JOIN parentage B ON A.parent = B.parent WHERE A.child <> B.child ⇒ = { (a,b) | (parent, a) in parentage , (parent, b) in parentage , not (a = b) }

  13. Recipe for a functional query language 1. Take a functional language 2. Add sets and set comprehensions 3. ... done?

  14. But can it go fast?

  15. Loop reordering { ... | x in EXPR1, y in EXPR2 } =? { ... | y in EXPR2, x in EXPR1 }

  16. Loop reordering { ... | x in EXPR1, y in EXPR2 } � = { ... | y in EXPR2, x in EXPR1 } 1. Side-effects 2. Nontermination

  17. Loop reordering { print x | x in { "hello" } , y in { 0,1 } } � = { print x | y in { 0,1 } , x in { "hello" } } 1. Side-effects 2. Nontermination

  18. Loop reordering { ... | x in {} , y in ∞ -loop } = ⇒ {} � = { ... | y in ∞ -loop, x in {} } ⇒ ∞ -loop = 1. Side-effects 2. Nontermination

  19. Recipe for a functional query language, v2 1. Take a pure, total functional language 2. Add sets and set comprehensions 3. Optimize!

  20. What have we gained? ◮ Can factor out repeated patterns with higher-order functions ◮ Sets are just ordinary values ◮ Sets, bags, lists: choose your container semantics!

  21. What have we gained? ◮ Can factor out repeated patterns with higher-order functions ◮ Sets are just ordinary values ◮ Sets, bags, lists: choose your container semantics! At what cost? ◮ Implementation complexity : GC, closures, nested sets, optimizing comprehensions... ◮ Re-inventing the wheel : persistence, transactions, replication...

  22. 1. What’s a functional query language? 2. From Datalog to Datafun 3. Incremental Datafun

  23. Parent Child Arathorn Aragorn Drogo Frodo E¨ arwen Galadriel Finarfin Galadriel . . . . . . Is E¨ arendil one of Aragorn’s ancestors?

  24. Datalog in a nutshell X is Z ’s ancestor if X is Z ’s parent. X is Z ’s ancestor if X is Y ’s parent and Y is Z ’s ancestor.

  25. Datalog in a nutshell ancestor( X , Z ) if parent( X , Z ). ancestor( X , Z ) if parent( X , Y ) and ancestor( Y , Z ).

  26. Datalog in a nutshell ancestor( X , Z ) :- parent( X , Z ). ancestor( X , Z ) :- parent( X , Y ), ancestor( Y , Z ).

  27. Datalog is deductive : it chases rules to their logical conclusions. Can we capture this feature functionally ?

  28. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros).

  29. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros).

  30. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros).

  31. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil). (new!)

  32. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil).

  33. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil). ancestor(E¨ arendil, Elros). (new!)

  34. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil). ancestor(E¨ arendil, Elros).

  35. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil). ancestor(E¨ arendil, Elros). ancestor(Idril, Elros). (new!)

  36. Procedure: 1. Pick a rule. 2. Find facts satisfying its premises. 3. Add its conclusion to the known facts. Rules: ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). Facts: parent(Idril, E¨ arendil). parent(E¨ arendil, Elros). ancestor(Idril, E¨ arendil). ancestor(E¨ arendil, Elros). ancestor(Idril, Elros).

  37. Repeatedly apply a set of rules until nothing changes

  38. Repeatedly apply a function until nothing changes

  39. Repeatedly apply a function until its output equals its input

  40. Repeatedly apply a function until its output equals its input i.e. it reaches a fixed point

  41. Repeatedly apply a function until its output equals its input i.e. it reaches a fixed point fix x = ... function of x ...

  42. // Datalog ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). // Datafun fix ancestor = parent ∪ { (x,z) | (x,y) in parent , (y,z) in ancestor }

  43. // Datalog ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). // Datafun fix ancestor = parent ∪ { (x,z) | (x,y) in parent , (y,z) in ancestor }

  44. // Datalog ancestor(X,Z) :- parent(X,Z). ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z). // Datafun fix ancestor = parent ∪ { (x,z) | (x,y) in parent , (y,z) in ancestor }

  45. Repeatedly applying: X �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in X } Where parent = { (Idril, E¨ arendil, Elros) } arendil), (E¨ Steps : ∅

  46. Repeatedly applying: X �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in X } Where parent = { (Idril, E¨ arendil, Elros) } arendil), (E¨ Steps : ∅ �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in ∅}

  47. Repeatedly applying: X �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in X } Where parent = { (Idril, E¨ arendil, Elros) } arendil), (E¨ Steps : ∅ �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in ∅} = parent

  48. Repeatedly applying: X �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in X } Where parent = { (Idril, E¨ arendil, Elros) } arendil), (E¨ Steps : ∅ �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in ∅} = parent �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in parent }

  49. Repeatedly applying: X �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in X } Where parent = { (Idril, E¨ arendil, Elros) } arendil), (E¨ Steps : ∅ �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in ∅} = parent �− → parent ∪ { (x,z) | (x,y) in parent, (y,z) in parent } = { (Idril, E¨ arendil), (E¨ arendil, Elros), (Idril, Elros) }

  50. But can it go fast?

  51. 1. What’s a functional query language? 2. From Datalog to Datafun 3. Incremental Datafun

  52. Three problems 1. View maintenance: How do we update a cached query efficiently after a mutation?

  53. Three problems 1. View maintenance: How do we update a cached query efficiently after a mutation? 2. Semina¨ ıve evaluation in Datalog: How do we avoid re-deducing facts we already know?

More recommend