Difgeological Spaces and Denotational Semantics for Difgerential Programming Ohad Kammar, Sam Staton, and Matthijs Vákár Domains 2018 Oxford 8 July 2018 Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming
What is difgerential programming? rather than symbolically or using fjnite differences. Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ PL in which all (some) constructs are difgerentiable… ▶ …and derivatives can be computed mechanically. ▶ Compute derivatives compositionally, using chain rule , ▶ AKA Automatic Difgerentiation.
Why study semantics of difgerential programming? Important: correctness non-trivial and good test-coverage hard to obtain; Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ Effjcient optimisation (in high dim); ▶ Effjcient integration/sampling (in high dim); ▶ Often combined with concurrency and probability ⇝
Why study semantics of difgerential programming? Non-Trivial: Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ How to difgerentiate through traditional language constructs? ▶ conditionals ▶ iteration and term recursion ▶ higher order functions ▶ probability, state and other efgects ▶ How to difgerentiate at structured types? ▶ inductive/recursive types ▶ refjnement types ▶ function types ▶ quotient types ▶ So need to move beyond traditional calculus!
Concrete example of higher-order difgerential prog Finding Shortest Arc Length -- Integrate :: ([0,1] => real) => real -- Differentiate :: ([0,1] => real) => [0,1] => real ArcLength :: ([0,1] => real) => [0,infty) ArcLength (f) = Integrate ( Sqrt (1 + Differentiate (f)^2)) Power :: (0,infty) => [0,1] => real Power (a)(t) = t^a -- Minimise :: ((0,infty) => real) => (0,infty) Minimal_a :: real Minimal_a = Minimise (\a -> ArcLength ( Power (a))) -- Should be 1 Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming
Is this not an artifjcial problem? Stan language: For specifying a smooth objective function and sampling from it / optimising it: while loops, recursive functions (for iterative approximations); pharmacokinetics); simplices, symmetric positive defjnite matrices); Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ conditionals (for custom combinations of objective functions), ▶ higher-order functions (ODE and algebraic solvers, used in ▶ certain refjnement types (constraints: intervals, unit vectors, ▶ fwd and rev mode AutoDifg through everything.
Kammar, Staton, and Vákár Calculus 101 Refresher Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ Traditional derivative (Jacobian) T x f : R n ⊸ R m of f : U → V at x , where U open in R n and V open in R m ▶ Best linear approximation: || f ( x + v ) − f ( x ) − T x f ( v ) || lim = 0 || v || v → 0
Categories for Smoothness? Kammar, Staton, and Vákár Difgeological Spaces and Denotational Semantics for Difgerential Programming Category Open : ▶ objects - opens U in some R n ; ▶ morphisms f : U → V - smooth ( C ∞ ) functions from U to V ; R m R n f U V R Category Mfd of fjn. dim. manifolds and smooth maps: ▶ Adds surfaces (certain refjnements); ▶ Whitney Embedding: idempotent completion of Open ; ▶ Great! If only there were more...
Several Categories of Smooth Maps types, dependent types; Kammar, Staton, and Vákár limits and coproducts, colimits of open covers; Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ Many full subcategories: ✲ Open ⊆ ✲ Mfd EuclSp ⊆ ∩ ∩ ❄ ❄ ✲ FreMfd FreSp ⊆ ∩ ∩ ❄ ❄ ✲ FroSp ✲ Diffeo ⊆ ✲ Sh ( Open ) ⊆ ✲ [ Open op , Set ] ConvSp ⊆ ⊆ ▶ Diffeo Grothendieck quasi-topos and well-pointed; ▶ Interprets tuples, function types, variant types, (co)inductive ▶ Conservative extension: embeddings into Diffeo preserve all
Difgeological spaces [Souriau] such that the plots are closed under: Kammar, Staton, and Vákár . gluing of compatible families along open covers precomposition with a smooth constant functions ; Difgeological Spaces and Denotational Semantics for Difgerential Programming Objects consists of: ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U
Difgeological spaces [Souriau] such that the plots are closed under: Kammar, Staton, and Vákár . gluing of compatible families along open covers Objects precomposition with a smooth Difgeological Spaces and Denotational Semantics for Difgerential Programming consists of: ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U ▶ constant functions c ; α ( r )= c �− − − − → c
Difgeological spaces [Souriau] such that the plots are closed under: Kammar, Staton, and Vákár . gluing of compatible families along open covers Objects Difgeological Spaces and Denotational Semantics for Difgerential Programming consists of: ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U ▶ constant functions c ; ▶ precomposition with a smooth ϕ : U → V ϕ α �− → �− →
Difgeological spaces [Souriau] consists of: Kammar, Staton, and Vákár Objects such that the plots are closed under: Difgeological Spaces and Denotational Semantics for Difgerential Programming ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U ▶ constant functions c ; ▶ precomposition with a smooth ϕ : U → V ▶ gluing of compatible families along open covers U . case { U .α U | U ∈U} �− − − − − − − − − − − →
Difgeological spaces [Souriau] such that...3 axioms... Kammar, Staton, and Vákár Example: Traditional Spaces Objects Difgeological Spaces and Denotational Semantics for Difgerential Programming consists of: ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U Morphisms f : X → Y Functions f : | X | → | Y | such that: α ∈ S X = ⇒ f ◦ α ∈ S Y ▶ For manifold M , defjne (Yoneda) plots S U M := Mfd ( U, M ) .
Categorical structure such that...3 axioms... Kammar, Staton, and Vákár Objects Products Difgeological Spaces and Denotational Semantics for Difgerential Programming consists of: ( ) A difgeological space X = | X | , S X ▶ a carrier set | X | ; X ⊆ | X | U for all U ∈ Open ▶ a set of plots S U { )� } S U � α ∈ S U X , β ∈ S U X × Y := ( r �→ α ( r ) , β ( r ) � Y
Categorical structure Function spaces Kammar, Staton, and Vákár (co)inductive types! Coproducts, limits, colimits, More structure Difgeological Spaces and Denotational Semantics for Difgerential Programming Morphisms f : X → Y Functions f : | X | → | Y | such that: α ∈ S X = ⇒ f ◦ α ∈ S Y � � Y X � � := Diffeo ( X, Y ) � � � � � Y X � { } � S U Y X := f : U → � uncurry f ∈ Diffeo ( U × X, Y ) � � � � NB: The exponential X U is the space of U -plots.
Categorical structure Function spaces Kammar, Staton, and Vákár More structure Difgeological Spaces and Denotational Semantics for Difgerential Programming Morphisms f : X → Y Functions f : | X | → | Y | such that: α ∈ S X = ⇒ f ◦ α ∈ S Y � � Y X � � := Diffeo ( X, Y ) � � � � � Y X � { } � S U Y X := f : U → � uncurry f ∈ Diffeo ( U × X, Y ) � � � � NB: The exponential X U is the space of U -plots. Coproducts, limits, colimits, . . . (co)inductive types!
Our example revisited Finding Shortest Arc Length Kammar, Staton, and Vákár of two arguments to smooth functions, so it is smooth. -- Should be 1 Minimal_a = Minimise (\a -> ArcLength ( Power (a))) Minimal_a :: real -- Minimise :: ((0,infty) => real) => (0,infty) Power (a)(t) = t^a Power :: (0,infty) => [0,1] => real ArcLength (f) = Integrate ( Sqrt (1 + Differentiate (f)^2)) ArcLength :: ([0,1] => real) => [0,infty) -- Differentiate :: ([0,1] => real) => [0,1] => real -- Integrate :: ([0,1] => real) => real Difgeological Spaces and Denotational Semantics for Difgerential Programming ▶ Uncurry to see Power is smooth; ▶ Integrate sends smooth functions
Difgeological Spaces and Denotational Semantics for Difgerential Programming D-Topology Kammar, Staton, and Vákár Diffeo -morphisms are continuous! ▶ Adjunction with topological spaces ( M ∈ Top , X ∈ Diffeo ): S U Diffeo M := Top ( U, M ) { � } � ∀ U ∈ Open .α ∈ S U X , α − 1 [ X ] ∈ O U := O Top X B ⊆ X � Top Top Diffeo ⊥ Diffeo = Set ▶ Generalises Euclidean topology.
Recommend
More recommend
