the tension between convenience and performance in
play

The tension between convenience and performance in automatic - PowerPoint PPT Presentation

Nov 01, 2023 •215 likes •2.07k views

The tension between convenience and performance in automatic differentiation Jeffrey Mark Siskind, qobi@purdue.edu NIPS 2016 Workshop on The Future of Gradient-Based Machine Learning Software Saturday 10 December 2016 Joint work with Barak

  1. Dynamic Overloading: SCMUTILS (define-structure bundle primal tangent) (define (primal p) (if (bundle? p) (bundle-primal p) p)) (define (tangent p) (if (bundle? p) (bundle-tangent p) 0)) (define + (let ((+ +)) (lambda (x1 x2) (make-bundle (+ (primal x1) (primal x2)) (+ (tangent x1) (tangent x2)))))) (define * (let ((+ +) (* *)) (lambda (x1 x2) (make-bundle (* (primal x1) (primal x2)) (+ (* (primal x1) (tangent x2)) (* (tangent x1) (primal x2))))))) (define ((derivative f) x) (tangent (f (make-bundle x 1)))) (define (f x) (* 2 (* x (* x x)))) (derivative f) (derivative (derivative f)) (derivative (lambda (x) ... (derivative (lambda (y) ... ) ... ) ... ) ... ) Convenient but slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 10 / 45

  2. Dynamic Overloading: SCMUTILS (define-structure bundle primal tangent) (define (primal p) (if (bundle? p) (bundle-primal p) p)) (define (tangent p) (if (bundle? p) (bundle-tangent p) 0)) (define + (let ((+ +)) (lambda (x1 x2) (make-bundle (+ (primal x1) (primal x2)) (+ (tangent x1) (tangent x2)))))) (define * (let ((+ +) (* *)) (lambda (x1 x2) (make-bundle (* (primal x1) (primal x2)) (+ (* (primal x1) (tangent x2)) (* (tangent x1) (primal x2))))))) (define ((derivative f) x) (tangent (f (make-bundle x 1)))) (define (f x) (* 2 (* x (* x x)))) (derivative f) (derivative (derivative f)) (derivative (lambda (x) ... (derivative (lambda (y) ... ) ... ) ... ) ... ) Convenient but slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 10 / 45

  3. Dynamic Overloading: SCMUTILS (define-structure bundle primal tangent) (define (primal p) (if (bundle? p) (bundle-primal p) p)) (define (tangent p) (if (bundle? p) (bundle-tangent p) 0)) (define + (let ((+ +)) (lambda (x1 x2) (make-bundle (+ (primal x1) (primal x2)) (+ (tangent x1) (tangent x2)))))) (define * (let ((+ +) (* *)) (lambda (x1 x2) (make-bundle (* (primal x1) (primal x2)) (+ (* (primal x1) (tangent x2)) (* (tangent x1) (primal x2))))))) (define ((derivative f) x) (tangent (f (make-bundle x 1)))) (define (f x) (* 2 (* x (* x x)))) (derivative f) (derivative (derivative f)) (derivative (lambda (x) ... (derivative (lambda (y) ... ) ... ) ... ) ... ) Convenient but slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 10 / 45

  4. Dynamic Overloading: SCMUTILS (define-structure bundle primal tangent) (define (primal p) (if (bundle? p) (bundle-primal p) p)) (define (tangent p) (if (bundle? p) (bundle-tangent p) 0)) (define ((derivative f) x) (fluid-let ((+ (lambda (x1 x2) (make-bundle (+ (primal x1) (primal x2)) (+ (tangent x1) (tangent x2))))) (* (lambda (x1 x2) (make-bundle (* (primal x1) (primal x2)) (+ (* (primal x1) (tangent x2)) (* (tangent x1) (primal x2))))))) (tangent (f (make-bundle x 1))))) (define (f x) (* 2 (* x (* x x)))) (derivative f) (derivative (derivative f)) (derivative (lambda (x) ... (derivative (lambda (y) ... ) ... ) ... ) ... ) Convenient but slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 10 / 45

  5. Dynamic Overloading: SCMUTILS (define-structure bundle primal tangent) (define (primal p) (if (bundle? p) (bundle-primal p) p)) (define (tangent p) (if (bundle? p) (bundle-tangent p) 0)) (define ((derivative f) x) (fluid-let ((+ (lambda (x1 x2) (make-bundle (+ (primal x1) (primal x2)) (+ (tangent x1) (tangent x2))))) (* (lambda (x1 x2) (make-bundle (* (primal x1) (primal x2)) (+ (* (primal x1) (tangent x2)) (* (tangent x1) (primal x2))))))) (tangent (f (make-bundle x 1))))) (define (f x) (* 2 (* x (* x x)))) (derivative f) (derivative (derivative f)) (derivative (lambda (x) ... (derivative (lambda (y) ... ) ... ) ... ) ... ) Convenient but slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 10 / 45

  6. Preprocessor: ADIFOR and T APENADE function f(x) double precision x, f f = 2.0d0*x*x*x end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  7. Preprocessor: ADIFOR and T APENADE function f(x) double precision x, f f = 2.0d0*x*x*x end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  8. Preprocessor: ADIFOR and T APENADE function f(x) double precision x, f f = 2.0d0*x*x*x end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  9. Preprocessor: ADIFOR and T APENADE function f(x) double precision x, f f = 2.0d0*x*x*x end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  10. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f f = 2.0d0*x*x*x end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  11. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  12. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  13. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) double precision x, gx, gf, gresult gf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  14. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  15. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx end function ggf(x, gx, gx, ggx, gresult, ggresult, gresult) double precision x, gx, gx, ggx, ggf, gresult, gresult, ggresult ggf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx gresult = 6.0d0*x*x*gx ggresult = 6.0d0*x*x*ggx+12.0d0*x*gx*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  16. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx end function ggf(x, gx, gx, ggx, gresult, ggresult, gresult) double precision x, gx, gx, ggx, ggf, gresult, gresult, ggresult ggf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx gresult = 6.0d0*x*x*gx ggresult = 6.0d0*x*x*ggx+12.0d0*x*gx*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  17. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx end function ggf(x, gx, gx, ggx, gresult, ggresult, gresult) double precision x, gx, gx, ggx, ggf, gresult, gresult, ggresult ggf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx gresult = 6.0d0*x*x*gx ggresult = 6.0d0*x*x*ggx+12.0d0*x*gx*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  18. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx end function ggf(x, gx, gx, ggx, gresult, ggresult, gresult) double precision x, gx, gx, ggx, ggf, gresult, gresult, ggresult ggf = 2.0d0*x*x*x gresult = 6.0d0*x*x*gx gresult = 6.0d0*x*x*gx ggresult = 6.0d0*x*x*ggx+12.0d0*x*gx*gx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  19. Preprocessor: ADIFOR and T APENADE function f(x) AD_TOP = f double precision x, f AD_IVARS = x f = 2.0d0*x*x*x AD_DVARS = f end function gf(x, gx, gresult) AD_TOP = gf double precision x, gx, gf, gresult AD_IVARS = x, gx gf = 2.0d0*x*x*x AD_DVARS = gf, gresult gresult = 6.0d0*x*x*gx AD_PREFIX = h end function hgf(x, hx, gx, hgx, gresult, hgresult, hresult) double precision x, hx, gx, hgx, hgf, hresult, gresult, hgresult hgf = 2.0d0*x*x*x hresult = 6.0d0*x*x*hx gresult = 6.0d0*x*x*gx hgresult = 6.0d0*x*x*hgx+12.0d0*x*gx*hx end Fast but inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 11 / 45

  20. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  21. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  22. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  23. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  24. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  25. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  26. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  27. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  28. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  29. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  30. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  31. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  32. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  33. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  34. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... template <typename T> T f(T x) {return 2*x*x*x;} T x; Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  35. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... template <typename T> T f(T x) {return 2*x*x*x;} T x; Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  36. Static Overloading: FADBAD ++ double f(double x) {return 2*x*x*x;} double x; ... f(x) ... F<double> f(F<double> x) {return 2*x*x*x;} F<double> x; x.diff(0, 1); ... f(x).d(0) ... F<F<double> > f(F<F<double> > x) {return 2*x*x*x;} F<F<double> > x; x.diff(0, 1); x.diff(0, 1).diff(0,1); ... f(x).d(0).d(0) ... template <typename T> T f(T x) {return 2*x*x*x;} T x; Slow and inconvenient Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 12 / 45

  37. Implementation of Reverse Mode by Overloading (define-structure tape value operation argments) (set! original+ +) (define (+ x y) (if (tape? x) (tape (+ (value x) (value y)) ’+ (list (arguments x) (arguments y))) (original+ x y))) Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 13 / 45

  38. Reverse Mode x 1 = f 1 ( x 0 ) ⋮ x n = f n ( x n − 1 ) x n − 1 = J ( f n )( x n − 1 ) × ` ` x n ⋮ x 0 = J ( f 1 )( x 0 ) × ` ` x 1 Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 14 / 45

  39. Implementation of Reverse Mode by Transformation—I subroutine sqr(x, y) y = x * x end subroutine l2(x1, y1, x2, y2, r) t1 = x2 - x1 sqr(t1, t2) t3 = y2 - y1 sqr(t3, t4) r = t2 + t4 end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 15 / 45

  40. Implementation of Reverse Mode by Transformation—II subroutine sqrf(xp, yp) push(xp) yp = xp * xp end subroutine l2f(x1p, y1p, x2p, y2p, rp) t1p = x2p - x1p sqr(t1p, t2p) t3p = y2p - y1p sqr(t3p, t4p) rp = t2p + t4p end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 16 / 45

  41. Implementation of Reverse Mode by Transformation—III subroutine sqrr(xc, yc) pop(xp) xc = yc * xp xc += xp * yc end subroutine l2r(x1c, y1c, x2c, y2c, rc) t2c = rc t4c = rc sqrr(t3c, t4c) y2c = -t3c y1c = t3c sqrr(t1c, t2c) x2c = -t1c x1c = t1c end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 17 / 45

  42. Key Idea Migrate reflective source-to-source transformation from run time to compile time with abstract interpretation Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 18 / 45

  43. Traditional AD by Source-to-Source Transformation Preprocessor at Compile Time function g(x) return x+1 end function f(x) return 2*g(x) end ... derivative(f, 3) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 19 / 45

  44. Traditional AD by Source-to-Source Transformation Preprocessor at Compile Time function g(x) return x+1 end function f(x) return 2*g(x) end local y, y_tangent = f_forward(3, 1) ... y_tangent ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 19 / 45

  45. Traditional AD by Source-to-Source Transformation Preprocessor at Compile Time function g(x) return x+1 end function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end local y, y_tangent = f_forward(3, 1) ... y_tangent ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 19 / 45

  46. Traditional AD by Source-to-Source Transformation Preprocessor at Compile Time function g_forward(x, x_tangent) local y, y_tangent = x, x_tangent return x+1, x_tangent end function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end local y, y_tangent = f_forward(3, 1) ... y_tangent ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 19 / 45

  47. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  48. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  49. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  50. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  51. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  52. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" compile("function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end") -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  53. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" compile("function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end") ==> f_forward -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  54. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" compile("function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end") ==> f_forward called_by(f) -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  55. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" compile("function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end") ==> f_forward called_by(f) ==> {g} -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  56. Source-to-Source Transformation at Run Time Reflection function f(x) return 2*g(x) end code(f) ==> "function f(x) return 2*g(x) end" transform("function f(x) return 2*g(x) end") ==> "function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end" compile("function f_forward(x, x_tangent) local y, y_tangent = g_forward(x, x_tangent) return return 2*y, 2*y_tangent end") ==> f_forward called_by(f) ==> {g} function derivative(f, x) for g in called_by(f) do compile(transform(code(g))) end local y, y_tangent = compile(transform(code(f)))(x, 1) return y_tangent end -- Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 20 / 45

  57. But How Can We Make This Efficient? while not converged() do x = x-eta*derivative(f, x) end Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 21 / 45

  58. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... add(x, y) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  59. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = DOUBLE , y = DOUBLE ... add(x, y) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  60. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = DOUBLE , y = DOUBLE ... add( DOUBLE , DOUBLE ) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  61. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = DOUBLE , y = DOUBLE ... add_1( DOUBLE , DOUBLE ) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  62. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) if DOUBLE =="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = DOUBLE , y = DOUBLE ... add_1( DOUBLE , DOUBLE ) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  63. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) if false then return vector_add(x, y) else return scalar_add(x, y) end end local x = DOUBLE , y = DOUBLE ... add_1( DOUBLE , DOUBLE ) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  64. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) return scalar_add(x, y) end local x = DOUBLE , y = DOUBLE ... add_1( DOUBLE , DOUBLE ) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  65. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) return scalar_add(x, y) end local x = 3, y = 4 ... scalar_add(x, y) ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  66. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_1( DOUBLE , DOUBLE ) return scalar_add(x, y) end local x = 3, y = 4 ... x+y ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  67. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  68. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add( ARRAY , ARRAY ) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  69. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_2( ARRAY , ARRAY ) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add_2( ARRAY , ARRAY ) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  70. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_2( ARRAY , ARRAY ) if ARRAY =="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add_2( ARRAY , ARRAY ) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  71. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_2( ARRAY , ARRAY ) if true then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add_2( ARRAY , ARRAY ) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  72. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add_2( ARRAY , ARRAY ) return vector_add(x, y) end local x = 3, y = 4 ... x+y ... local x = ARRAY , y = ARRAY ... add_2( ARRAY , ARRAY ) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  73. Abstract Interpretation aka (Polyvariant) Flow Analysis function scalar_add(x, y) return x+y end function vector_add(x, y) local n = x:size(1) local z = torch.Tensor(n) for i = 1, n do z[i] = x[i]+y[i] end return z end function add(x, y) if x:type()=="torch.Tensor" then return vector_add(x, y) else return scalar_add(x, y) end end local x = 3, y = 4 ... x+y ... local x = torch.Tensor(5):zeros(), y = torch.Tensor(5):zeros() ... vector_add(x, y) ... Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 22 / 45

  74. A Single Powerful Optimization {x = e1, y = e2}.x Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 23 / 45

  75. A Single Powerful Optimization {x = e1, y = e2}.x ↝ e1 Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 23 / 45

  76. A Single Powerful Optimization {x = e1, y = e2}.x ↝ e1 ▸ can eliminate storage allocation Siskind (Purdue) Tension in AD NIPS 2016 WS 10 December 2016 23 / 45

Recommend

Belt Tension JellyBox HowTo: Belt Tension In this video, we set up the correct belt tension on X

Belt Tension JellyBox HowTo: Belt Tension In this video, we set up the correct belt tension on X

Belt Tension JellyBox HowTo: Belt Tension In this video, we set up the correct belt tension on X and Y axes. Belt Tension? If the belts are too loose, they may skip steps or even slip off idlers. You may also observe more 'ringing' artifacts

250 views • 4 slides
Relax! Improve Your Playing by Releasing Tension Dr. Liz Aleksander and Andrew Morency

Relax! Improve Your Playing by Releasing Tension Dr. Liz Aleksander and Andrew Morency

Relax! Improve Your Playing by Releasing Tension Dr. Liz Aleksander and Andrew Morency University of Tennessee at Martin Tension Goal when playing: minimize tension Tension radiates to other areas Physical and Mental Tension

1.36k views • 28 slides
Surface Tension 4-1. The key thing about surface tension is that it is a tension. The force is

Surface Tension 4-1. The key thing about surface tension is that it is a tension. The force is

1. We are almost done with the complete code. However, a few important steps remain. DNS of Multiphase Flows Simple Front Tracking Direct Numerical Simulations of Multiphase Flows-6 Surface Tension, Unequal Viscosities, and Higher

1.04k views • 9 slides
The stress in an axially loaded tension member is : f

The stress in an axially loaded tension member is : f

Analysis and Design of Tension Members

467 views • 36 slides
Direct Tension Test The Direct Tension Test provides a information on facture and strain

Direct Tension Test The Direct Tension Test provides a information on facture and strain

Direct Tension Test The Direct Tension Test provides a information on facture and strain tolerance of the binder. New low temperature binder Spec. Thermal stress compared to strength Strain at Failure vs Temperature 20 10/20 19 18

196 views • 7 slides
Overview Overview Modeling Tension and Modeling Tension and Goal Relaxation for Computer

Overview Overview Modeling Tension and Modeling Tension and Goal Relaxation for Computer

Overview Overview Modeling Tension and Modeling Tension and Goal Relaxation for Computer Relaxation for Computer Expressive, realistic character animation Animation Animation Approach Identify characteristics that make human

346 views • 8 slides
Clinical Supervision in an Educational Setting: AN UNAVOIDABLE TENSION BETWEEN HOLDING AND

Clinical Supervision in an Educational Setting: AN UNAVOIDABLE TENSION BETWEEN HOLDING AND

Clinical Supervision in an Educational Setting: AN UNAVOIDABLE TENSION BETWEEN HOLDING AND ASSESSING Dr Pl Dominic McCann Background After 2.5 years working in supervision at ACAP , the main tension noted is that of mentor vs

411 views • 20 slides
Towards Relay: a New IR for Machine Learning Frameworks Jared Roesch , Steven Lyubomirsky, Logan

Towards Relay: a New IR for Machine Learning Frameworks Jared Roesch , Steven Lyubomirsky, Logan

Towards Relay: a New IR for Machine Learning Frameworks Jared Roesch , Steven Lyubomirsky, Logan Weber, Josh Pollock, Marisa Kirisame, Tianqi Chen, Zachary Tatlock 2 Tension between performance and flexibility 3 Tension between

947 views • 67 slides
LAN Performance Improvements October 20, 2015 LAN Performance Improvements This information

LAN Performance Improvements October 20, 2015 LAN Performance Improvements This information

LAN Performance Improvements October 20, 2015 LAN Performance Improvements This information contains forward looking statements and is provided solely for your convenience. While the information herein is based on our current best estimates,

409 views • 15 slides
with Cold Electronics Wenqiang Gu, Xin Qian BNL 1 Wire tension measurement T = 4 f 2 L

with Cold Electronics Wenqiang Gu, Xin Qian BNL 1 Wire tension measurement T = 4 f 2 L

Exploration of Wire Tension with Cold Electronics Wenqiang Gu, Xin Qian BNL 1 Wire tension measurement T = 4 f 2 L 2 , f: fundamental frequency Laser method [1] Time-consuming expensive, complicated laser positioning system

508 views • 28 slides
Pension Tension: How big is this problem? How did we get here? What is being --and can be --

Pension Tension: How big is this problem? How did we get here? What is being --and can be --

Pension Tension: How big is this problem? How did we get here? What is being --and can be -- done about it? Citizens League Mind Opener Series 4.17.18 Mark Haveman, Executive Director Minnesota Center for Fiscal Excellence

256 views • 11 slides
Convenience and Petroleum Retailing Convenience and Petroleum Retailing Industry Update: Industry

Convenience and Petroleum Retailing Convenience and Petroleum Retailing Industry Update: Industry

Convenience and Petroleum Retailing Convenience and Petroleum Retailing Industry Update: Industry Update: Facts, Figures, and Best Practices to Help Marketers and Dealers Succeed February 16 2010 February 16, 2010 T d Today Ab About

1.1k views • 65 slides
Azrieli Group LTD Financial statements Q2 2010 August 29 2010 CONVENIENCE TRANSLATION FROM HEBREW

Azrieli Group LTD Financial statements Q2 2010 August 29 2010 CONVENIENCE TRANSLATION FROM HEBREW

Azrieli Group LTD Financial statements Q2 2010 August 29 2010 CONVENIENCE TRANSLATION FROM HEBREW - Important Notice Set out below for your convenience is a convenience translation into English of the presentation with respect of the financial

467 views • 17 slides
Effects of a Time-Varying String Tension &amp; String Repulsion in Momentum Space Tau-dependent

Effects of a Time-Varying String Tension &amp; String Repulsion in Momentum Space Tau-dependent

Effects of a Time-Varying String Tension &amp; String Repulsion in Momentum Space Tau-dependent string tension with N. Hunt-Smith Publication in preparation Physics motivations? A primitive model Results: Strangeness-pT correlations String

563 views • 31 slides
TENSION CALIBRATION SERVICE OV E R V I E W About Us Anchor-line Tension Calibration Service

TENSION CALIBRATION SERVICE OV E R V I E W About Us Anchor-line Tension Calibration Service

ANCHOR-LINE w w w . i n c o n t r o l . n o TENSION CALIBRATION SERVICE OV E R V I E W About Us Anchor-line Tension Calibration Service InControl AS Bergen, Norway www.incontrol.no About Us InControl AS descends from the Instrumentation

393 views • 10 slides
Dining on campus isnt one-size-fits-all Quality Convenience Flexibility

Dining on campus isnt one-size-fits-all Quality Convenience Flexibility

Dining on campus isnt one-size-fits-all Quality Convenience Flexibility Budget-friendly Residential Dining Caesar Rodney Fresh Food Company Russell Dining Pencader Dining Convenience Stores

511 views • 12 slides
Results For the half year ended 31 March 2017 HIGHLIGHTS CONVENIENCE FOODS UK &amp; IRELAND

Results For the half year ended 31 March 2017 HIGHLIGHTS CONVENIENCE FOODS UK &amp; IRELAND

Results For the half year ended 31 March 2017 HIGHLIGHTS CONVENIENCE FOODS UK &amp; IRELAND Strong growth Significant project activity to enable new commercial wins Raw material and labour inflation fully mitigated CONVENIENCE

833 views • 34 slides
Convenience Trends 2016 September 27, 2016 Chelsea Gross | chelsea.gross@retailnetgroup.com

Convenience Trends 2016 September 27, 2016 Chelsea Gross | chelsea.gross@retailnetgroup.com

CDA Convenience Trends 2016 September 27, 2016 Chelsea Gross | chelsea.gross@retailnetgroup.com Todays Session Convenience Learning from Intro Forecasts Summary Store Initiatives the Disruptors Real-time Retail Channel View

767 views • 33 slides
Policing Social Tension: The Long View Cumberland Lodge 2016 Dr Timothy Brain Alternative: the

Policing Social Tension: The Long View Cumberland Lodge 2016 Dr Timothy Brain Alternative: the

Policing Social Tension: The Long View Cumberland Lodge 2016 Dr Timothy Brain Alternative: the British Policing model has historically implicitly recognised the use of extreme force in extreme circumstances Social Tension? Stress

241 views • 22 slides
twenty steel construction: columns &amp; tension members Steel Columns &amp; Tension 1

twenty steel construction: columns &amp; tension members Steel Columns &amp; Tension 1

A RCHITECTURAL S TRUCTURES : F ORM, B EHAVIOR, AND D ESIGN A RCH 331 Cor-Ten Steel Sculpture By Richard Serra Museum of Modern Art Fort Worth, TX (AISC - Steel Structures of the Everyday) D R. A NNE N ICHOLS S PRING 2018 lecture twenty steel

713 views • 23 slides
Statewide Contract Maintenance Repair Operations Statewide Convenience Contract Webinar June

Statewide Contract Maintenance Repair Operations Statewide Convenience Contract Webinar June

MRO- HD Supply Facilities Maintenance Convenience Statewide Contract Maintenance Repair Operations Statewide Convenience Contract Webinar June 16, 2010 Georgia State Purchasing Division http://statepurchasing.doas.georgia.gov SPD-CP031b

561 views • 41 slides
Enabling Practical SDN Security Applications with OFX (The O pen F low e X tension Framework)

Enabling Practical SDN Security Applications with OFX (The O pen F low e X tension Framework)

Enabling Practical SDN Security Applications with OFX (The O pen F low e X tension Framework) John Sonchack, Adam J. Aviv, Eric Keller, and Jonathan M. Smith Outline Introduction Overview of OFX Using OFX Benchmarks 2 Basic Networking:

604 views • 35 slides
Introduction Tension between Corporate Law and Family Law came to a head this year with the

Introduction Tension between Corporate Law and Family Law came to a head this year with the

Introduction Tension between Corporate Law and Family Law came to a head this year with the landmark case of Prest v Petrodel. It highlighted the difficulties in identifying and allocating assets necessary for the process of

687 views • 26 slides
The Tension of the Advent Season At Christmas there is so much about which to feel happy and

The Tension of the Advent Season At Christmas there is so much about which to feel happy and

The Tension of the Advent Season At Christmas there is so much about which to feel happy and joyful. But Im not finding it that way! There are ghosts lurking behind the scenes stealing every bit of hope and joy that I have. Lifes issues

366 views • 18 slides

More recommend