Adjoint code development and optimization using automatic - PowerPoint PPT Presentation

Adjoint code development and optimization using automatic differentiation (AD) Praveen. C Computational and Theoretical Fluid Dynamics Division National Aerospace Laboratories Bangalore - 560 037 CTFD Seminar, March 31, 2006 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 1 / 56

Outline Mathematical formulation 1 Computing gradients 2 Quasi 1-D flow 3 Gradient smoothing 4 Quasi 1-D optimization: Pressure matching 5 2-D adjoint Euler solver 6 Future work 7 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 2 / 56

Outline Mathematical formulation 1 Computing gradients 2 Quasi 1-D flow 3 Gradient smoothing 4 Quasi 1-D optimization: Pressure matching 5 2-D adjoint Euler solver 6 Future work 7 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 3 / 56

Mathematical formulation Constrained minimization problem min α J ( α, u ) subject to R ( α, u ) = 0 Find δα such that δ J < 0 ∂ J ∂αδα + ∂ J δ J = ∂ u δ u ∂αδα + ∂ J ∂ J ∂ u = ∂αδα ∂ u = G δα Steepest descent δα = − ǫ G ⊤ δ J = − ǫ GG ⊤ = − ǫ � G � 2 < 0 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 4 / 56

Mathematical formulation Constrained minimization problem min α J ( α, u ) subject to R ( α, u ) = 0 Find δα such that δ J < 0 ∂ J ∂αδα + ∂ J δ J = ∂ u δ u ∂αδα + ∂ J ∂ J ∂ u = ∂αδα ∂ u = G δα Steepest descent δα = − ǫ G ⊤ δ J = − ǫ GG ⊤ = − ǫ � G � 2 < 0 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 5 / 56

Sensitivity approach Linearized state equation ∂ R ∂α δα + ∂ R ∂ u δ u = 0 or ∂ R ∂α = − ∂ R ∂ u ∂ u ∂α Solve sensitivity equation iteratively ∂ ∂ u ∂α + ∂ R ∂ u ∂α = − ∂ R ∂ t ∂ u ∂α Same eigenvalues as original system Costly if number of α are large Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 6 / 56

Adjoint approach We have δ J = ∂ J ∂αδα + ∂ J ∂ R ∂α δα + ∂ R ∂ u δ u and ∂ u δ u = 0 Introduce a new unknown v ∂ J ∂αδα + ∂ J � ∂ R ∂α δα + ∂ R � ∂ u δ u + v ⊤ δ J = ∂ u δ u � ∂ J ∂α + v ⊤ ∂ R � � ∂ J ∂ u + v ⊤ ∂ R � = δα + δ u ∂α ∂ u Adjoint equation � ⊤ � ⊤ � ∂ R � ∂ J v = − ∂ u ∂ u Iterative solution � ⊤ � ⊤ � ∂ R � ∂ J ∂ v ∂ t + v = − ∂ u ∂ u Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 7 / 56

Continuous vs Discrete Continuous approach: Start with governing PDE Derive adjoint PDE and boundary conditions Discretize adjoint PDE and solve Must be re-derived whenever cost function changes Not all cost functions admissible Discrete approach: Start with discrete approximation like FVM Derive discrete adjoint equations Solve discrete adjoint equations True sensitivity of discrete solution Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 8 / 56

Outline Mathematical formulation 1 Computing gradients 2 Quasi 1-D flow 3 Gradient smoothing 4 Quasi 1-D optimization: Pressure matching 5 2-D adjoint Euler solver 6 Future work 7 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 9 / 56

Techniques for computing gradients Hand differentiation Finite difference method Complex variable method Automatic differentiation Computer code to compute some function Chain rule of differentiation Generates a code to compute derivatives ADIFOR, ADOLC, ODYSEE, TAMC, TAF , TAPENADE Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 10 / 56

Differentiation: Example A simple example f = ( xy + sin x + 4 )( 3 y 2 + 6 ) Computer code, f = t 10 t 1 = x t 2 = y t 3 = t 1 t 2 t 4 = sin t 1 t 5 = t 3 + t 4 t 6 = t 5 + 4 t 2 = t 7 2 t 8 = 3 t 7 = t 8 + 6 t 9 t 10 = t 6 t 9 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 11 / 56

Automatic Differentiation: F77 code subroutine costfunc(x, y, f) t1 = x t2 = y t3 = t1*t2 t4 = sin(t1) t5 = t3 + t4 t6 = t5 + 4 t7 = t2** 2 t8 = 3.0 *t7 t9 = t8 + 6.0 t10 = t6*t9 f = t10 end Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 12 / 56

Differentiation: Direct mode Some definitions I k := { i : i < k and t k depends explicitly on t i } t i := ∂ t i t i , k := ∂ t i ˙ ∂α, ∂ t k Apply chain rule of differentiation t k = ∂ t k ∂ t i ∂ t k ˙ � � ˙ ∂α = = k = 1 , 2 , . . . , 9 , 10 t i t k , i ∂α ∂ t i i ∈ I k i ∈ I k α = x , or y Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 13 / 56

Differentiation: Direct mode Apply chain rule of differentiation ˙ ˙ t 1 = x t 1 = x ˙ ˙ t 2 = y t 2 = y ˙ ˙ t 1 t 2 + t 1 ˙ t 3 = t 1 t 2 t 3 = t 2 ˙ cos ( t 1 )˙ t 4 = sin ( t 1 ) t 4 = t 1 ˙ ˙ t 3 + ˙ = t 3 + t 4 = t 5 t 5 t 4 ˙ ˙ t 6 = t 5 + 4 t 6 = t 5 ˙ 2 t 2 ˙ t 2 t 7 = t 7 = t 2 2 ˙ 3 ˙ t 8 = 3 t 7 t 8 = t 7 ˙ ˙ t 9 = t 8 + 6 t 9 = t 8 ˙ ˙ t 6 t 9 + t 6 ˙ t 10 = t 6 t 9 t 10 = t 9 y = 0, ˙ t 10 = ∂ f y = 1, ˙ t 10 = ∂ f x = 1, ˙ ˙ ∂ x and ˙ x = 0, ˙ ∂ y tapenade -d -vars "x y" -outvars f costfunc.f Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 14 / 56

Automatic Differentiation: Direct mode SUBROUTINE COSTFUNC_D(x, xd, y, yd, f, fd) t1d = xd t1 = x t2d = yd t2 = y t3d = t1d*t2 + t1*t2d t3 = t1*t2 t4d = t1d*COS(t1) t4 = SIN(t1) t5d = t3d + t4d t5 = t3 + t4 t6d = t5d t6 = t5 + 4 t7d = 2*t2*t2d t7 = t2**2 t8d = 3.0*t7d t8 = 3.0*t7 t9d = t8d t9 = t8 + 6.0 t10d = t6d*t9 + t6*t9d t10 = t6*t9 fd = t10d f = t10 END Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 15 / 56

Differentiation: Reverse mode Some definitions J k := { i : i > k and t i depends explicitly on t k } t i := ∂ f = ∂ t 10 t i , k := ∂ t i ¯ , ∂ t i ∂ t i ∂ t k Apply chain rule of differentiation in reverse t k = ∂ t 10 ∂ t 10 ∂ t i ¯ � � ¯ = = t i t i , k k = 10 , 9 , . . . , 2 , 1 ∂ t k ∂ t i ∂ t k i ∈ J k i ∈ J k Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 16 / 56

Differentiation: Reverse mode Apply chain rule of differentiation in reverse ¯ t 1 = x t 10 = 1 ¯ ¯ t 2 = y t 9 = t 10 t 10 , 9 = t 6 ¯ ¯ t 3 = t 1 t 2 t 8 = t 9 t 9 , 8 = t 6 ¯ ¯ = sin ( t 1 ) = = t 4 t 7 t 8 t 8 , 7 3 t 6 ¯ ¯ t 5 = t 3 + t 4 t 6 = t 10 t 10 , 6 = t 9 ¯ ¯ t 6 = t 5 + 4 t 5 = t 6 t 6 , 5 = t 9 ¯ ¯ t 2 t 7 = t 4 = t 5 t 5 , 4 = t 9 2 ¯ ¯ t 8 = 3 t 7 t 3 = t 5 t 5 , 3 = t 9 ¯ ¯ t 7 t 7 , 2 + ¯ t 9 = t 8 + 6 t 2 = t 3 t 3 , 2 = 6 t 2 t 6 + t 1 t 9 ¯ ¯ t 4 t 4 , 1 + ¯ t 10 = t 6 t 9 t 1 = t 3 t 3 , 1 = t 9 cos ( t 1 ) + t 9 t 2 ¯ t 1 = ∂ f ∂ x , ¯ t 2 = ∂ f ∂ y tapenade -b -vars "x y" -outvars f costfunc.f Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 17 / 56

Automatic Differentiation: Reverse mode SUBROUTINE COSTFUNC_B(x, xb, y, yb, f, fb) t1 = x t2 = y t3 = t1*t2 t4 = SIN(t1) t5 = t3 + t4 t6 = t5 + 4 t7 = t2**2 t8 = 3.0*t7 t9 = t8 + 6.0 t10b = fb t6b = t9*t10b t9b = t6*t10b t8b = t9b t7b = 3.0*t8b t5b = t6b t3b = t5b t2b = t1*t3b + 2*t2*t7b t4b = t5b t1b = t2*t3b + COS(t1)*t4b yb = t2b xb = t1b fb = 0.0 END Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 18 / 56

Direct versus reverse AD Direct mode: cost is proportional to number of independent variables Reverse mode: cost does not depend on the number of independent variables Reverse mode preferred when number of independent variables is large Scalar output f Direct mode gives ∇ f · s for given vector s Reverse mode gives ∇ f Vector output R ∈ R n Direct mode gives ∇ R · s for given vector s Reverse mode gives ( ∇ R ) ⊤ · s Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 19 / 56

Issues in reverse AD Intermediate variables required in reverse order Some variables may be over-written Iterative solvers Only fi nal solution required AD differentiates the iterative loop Intermediate solutions stored in stack Huge memory requirements Not practical for large problems Piecemeal differentiation approach (Courty et al., Giles et al.): Modular fl ow solver Adjoint solver written manually AD for differentiating the modules Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 20 / 56

Outline Mathematical formulation 1 Computing gradients 2 Quasi 1-D flow 3 Gradient smoothing 4 Quasi 1-D optimization: Pressure matching 5 2-D adjoint Euler solver 6 Future work 7 Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 21 / 56

Discrete 1-D adjoint equations Finite volume residual for i’th cell R i := F i + 1 / 2 − F i − 1 / 2 = 0 Numerical flux function F = F ( X , Y ) , F i + 1 / 2 = F ( U i , U i + 1 ) Perturbation equation ∂ ∂ δ R i = ∂ X F i + 1 / 2 δ U i + ∂ Y F i + 1 / 2 δ U i + 1 ∂ Y F i − 1 / 2 δ U i + ∂ R i − ∂ ∂ ∂ X F i − 1 / 2 δ U i − 1 − ∂α δα = 0 Introduce adjoint variable V i for i’th cell δ J = ∂ J ∂ J � � V ⊤ ∂αδα + δ U i + i δ R i ∂ U i i i Praveen. C (CTFD, NAL) Adjoints and optimization CTFD Seminar 22 / 56