Higher Derivatives in Matlab Using MAD Shaun Forth Engineering - PowerPoint PPT Presentation

Higher Derivatives in Matlab Using MAD Shaun Forth Engineering Systems Department Cranfield University (DCMT Shrivenham) Shrivenham, Swindon SN6 8LA, U.K. email: S.A.Forth@cranfield.ac.uk www.amorg.co.uk/AD/staff_SAF.html 5th European Workshop on Automatic Differentiation University of Hertfordshire, UK, May 21-22 2007 1/ 29 Higher Derivatives in Matlab Using MAD

Plan Introduction 1 MAD’s Forward Mode 2 fmad class derivvec class Higher Derivatives 3 Second Derivatives How The Heck Does This Work? Required Changes to MAD Reverse Mode AD in Matlab 4 Results 5 Minpack Elastic-Plastic Torsion (EPT) Problem Object-Oriented Test Case Conclusions & Future Work 6 2/ 29 Higher Derivatives in Matlab Using MAD

Introduction MAD’s overloaded forward mode is efficient for first derivatives [For06]. Requests for higher derivatives of order 2 to 4 for uncertainty analysis and robust optimisation. Could implement a Taylor series class ` a la ADOL-C [GJU96] - manpower expensive. OR use MAD to recursively differentiate itself (c.f. EuroAD2 - Barak Perlmutter and Jeffrey Siskind, [Gri00, p109]). 3/ 29 Higher Derivatives in Matlab Using MAD

MAD’s Forward Mode Based on two classes: fmad class - forward mode AD by operator overloading derivvec class - for storage and combination of multiple directional derivatives. 4/ 29 Higher Derivatives in Matlab Using MAD

fmad Class Constructor e.g. x=fmad([1.1 2 3],[4 5 6]); 5/ 29 Higher Derivatives in Matlab Using MAD

fmad Class Constructor e.g. x=fmad([1.1 2 3],[4 5 6]); Defines fmad object with ◮ value component - row vector [1.1 2 3] ◮ deriv component - single directional derivative [4 5 6] 5/ 29 Higher Derivatives in Matlab Using MAD

fmad Class Constructor e.g. x=fmad([1.1 2 3],[4 5 6]); Defines fmad object with ◮ value component - row vector [1.1 2 3] ◮ deriv component - single directional derivative [4 5 6] Perform overloaded operations, e.g., element-wise multiplication via times z=x.*x value = 1.2100 4.0000 9.0000 derivatives = 8.8000 20.0000 36.0000 AD enabled by the times.m function of the fmad class. 5/ 29 Higher Derivatives in Matlab Using MAD

fmad times function for z=x.*y function z=times(x,y) if isa(x,’fmad’) z=x; % deep copy to avoid constructor, no refs. to x if isa(y,’fmad’) z.deriv=y.value.*z.deriv+z.value.*y.deriv; z.value=z.value.*y.value; else z.value=z.value.*y; z.deriv=y.*z.deriv; end else z=y; % deep copy to avoid constructor, no refs. to y z.value=x.*z.value; z.deriv=x.*z.deriv; end 6/ 29 Higher Derivatives in Matlab Using MAD

Working with Multiple Directional Derivatives What if we want the Jacobian? 7/ 29 Higher Derivatives in Matlab Using MAD

Working with Multiple Directional Derivatives What if we want the Jacobian? Seed derivatives with identity I 3 x=fmad([1.1 2 3],eye(3)); 7/ 29 Higher Derivatives in Matlab Using MAD

Working with Multiple Directional Derivatives What if we want the Jacobian? Seed derivatives with identity I 3 x=fmad([1.1 2 3],eye(3)); Overloaded operation with same times function gives value = 1.2100 4.0000 9.0000 Derivatives Size = 1 3 No. of derivs = 3 derivs(:,:,1) = 2.2000 0 0 derivs(:,:,2) = 0 4 0 derivs(:,:,3) = 0 0 6 7/ 29 Higher Derivatives in Matlab Using MAD

The fmad and derivvec classes The fmad constructor function function xad=fmad(x,dx) % FUNCTION: FMAD % SYNOPSIS: Class constructor for forward Matlab AD objects xad.value=x; sx=size(xad.value); sd=size(dx); if prod(sx)==prod(sd) % #values=#derivs => single directional derivative xad.deriv=reshape(dx,sx); else % #values~=#derivs => multiple directional derivatives % Pass derivatives to derivvec constructor xad.deriv=derivvec(dx,size(xad.value)); end 8/ 29 Higher Derivatives in Matlab Using MAD

The fmad and derivvec classes The fmad constructor function function xad=fmad(x,dx) % FUNCTION: FMAD % SYNOPSIS: Class constructor for forward Matlab AD objects xad.value=x; sx=size(xad.value); sd=size(dx); if prod(sx)==prod(sd) % #values=#derivs => single directional derivative xad.deriv=reshape(dx,sx); else % #values~=#derivs => multiple directional derivatives % Pass derivatives to derivvec constructor xad.deriv=derivvec(dx,size(xad.value)); end 8/ 29 Higher Derivatives in Matlab Using MAD

The derivvec class Store derivatives as a matrix with each directional derivative unrolled into a column. 9/ 29 Higher Derivatives in Matlab Using MAD

The derivvec class Store derivatives as a matrix with each directional derivative unrolled into a column. e.g. derivvec(eye(3),[1 3]) derivatives conceptualised as as, � direc 1 � direc 2 direc 3 [1, 0, 0] [0, 1, 0] [0, 0, 1] 9/ 29 Higher Derivatives in Matlab Using MAD

The derivvec class Store derivatives as a matrix with each directional derivative unrolled into a column. e.g. derivvec(eye(3),[1 3]) derivatives conceptualised as as, � direc 1 � direc 2 direc 3 [1, 0, 0] [0, 1, 0] [0, 0, 1] But stored as,  direc 1 direc 2 direc 3    1 0 0       1 0 0    = 0 1 0     0 1 0        0 0 1 0 0 1 9/ 29 Higher Derivatives in Matlab Using MAD

The times operation of the derivvec class e.g. Need to calculate,   direc 1 direc 2 direc 3  1   0   0  � �   1 . 1 2 3 . ∗   0 1 0         0 0 1 with multiplication of each of the 3 directional derivatives. Convert value to column matrix and replicate 3 times  1 . 1 1 . 1 1 . 1   1 0 0   1 . 1 0 0   . ∗  = 2 2 2 0 1 0 0 2 0     3 3 3 0 0 1 0 0 3 columns give required directional derivatives. 10/ 29 Higher Derivatives in Matlab Using MAD

Accessor Functions Getting the value getvalue(z) ans = 1.2100 4.0000 9.0000 Getting external representation of derivatives getderivs(z) ans(:,:,1) = 2.2000 0 0 ans(:,:,2) = 0 4 0 ans(:,:,3) = 0 0 6 Getting unrolled internal representation getinternalderivs(z) ans = 2.2000 0 0 0 4 0 0 0 6 11/ 29 Higher Derivatives in Matlab Using MAD

Higher Derivatives Basic idea is to be able to use MAD recursively. For first derivatives: x=fmad([1.1 2 3],eye(3)); z=x.*x zvalue=getvalue(z) zvalue = 1.2100 4.0000 9.0000 zderivs=getinternalderivs(z) zderivs = 2.2000 0 0 0 4.0000 0 0 0 6.0000 Can we just differentiate the above process a second time with MAD? 12/ 29 Higher Derivatives in Matlab Using MAD

Second Derivatives xx=fmad([1.1 2 3],eye(3)); % xx’s value and derivs. D1=I_3 x=fmad(xx,eye(3)); % x’s value =xx, x’s derivs. D2=I_3 z=x.*x; zvalue=getvalue(getvalue(z)) % z’s value’s value zvalue = 1.2100 4.0000 9.0000 zderivs=getinternalderivs(getvalue(z)) % z’s value’s derivs % in D1 direcs. zderivs = 2.2000 0 0 0 4.0000 0 0 0 6.0000 zderivs=getvalue(getinternalderivs(z)) % value of z’s derivs % in D2 direc. zderivs = 2.2000 0 0 0 4.0000 0 0 0 6.0000 13/ 29 Higher Derivatives in Matlab Using MAD

Second Derivatives (ctd) z2ndderivs=reshape(... getinternalderivs(getinternalderivs(z)),[3 3 3]) % derivs. in D1 direc. of z’s derivs. in D2 direc. z2ndderivs(:,:,1) = 2 0 0 0 0 0 0 0 0 z2ndderivs(:,:,2) = 0 0 0 0 2 0 0 0 0 z2ndderivs(:,:,3) = 0 0 0 0 0 0 0 0 2 14/ 29 Higher Derivatives in Matlab Using MAD

Why The Heck Does This Work? xx value deriv xx=fmad([1.1 2 3],eye(3)); creates object, [1.1 2 3] eye(3) x deriv value x=fmad(xx,eye(3)); creates object, xx eye(3) value deriv [1.1 2 3] eye(3) 15/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv x.value.*x.value 2.*x.value.*x.deriv 16/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv 2.*x.value.*x.deriv xx xx value deriv value deriv .* [1.1 2 3] eye(3) [1.1 2 3] eye(3) 16/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv 2.*x.value.*x.deriv deriv value   2 . 2 0 0 [1 . 21 , 4 . 0 , 9 . 0] 0 4 0   0 0 6 16/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv deriv value xx   2 . 2 0 0 value deriv [1 . 21 , 4 . 0 , 9 . 0] 0 4 0 eye(3) .* 2   0 0 6 [1.1 2 3] eye(3) 16/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv deriv value xx xx   2 . 2 0 0 value deriv deriv value [1 . 21 , 4 . 0 , 9 . 0] 0 4 0 eye(3) .* 2 I 3 .* 2     1 . 1 1 . 1 1 . 1 0 0 6 [1.1 2 3] 2 2 eye(3) 2 [ I 3 , I 3 , I 3 ]   3 3 3 16/ 29 Higher Derivatives in Matlab Using MAD

How The Heck (ctd.) z=x.*x; forms object, z value deriv deriv value xx deriv   2 . 2 0 0 value value deriv [1 . 21 , 4 . 0 , 9 . 0] 0 4 0 eye(3) .* 2       2 . 2 0 0 e 1 0 0 6  [ I 3 , I 3 , I 3 ] .* 2 [1.1 2 3] 0 4 0 eye(3) e 2    0 0 6 e 3 16/ 29 Higher Derivatives in Matlab Using MAD