Rosenbrock-like Problems: SMF Versus Other SBO Implementations A.S. - PowerPoint PPT Presentation

Rosenbrock-like Problems: SMF Versus Other SBO Implementations A.S. Mohamed, S. Koziel, J.W. Bandler, M.H. Bakr, and Q.S. Cheng Simulation Optimization Systems Research Laboratory Electrical and Computer Engineering Department, McMaster University Bandler Corporation, www.bandler.com john@bandler.com presented at SURROGATE MODELLING AND SPACE MAPPING FOR ENGINEERING OPTIMIZATION SMSMEO-06, November 9-11, 2006, Technical University of Denmark

Outline space mapping surrogate Rosenbrock function: the benchmark our Rosenbrock test examples SMF and other SBO implementations comparison conclusions

A Space-Mapping-based Surrogate

SMF: Optimization Flowchart

Generalized Space Mapping (GSM) Framework ( Koziel, Bandler, and Madsen, 2006 ) ( i ) : X → R m used at iteration i , a surrogate model R s by the GSM optimization algorithm is defined as = ⋅ ⋅ + + + ⋅ − ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) i i i i i i i R x A R B x c d E x x s c where { ∑ i = − ⋅ ⋅ + + ( ) ( ) ( ) ( ) ( ) ( , , ) arg min || ( ) ( )|| i i i k k A B c R x A R B x c w = k f c 0 k ( , , ) A B c } ∑ + i − ⋅ ⋅ + ⋅ ( ) ( ) || ( ) ( ) || k k v J x A J B x c B = k R R 0 k f c = − ⋅ ⋅ + ⋅ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) i i i i i i i E J x A J B x c B R R f c = − ⋅ ⋅ + ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) i i i i i i d R x A R B x c f c

Rosenbrock Banana Function Rosenbrock, 1960 Fletcher, Practical Methods of Optimization , 1987 Bakr, Bandler, Georgieva, and K. Madsen, 1999 Bandler, Mohamed, Bakr, Madsen, and Søndergaard, 2002 Søndergaard, 2003 Bandler, Cheng, Dakroury, Mohamed, Bakr, Madsen, and Søndergaard, 2004 Giunta and Eldred, 2000; Eldred, Giunta, and Collis, 2004 Robinson, Eldred, Willcox, and Haimes, 2006

Original Rosenbrock Function (Coarse Model) ( Bandler et al., 1999, 2002 ) 2 2 2 = − + − ( ) 100( ) (1 ) x R x x x 2 1 1 c c ⎡ ⎤ ⎡ 1.0 ⎤ x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ c c 1.0 ⎣ ⎦ ⎣ ⎦ x 2 2 ⎡ ⎤ 1.0 1.5 * = ⎢ x ⎥ c 1.0 ⎣ ⎦ 1 * 0.5 * = ( ) 0 R x c c 0 x 2 -0.5 -1 -1.5 -2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 x 1

Transformed Rosenbrock Function (Fine Model) ( Bandler et al., 2002 ) parameter transformation of the original Rosenbrock function 2 2 2 = − + − ( ) 100( ) (1 ) x R u u u 2 1 1 f f − − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1.1 0.2 0.3 u 1 = = + where u ⎢ ⎥ x ⎢ ⎥ ⎢ ⎥ f 0.2 0.9 0.3 ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ u 2 ⎡ 1.2718447 ⎤ * = ⎢ x ⎥ f 0.4951456 ⎣ ⎦

Transformed Rosenbrock Function ( Mohamed et al., 2006 )

Transformed Rosenbrock Function ( Mohamed et al., 2006 ) − − ⎡ ⎤ ⎡ ⎤ 1.1083 0.2035 1.1 0.2 (9) ( ) = ⎢ = ⎢ true B ⎥ B ⎥ 0.2177 0.8928 0.2 0.9 ⎣ ⎦ ⎣ ⎦ − − ⎡ ⎤ ⎡ ⎤ 0.3088 0.3 (9) ( ) = ⎢ = ⎢ true c ⎥ c ⎥ 0.2810 0.3 ⎣ ⎦ ⎣ ⎦ ⎡ ⎤ ⎡ ⎤ 1.2718446 1.2718447 (9) * = ⎢ = ⎢ x ⎥ x ⎥ f f 0.4951456 0.4951456 ⎣ ⎦ ⎣ ⎦ (9) * = − = 5.4e 16 0 R R f f

Response-Transformed Rosenbrock Function (Fine Model) ( Mohamed et al., 2006 ) a response linear transformation of the original Rosenbrock function ⎡ ⎤ 2 2 2 = − + − + ( ) 2 100( ) (1 ) 3 R x x x x ⎣ ⎦ 2 1 1 f f ⎡ ⎤ ⎡ ⎤ 1.0 x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ f f 1.0 ⎣ ⎦ ⎣ ⎦ x 2

Response-Transformed Rosenbrock Function ( Mohamed et al., 2006 )

Response-Transformed Rosenbrock Function ( Mohamed et al., 2006 ) ( ) = true 2.0 (6) = 2.0007 A A ( ) = true (6) 3.0 = 3.0 D D ⎡ ⎤ 1.0 ⎡ ⎤ 1.0000003 * = ⎢ (6) = ⎢ x ⎥ x ⎥ f 1.0 f ⎣ ⎦ 1.0000005 ⎣ ⎦ * = 0 (6) = − 1.4e 13 R R f f

Response and Parameter-Transformed Rosenbrock Function (Fine Model) ( Mohamed et al., 2006 ) a response (scale + shift) and parameter (rotation + shift) transformation of the original Rosenbrock function ⎡ ⎤ 2 2 2 = − + − + ( ) 2 100( ) (1 ) 3 x R u u u ⎣ ⎦ 2 1 1 f f − − ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 1.1 0.2 0.3 u 1 = = + where u ⎢ ⎥ ⎢ ⎥ x ⎢ ⎥ f 0.2 0.9 0.3 ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ u 2 ⎡ ⎤ 1.2718447 * = ⎢ x ⎥ f 0.4951456 ⎣ ⎦

Response and Parameter-Transformed Rosenbrock Function ( Mohamed et al., 2006 )

Response and Parameter-Transformed Rosenbrock Function ( Mohamed et al., 2006 ) ( ) = (15) = true 2.0 4.8715 A A − − ⎡ ⎤ ⎡ ⎤ 1.1 0.2 0.9862 0.6372 ( ) (15) = ⎢ = ⎢ true B ⎥ B ⎥ − 0.2 0.9 1.8238 1.1784 ⎣ ⎦ ⎣ ⎦ − ⎡ ⎤ 0.3 ⎡ 0.8446 ⎤ ( ) = ⎢ (15) = ⎢ true c ⎥ c ⎥ 0.3 1.449 ⎣ ⎦ ⎣ ⎦ ( ) (15) = = true 3.0 0.0 d d ⎡ ⎤ ⎡ ⎤ 1.2718447 1.2718442 * = ⎢ (15) = ⎢ x ⎥ x ⎥ f f 0.4951456 0.4951449 ⎣ ⎦ ⎣ ⎦ * (15) = = − − 3 3 (4.6e 13) R R f f

Rosenbrock Function (Low Fidelity Model with Offsets) ( Eldred, Giunta, and Collis, AIAA, 2004 ) low fidelity model 2 2 2 = − + + − ( ) 100( 0.2) (0.8 ) R x x x x 2 1 1 c c ⎡ ⎤ ⎡ ⎤ 0.8 x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ c c 0.44 ⎣ ⎦ ⎣ ⎦ x 2 high fidelity model 2 2 2 = − + − ( ) 100( ) (1 ) R x x x x 2 1 1 f f ⎡ ⎤ ⎡ ⎤ 1.0 x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ f f 1.0 ⎣ ⎦ ⎣ ⎦ x 2

Rosenbrock Function (Low Fidelity Model with Offsets) ( Mohamed et al., 2006 ) method #of iters FM Evals R f Full 2nd add ¹ 5 11 1.24e – 15 Full 2nd mult ¹ 31 59 8.96e – 15 SR1 2nd comb ¹ 23 42 4.73e – 15 FD 2nd add ¹ 5 23 1.53e – 10 SMF 6 35 2.79e – 14 ¹ Eldred, Giunta, and Collis, AIAA, 2004

Rosenbrock Function (Low Fidelity Model with Scalings) ( Eldred, Giunta, and Collis, AIAA, 2004 ) low fidelity model 2 2 2 = − + − ( ) 100(1.25 ) (1 1.25 ) R x x x x 2 1 1 c c ⎡ ⎤ ⎡ ⎤ 0.8 x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ c c 0.512 ⎣ ⎦ ⎣ ⎦ x 2 high fidelity model 2 2 2 = − + − ( ) 100( ) (1 ) R x x x x 2 1 1 f f ⎡ ⎤ ⎡ ⎤ 1.0 x 1 * = = where and x ⎢ ⎥ x ⎢ ⎥ f f 1.0 ⎣ ⎦ ⎣ ⎦ x 2

Rosenbrock Function (Low Fidelity Model with Scalings) ( Mohamed et al., 2006 ) method #of iters FM Evals R f BFGS 2nd comb ¹ 292 514 1.68e – 14 BFGS 2nd mult ¹ 87 154 1.38e – 13 Full 2nd mult ¹ 42 76 2.59e – 12 FD 2nd add ¹ 17 68 4.58e – 9 SMF 14 77 9.39e – 15 ¹ Eldred, Giunta, and Collis, AIAA, 2004

Multi-Fidelity Optimization (MFO) Algorithm ( Castro, Gray, Giunta, and Hough, 2006 ) the MFO algorithm incorporates a derivative free optimization approach based on two techniques: 1. Asynchronous Parallel Pattern Search (APPS) 2. Space Mapping (SM)

Multi-Variable Rosenbrock Function (Case 1) ( Castro, Gray, Giunta, and Hough, 2006 ) high fidelity model 2 2 2 2 2 2 = − + − + − + − ( ) 100( ) (1 ) 100( ) (1 ) x R x x x x x x 2 1 1 3 2 2 f f [ ] [ ] T * T = = where and 1 1 1 x x x x x 1 2 3 f f low fidelity model 2 2 2 = − + − ( ) 100( ) (1 ) R x x x x 2 1 1 c c [ ] [ ] T * T = = where and 1 1 x x x x 1 2 c c

Multi-Variable Rosenbrock Function (Case 1, using B) ( Mohamed et al., 2006 ) six SM parameters − − ⎡ ⎤ 0.05 0.15 0.45 ⎢ ⎥ (6) = ⎢ − 0.19 1.0 3 1.02 B ⎥ ⎢ ⎥ 0.0 0.0 1.0 ⎣ ⎦ # of function * method R f x evaluations f [ ] T MFO ¹ 1.35 87 0.3 0.68 0.46 [ ] T 1.05 1.09 1.14 SMF 0.38 30 ¹ Castro, Gray, Giunta, and Hough, 2006

Multi-Variable Rosenbrock Function (Case 2) ( Castro, Gray, Giunta, and Hough, 2006 ) high fidelity model 2 2 2 2 2 2 = − + − + − + − ( ) 100( ) (1 ) 100( ) (1 ) x R x x x x x x 2 1 1 3 2 2 f f 2 2 2 + − + − 100( ) (1 ) x x x 4 3 3 [ ] [ ] T * T = = where and 1 1 1 1 x x x x x x 1 2 3 4 f f low fidelity model 2 2 2 = − + − ( ) 100( ) (1 ) x R x x x 2 1 1 c c [ ] [ ] T * T = = where and 1 1 x x x x 1 2 c c

Multi-Variable Rosenbrock Function (Case 2, using B) ( Mohamed et al., 2006 ) eight SM parameters − − ⎡ ⎤ 5.49 1.92 2.81 0.31 ⎢ ⎥ − 3.56 2.56 5.07 0.6 8 ⎢ ⎥ (9) = ⎢ B ⎥ 0.0 0.0 1.0 0.0 ⎢ ⎥ 0.0 0.0 0.0 1.0 ⎣ ⎦ # of function * method x R f evaluations f [ ] T MFO ¹ − 1.58 154 0.55 0.29 0.087 0.003 [ ] T − 0.99 0.93 0.89 0.64 SMF 0.451 103 ¹ Castro, Gray, Giunta, and Hough, 2006