A Variable-Neighbourhood Search Algorithm for Finding Optimal Run - PowerPoint PPT Presentation

8th Model-Oriented Design and Analysis workshop A Variable-Neighbourhood Search Algorithm for Finding Optimal Run Orders of Experimental Designs in the Presence of Serial Correlation JJ. Garroi, , P. Goos and K. Sörensen, , Universiteit Antwerpen Katholieke Universiteit Leuven

Table of contents • Context of the research • The importance of the run order • Some solutions given in the literature • Features of our algorithm • Brief analysis of the optimalisation results • Conclusions 2 2

Context of the research The source of variation of an industrial experiment is influenced by environmental factors, which are not subject to the experimentation. Industries perform • classical designs function of their prior belief • one run at a time, • without restriction on the order of the run. As pointed out by Constantine (1989), it is likely that the successive observations are correlated and that this influences the design strategy of the experiment. My research elaborates on how to find optimal run order in such situations. 3 3

Assumptions for this talk Only central composite designs will be examined. An AR(1) model is assumed to fit the real correlation structure. We assume that there is a positive dependence between the observations. 3 2 3 3 β β β β ∑ ∑ ∑ ∑ ε ε ( 0 , ) 2 ~ . = + + + + V Y X X X X with i i j i 0 i ij ii = = = + = 1 1 1 1 i i j i i ( ) ( ) . − − 1 1 ˆ ˆ − − − 1 β 1 β 1 = T . = T T ( ) X V Y Var V V X X X X GLS GLS − ρ ⎛ ⎞ 1 0 0 ⎜ ⎟ − ρ + ρ − ρ 2 1 0 ⎜ ⎟ − 1 ρ > = 0 . with V ⎜ ⎟ − ρ + ρ − ρ 2 0 1 ⎜ ⎟ ⎜ ⎟ − ρ 0 0 1 ⎝ ⎠ 4 4

The importance of run orders In the presence of serial correlation , run orders play a role 1 1 2 10 2 10 13 3 11 9 14 14 3 4 15 15 8 9 11 4 13 12 12 8 5 5 6 7 7 6 127.3957 D crit = 120.528 D crit = 5.813 A crit = 6.146 A crit = V crit = 4.785 V crit = 4.847 to have precise estimations ? How to organize the measurements to have accurate predictions ? 5 5

Some solutions given in the literature When the data are correlated, Constantine (1989) sought efficient run orders for factorial designs and showed 1. that designs with maximal level changes possess optimal properties with respect to the D-optimality. Cheng & Steinberg (1991) outlined a reverse foldover algorithm to produce 2. a design with maximal level changes. A simulated annealing was suggested to find D-optimal design for factorial 3. experiments. See Elliot, Eccleston & Martin (1998) and Zhu (2003). An exchange-type algorithm was proposed to construct D-optimum design. 4. See Brimkulov (1980), Ucinski & Atkinson (2004), Stehlik (2006). 6 6

Some solutions given in the literature Criticism : - Poor results of the foldover algorithm if interaction terms have to be estimated, - Exchanges are not the only action to do to maximize the D-criterion, - No attention has been paid to designs involving a large number of factors. We solve these problems by implementing a Variable-Neighbourhood algorithm. We develop a new heuristic to solve this problem. • It has to produce good solutions, • It has to gain computational performance, There is a drawback : the solution proposed may be not the (global) optimal one. Definition : A neighbourhood of the solution x is the set of all solutions reached from x with a specific move. 7 7

Variable Neighbourhood Search Algorithm � It is a heuristic developed by Mladenovic and Hansen (1997), � It explores a multi-neighbourhood structure, � The neighbourhoods are sortered by size, � It systematically changes of neighbourhoods during the search, � It is composed by a local search and a diversification procedure : � The local search will intensify the search and converge to a local optimum. � A perturbation is used as a diversification factor to escape from a local optimum. 8 8

Pseudo-code Notations : N k , k=1,...,6 the set of neighbourhood structures N k (x) the set of solutions in the kth neighbourhood of x Let x a starting solution Repeat Set k <-1 While k<=k max , do : Find the best solution x’ Є N k (x) If x’ better than x then : Set x<-x’ Set k<-1 Else Set k<-k+1 Perturb x until the stopping condition is met. 9 9

Description of the neighbourhoods’structure Linear neighbourhoods Quadratic neighbourhoods 4°) Swap 1°) Shift left 12345 42315 12345 23451 2°) Swap adjacent 5°) Move 12345 21345 12345 23145 6°) 2-Opt 3°) Relabel 12345 43215 12435 23541 10 10

Ordering the neighbourhoods To determine the neighbourhoods’ order and the strategy to adopt, a study of the efficiency of each neighbourhood was carried out. D-efficiency (%) in 10 seconds CPU time Central Composite Designs 20 40 60 80 100 200 Swap adjacent 93,2 92,33 93,39 89,3 90,02 75,79 Relabeling 75,38 70,54 67,15 63,41 62,27 61,15 Shift left 76,58 72,45 67,91 64,11 64,27 63,51 Swap 98,49 99,45 98,34 74,05 69,15 63,05 Move 99,51 97,18 77,63 66,83 66 63,05 2-Opt 99,78 82,99 69,43 63,96 63,78 62,84 The performance of the neighbourhoods decreases with the problem size. 11 11

Ordering the neighbourhoods A good order of the neighbourhoods is : 1. Swap adjacent Linear Neighbourhoods 2. Shift left 3. Relabeling 4. Swap 5. Move Quadratic Neighbourhoods 6. 2-opt 12 12

Brief analysis of the optimalisation results Time required to outperform the randomization We generate random samples of 1000 run orders and pick the best run order. This selection lasts 60 sec and is executed 100 times. Starting from the same 100 random start solutions, the VNS algorithm is applied during 60 sec. The time needed to the VNS to outperform the randomization method is showed in the graph. Applying randomization is consequently not appropriate. 13 13

Brief analysis of the optimalisation results Loss in D-efficiency (%) if randomization Max 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 2 Factors 0.70 1.29 1.87 2.58 3.44 3.96 4.36 4.84 5.37 3 Factors 1.07 2.15 3.23 4.25 5.33 6.02 6.64 7.12 7.48 4 Factors 1.45 3.01 4.58 5.93 7.22 8.08 8.92 9.41 9.59 5 Factors 1.05 2.28 3.50 4.49 5.69 6.44 7.15 7.79 8.29 6 Factors 1.00 1.95 2.90 3.71 5.17 5.47 5.56 6.14 6.35 Min 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 2 Factors 0.01 0.03 0.09 0.14 0.18 0.22 0.28 0.28 0.29 3 Factors 0.39 0.84 1.38 1.82 2.23 2.79 2.95 3.19 3.27 4 Factors 0.78 1.68 2.77 3.64 4.45 5.59 5.90 6.39 6.55 5 Factors 0.49 1.09 1.42 2.35 3.31 3.74 4.50 4.37 4.74 6 Factors 0.10 0.10 0.31 0.41 0.81 0.75 0.88 0.46 1.50 Applying randomization is consequently not appropriate. 14 14

Brief analysis of the optimalisation results In the following, the results for a rotatable central composite design involving three factors and three center points will be presented. 1. The features of D-optimal run orders are outlined. 2. The structure of D-optimal run orders are showed for GLS estimations. 3. The structure of D-optimal run orders are showed for OLS estimations 15 15

Brief analysis of the results � There are many D-optimal run orders for each specification of the correlation parameter. � The D-optimal run order are Bayesian D-optimal indeed, i.e. they are nearly D-optimal for all specification of the correlation parameter. � There are many zero in the information matrix for each specification. � The optimal run order has a specific structure. When ρ = 0.7 When ρ = 0.1 2 0 0 0 0 0 0 1 1 1 14 0 0 0 0 0 0 11 11 11 23 -4 3 -3 0 -6 0 0 2 14 -1 0 0 0 -1 0 0 0 23 -1 0 0 0 3 3 -3 14 0 0 0 0 0 0 0 26 4 -1 -2 1 -2 -1 15 1 0 0 0 0 0 I GLS = 15 0 0 0 0 2 I GLS = 8 0 0 0 0 0 15 0 0 0 2 8 0 0 0 0 15 0 0 -2 8 0 0 0 27 -8 -10 23 5 5 27 -4 23 6 23 22 16 16

Brief analysis of the results A) Optimal run order for GLS estimations 4 � The axial points always appear � They begin and together in the 5 end with a sequence and 3 6 1 centerpoint. one change each 2 time of axis. 7 � There is a symmetry in the sequence with respect to the center point. 17 17

Brief analysis of the results B) Optimal run order for GLS estimations 13 8 � A centerpoint � The factorial separates the points appear in 9 15 groups. small dimensions in two groups ABC=1 � A centerpoint 12 & ABC=-1 . 17 forms the end of the 16 11 sequence. 14 10 18 18

Brief analysis of the results C) Optimal run order for OLS estimations � The axial points 4 15 � The factorial points always appear appear in small together in the 6 dimensions in two sequence and one groups ABC=1 & 13 change each time of 3 ABC=-1 . axis. 10 11 7 17 5 12 2 8 � A centerpoint 16 � A centerpoint separates the forms the end of the 9 groups. sequence. 1 14 For large values of ρ , OLS is not a good idea (6% in loss of D-efficiency ) D-optimal run orders are not robust to the estimation method 19 19