Weighted space-filling designs Dave Woods D.Woods@southampton.ac.uk - PowerPoint PPT Presentation

Weighted space-filling designs Dave Woods D.Woods@southampton.ac.uk www.southampton.ac.uk/ ∼ davew Joint work with Veronica Bowman (Dstl) Supported by the Defence Threat Reduction Agency 1 / 34

Outline • Motivation & background • Designs incorporating prior information • Illustrative examples • Application to dispersion modelling • Graphical method of design evaluation • Work in progress . . . 2 / 34

Dispersion modelling Aim: Prediction of the downwind hazard generated by a chemical or biological (or other) release • Accident response; military 120 planning; volcanic ash; . . . 100 • Variety of models 80 y.cord • Gaussian plume (Clarke, 1979) 60 • Gaussian puff (Sykes et al., 1998) 40 • NAME (Jones et al., 2007) 20 20 40 60 80 100 120 • Sensor placement x.cord 3 / 34

Dispersion modelling Ex. 3 results 4 / 34

Dispersion modelling 1. The input variables are usually of two types (meteorological and source), and can be quantitative or qualitative 2. There is substantial prior information about the distribution of the input variables from, for example, empirical observations (meteorological) or expert prior knowledge (source) 3. These prior distributions are not usually independent, either within type (for example, wind direction and speed is defined via a wind rose) or between type (wind direction and source location) 4. The distributions define a joint probability density (or weight function) on the design region, which is likely to have substantial areas of low weight 5 / 34

Prior information 6 / 34

Aim of this work • Reduce the number of model evaluations required through designed experiments • Develop a class of criteria for constructing space-filling designs that take account of prior information and, particularly, relationships between input variables • Evaluate designs from competing criteria in terms of (a) sampling properties and (b) space-filling properties 7 / 34

Weight function For any point x ∈ X , define a weight function w ( x ) • w ( x ) ≥ 0 is a problem-specific weight function, e.g. defining the probability of obtaining a useful response We can (almost) think about this in terms of fuzzy sets ˜ X = { ( x , w ( x )) , x ∈ X} , where w ( x ) is the (unnormalised) membership function of the fuzzy set ˜ X [e.g. Dubois & Prade, 1980] Space-filling designs for fuzzy design spaces? 8 / 34

Space-filling designs • Collections of combinations of values of the input variables that (attempt to) provide information on every (relevant) region of the design space • Typically, they either • Spread the design points as far apart as possible • Cover the design region as evenly as possible • Only take one observation at each combination of input values (ideal for deterministic models) • Allow a variety of statistical models to be fitted • Distance-based, Latin Hypercubes, Uniform designs [see, e.g., Santner et al., 2003] 9 / 34

Extensions To apply space-filling designs to dispersion modelling, we need to include • Qualitative variables • Weight function, including dependencies between variables Consider • Quantitative variables x 1 , . . . , x k 1 • Qualitative variables x k 1 +1 , . . . , x k 1 + k 2 , with x j having m j levels denoted by M j = { 1 , . . . , m j } 10 / 34

Distance metrics Define the distance between two points x , y ∈ X = R × � j M j , where R ⊂ R k 1 , as � k 1 + k 2 k 1 � � ( x i − y i ) 2 + α � � d ( x , y ) = I[ x j � = y j ) ] , � i =1 j = k 1 +1 where I[ r � = s ] is the indicator function that takes the value 1 if r � = s and 0 otherwise In this talk, α = 1 Similar for ordered categorical variables [Qian et al. (2008)] 11 / 34

Space-filling criteria • Coverage: minimise � 1 / m � m �� � φ um ( d ) = min x ∈ d w ( y ) d ( x , y ) d y X • Spread: minimise � n � − p � 1 / p � � φ sp ( d ) = x ∈ d \{ x i } w ( x ) w ( x i ) d ( x , x i ) min i =1 • In this talk, m = p = 1 [See, e.g., Johnson et al. (1990) for unweighted versions] 12 / 34

Space-filling criteria • For coverage designs, we want to “attract” the design to relevant areas of the design space • Note that if w ( y ) = 0, min x ∈ d w ( y ) d ( x , y ) = 0 for all d • For spread designs, we want the design points to “repel” away from each other � − k → ∞ � • Note that as w ( x i ) → 0, min x ∈ d \{ x i } w ( x ) w ( x i ) d ( x , x i ) 13 / 34

Implementation • Computer search • Row exchange (e.g. Royle, 2002 and SAS) • Co-ordinate exchange • Efficient storage/updating of distances • e.g. cover.design in Fields • Quasi-random numbers (low-discrepancy sequence) used for the candidate list and also to approximate X 14 / 34

Alternative and connected approaches • Transform an iid (quasi-) random sample or space-filling design • Latin Hypercube Sample transformed to match marginal distributions and pairwise correlations [McKay et al., 1979; Iman & Conover, 1982] • Design spaces with hard constraints on X • Constrained optimisation [e.g. Kleijnen et al., 2010] 15 / 34

Graphical design assessment 1. Fraction of Design Space (FDS) with respect to the distance [Zahran et al., 2003] – assess the space-filling properties of a design 2. Fraction of Design Points (FDP) with respect to the weight function – assess the sampling properties of a design 16 / 34

Weight functions Assume some prior distribution, p ( x ), on X can be elicited from subject experts and/or derived from historical data We consider two weight functions (1) w ( x ) = p ( x ) w ( x ) ≥ 0 (2) w ( x ) = (1 − α p ( x )) − γ w ( x ) ≥ 1 [Joseph et al., 2010] with α < 1 / max p ( x ) and γ > 0 17 / 34

Examples • Two quantitative variables ( x 1 , x 2 ); p ( x ) is a logistic function • Two quantitative variables ( x 1 , x 2 ) & one qualitative ( x 3 ); conditional on x 3 , p ( x ) is a bivariate normal pdf • Seven quantitative variables ( x 1 − x 7 ); application to the dispersion example Ex. 1 Ex. 2 Ex. 3 18 / 34

Example 1 - coverage (1) (2) α = 1 , γ = 1 3 3 2 2 1 1 x 2 0 x 2 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 (2) α = 1 , γ = 0.75 (2) α = 0 , γ = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 19 / 34

Example 1 - coverage (1) 1.0 (2) α = 1 , γ = 1 (2) α = 1 , γ = 0.75 (2) α = 0 , γ = 0 6 0.8 5 0.6 4 p(x) Dist 3 0.4 2 0.2 1 0.0 0 0 20 40 60 80 100 0 20 40 60 80 100 % design points % design space 20 / 34

Example 1 - spread (1) (2) α = 1 , γ = 1 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 (2) α = 0.75 , γ = 0.5 (2) α = 0 , γ = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 21 / 34

Example 1 - spread (1) 1.0 (2) α = 1 , γ = 1 (2) α = 1 , γ = 0.75 (2) α = 0 , γ = 0 0.8 6 0.6 4 p(x) Dist 0.4 2 0.2 0.0 0 0 20 40 60 80 100 0 20 40 60 80 100 % design points % design space 22 / 34

Example 1 - comparison (1) Coverage 1.0 (1) Spread (2) Coverage α = 0 , γ = 0 (2) Spread α = 0 , γ = 0 4 0.8 3 0.6 p(x) Dist 2 0.4 1 0.2 0.0 0 0 20 40 60 80 100 0 20 40 60 80 100 % design points % design space Ex. 3 23 / 34

Example 2 - coverage (1) x 3 = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 x 3 = 1 x 3 = 2 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 24 / 34 x 1 x 1

Example 2 - coverage (2) x 3 = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 x 3 = 1 x 3 = 2 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 25 / 34 x 1 x 1

Example 2 - spread (1) x 3 = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 x 3 = 1 x 3 = 2 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 26 / 34 x 1 x 1

Example 2 - spread (2) x 3 = 0 3 3 2 2 1 1 x 2 x 2 0 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 x 3 = 1 x 3 = 2 3 3 2 2 1 1 x 2 0 x 2 0 -1 -1 -2 -2 -3 -3 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 x 1 x 1 27 / 34

Example 2 - comparison (1) Coverage 3.0 4 (2) Coverage α = 1 2.71 , γ = 0.75 (1) Spread (2) Spread α = 1 2.71 , γ = 0.3 Coverage 2.5 3 2.0 p(x) Dist 1.5 2 1.0 1 0.5 0.0 0 0 20 40 60 80 100 0 20 40 60 80 100 % design points % design space 28 / 34

Example 3 • Seven quantitative variables x 1 − x 7 • Wind speed and direction • Cloud cover • x − y location, mass and time of release • Prior information . . . • from past data (meteorological) and subject experts (source) • defines a release scenario • Compare: • Weighted space-filling designs with weight function (1) w ( x ) = p ( x ) • Latin hypercube samples transformed to match marginal distributions and pairwise correlations [McKay et al., 1979; Iman & Conover, 1982] 29 / 34