A bisection method for generating random utility functions in SMAA - PowerPoint PPT Presentation

A bisection method for generating random utility functions in SMAA Rudolf Vetschera 1 Luis Dias 2 1 University of Vienna, Austria rudolf.vetschera@univie.ac.at 2 University of Coimbra, Portugal lmcdias@fe.uc.pt DA2PL 2018 Poznan, Nov. 22-23, 2018

Outline • Problem statement • Arbitrary functjons – Generatjon methods – Computatjonal results • Concave functjons – Generatjon methods – Computatjonal results • Conclusions 2

Problem statement u( x ) x Given a set of performance levels { x 0 , x 1 , x 2 , … x n } assign to them utility values {u( x 0 ), u( x 1 ), u( x 2 ),... u( x n )} so that •the resulting utility function is monotonic •the entire range of possible utility functions is covered •there is no bias toward a specific shape or •(optional) the function has a predefined shape 3

A trivial problem? 1)Generate random values from uniform distributjon 2)Sort them 3)Assign to x values in increasing order 4

A trivial problem? x 1)Generate random values 8 from uniform distributjon 36 2)Sort them 40 3)Assign to x values in 44 increasing order 48 60 68 72 80 84 88 90 96 Data from: Kadzinski, M. and Tervonen, T. (2013). Robust multj-criteria ranking with additjve value models and holistjc pair-wise preference statements. European Journal of Operatjonal Research, 228(1):169 - 180. 5

A trivial problem? x 1)Generate random values 8 from uniform distributjon 1.0 36 2)Sort them 40 0.8 3)Assign to x values in 44 increasing order 48 0.6 30 such runs: u(x) 60 ● Distjnct convex shape 68 0.4 ● Most functjons are in 72 narrow band 0.2 80 84 0.0 88 90 20 40 60 80 x 96 Data from: Kadzinski, M. and Tervonen, T. (2013). Robust multj-criteria ranking with additjve value models and holistjc pair-wise preference statements. European Journal of Operatjonal Research, 228(1):169 - 180. 6

Another problem 100 performance levels 10 performance levels 1.0 1.0 0.8 0.8 0.6 0.6 u(x) u(x) 0.4 0.4 0.2 0.2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x x All data points drawn from uniform distributjon (not skewed) 7

The general idea: Bisection 1)Find point (aturibute value) between existjng points 8

The general idea: Bisection 1)Find point between existjng points 2)Assign utjlity to it 9

The general idea: Bisection 1)Find point between 3)and repeat for all new intervals existjng points 2)Assign utjlity to it 10

Main step 1: Find a split point Given: Two points x lo and x up and their utjlity values, fjnd index s of split point: s = argmin i | x i −( x lo + x up )/ 2 | Approximate midpoint: s =⌈( lo + up )/ 2 ⌉ s =⌊( lo + up )/ 2 ⌋ Median random choice: s : lo < s < up Random choice: x i + 1 − x i − 1 p i = Weighted random choice: ( x up − x lo )+( x up − 1 − x lo + 1 ) 11

Main step 2: Determine u(x s ) Straightgorward: uniform distributjon between u( x lo ) and u( x up ) Correctjon if split point is not the midpoint p =0.5 p =0.5 12 x lo x up

Evaluation criteria (1): Coverage max ( x i − 1 )+ u max ( x i ) n ( 1 − u ) ( x i − x i − 1 ) Above = ∑ 2 i = 2 1.0 0.8 Coverage = 1 −( Above + Below ) 0.6 u(x) 0.4 n u min ( x i − 1 )+ u min ( x i ) Below = ∑ ( x i − x i − 1 ) 2 i = 2 0.2 0.0 0.2 0.4 0.6 0.8 1.0 13 x

Evaluation criteria (2): Bias Average area below utjlity functjon: ● Should be 0.5 for well-balanced shapes Absolute difgerence in fractjons of points below and above straight line (risk ● neutral utjlity): Should be zero for balanced (symmetric) shapes 14

Compared approaches Splittjng points: ● Next to mean (xB), median (xM), random (xR), random proportjonal (xP) Correctjon in generatjng u(x): ● Direct random value (Dy), weighted (Wy) e.g. DM is direct with median split point As comparison: ● – Direct assignment of Random Values (RV) – Using random values as difgerences (RD) – Creatjng utjlitjes at evenly spaced points using bisectjon, and linear interpolatjon (BL) 15

Coverage: Effect of attribute levels 10 levels 100 levels cover cover 1.0 1.0 0.8 0.8 Random methods improve 0.6 0.6 0.4 0.4 Simple methods degrade 0.2 0.2 0.0 0.0 RV RD BL DB WB DM WM DR WR DP WP RV RD BL DB WB DM WM DR WR DP WP nUtil: 1000, nPoint: 10, Delta: 0.01 nUtil: 1000, nPoint: 100, Delta: 0.01 16

Bias: Effects of skewness Δ = 0.01 Δ = 2 diffAbs diffAbs 1.0 1.0 Simple methods become 0.8 0.8 completely biased 0.6 0.6 0.4 0.4 Weighted random remains unbiased 0.2 0.2 0.0 0.0 RV RD BL DB WB DM WM DR WR DP WP RV RD BL DB WB DM WM DR WR DP WP nUtil: 1000, nPoint: 100, Delta: 0.01 nUtil: 1000, nPoint: 100, Delta: 2 17

Combining coverage and bias 100 points, delta=2 0.50 WB DB BL WP DP 0.45 areaMean WR WM 0.40 DR 0.35 RD RV DM 0.2 0.4 0.6 0.8 cover 18

Bisection for concave function 1) Randomly generate slope between slopes of adjacent segments 19

Bisection for concave function Possible range 1) Randomly generate slope between slopes of adjacent segments 2) Select split point 3) At split point, randomly generate new point between current segment and lower of the two slope lines 20

Variants concave function Code Descriptjon Benchmark: Sort random slopes, CD apply them sequentjally CB Split point closest to mean CM Split point median CR Random selectjon of split point Random selectjon with proportjonal CP probabilitjes 21

Results: Coverage Delta = 0.01 Delta = 2 Points Method above below coverage above below coverage CD 20.60% 53.50% 25.91% 27.24% 53.05% 19.71% CB 11.67% 50.03% 38.30% 17.22% 50.03% 32.75% 10 CM 10.71% 50.03% 39.26% 21.11% 50.02% 28.88% CR 5.57% 50.02% 44.41% 14.93% 50.01% 35.06% CP 5.58% 50.02% 44.40% 14.85% 50.02% 35.13% CD 29.16% 62.32% 8.52% 34.24% 59.41% 6.35% CB 11.28% 50.03% 38.69% 11.88% 50.03% 38.09% 100 CM 11.34% 50.03% 38.63% 22.60% 50.02% 27.39% CR 0.82% 50.02% 49.16% 5.49% 50.01% 44.50% CP 0.83% 50.02% 49.16% 4.78% 50.02% 45.20% 22

Summary Criteria for "good" sampling of utjlity functjons: ● – Coverage – Bias Directly generatjng utjlity values has problems in both criteria ● Bisectjon approach with (proportjonal) random split point and correctjon for ● ofg-center values provides best results Approach can be modifjed to generate only functjons of specifjc shape ● 23

Thank you for your attention!

Why bias toward convexity? Distributjon of values skewed to the right ● – Difgerence between values decreases Utjlity values drawn from uniform distributjon ● – Expected difgerence between them is constant Expected slope increases - convex ● 25

Skewed density of performance Density f( x ) 0 ≤Δ≤ 2 d 0 x Performance levels between 0 and 1 1 26

Generating skewed performance levels F ( x )= d 0 x + Δ f ( x )= d 0 +Δ x 2 Density: cdf 2 x F ( 1 )= 1 ⇒ d 0 = 1 −Δ/ 2 ⇒ F ( x )=( 1 − Δ 2 ) x + Δ 2 2 x Inverse method: F ( x )= r 2 x = 1 2 − 1 Δ + √ ( 2 / 4 + 2 r Δ− 1 ) Δ 27

Simulation parameters Performance levels 10, 25, 100 Skewness (Δ) 0.01, 0.5, 1, 1.5, 2 Functjons generated 100, 1000 28

Bias: Effects of skewness Δ = 0.01 Δ = 2 areaMean areaMean 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 RV RD BL DB WB DM WM DR WR DP WP RV RD BL DB WB DM WM DR WR DP WP nUtil: 1000, nPoint: 100, Delta: 0.01 nUtil: 1000, nPoint: 100, Delta: 2 29

Coverage: Mean values 10 points 100 points delta 0.01 0.5 1 1.5 2 0.01 0.5 1 1.5 2 RV 65.9% 65.1% 64.2% 62.0% 59.2% 25.1% 24.9% 24.2% 23.2% 21.5% RD 48.4% 47.6% 46.7% 44.7% 42.2% 14.8% 14.6% 14.2% 13.6% 12.4% BL 78.3% 77.7% 77.2% 75.8% 74.1% 86.3% 86.2% 86.2% 85.9% 84.6% DB 83.3% 82.7% 81.9% 80.0% 77.2% 86.8% 86.6% 86.6% 86.3% 84.9% WB 83.5% 82.8% 82.1% 80.1% 77.4% 86.8% 86.7% 86.7% 86.4% 85.1% DM 84.1% 83.1% 82.2% 79.6% 76.5% 86.8% 86.3% 84.7% 81.9% 76.5% WM 84.8% 83.9% 83.2% 81.2% 78.3% 87.1% 86.8% 86.2% 84.8% 81.8% DR 88.9% 88.0% 87.2% 85.0% 82.0% 98.4% 98.3% 98.0% 97.2% 94.0% WR 88.7% 87.8% 87.1% 84.8% 81.9% 97.9% 97.8% 97.5% 96.6% 93.5% DP 88.8% 88.0% 87.2% 85.0% 82.1% 98.4% 98.4% 98.1% 97.5% 94.6% WP 88.6% 87.8% 87.0% 84.8% 81.9% 97.9% 97.9% 97.6% 97.0% 94.2% 30

Difference in shares: Means 10 points 100 points delta 0.01 0.5 1 1.5 2 0.01 0.5 1 1.5 2 RV 33.6% 37.6% 41.6% 53.0% 64.2% 32.6% 51.9% 80.4% 92.2% 96.0% RD 42.2% 46.4% 50.8% 63.5% 75.3% 41.1% 61.8% 87.3% 95.5% 98.0% BL 6.4% 7.5% 8.3% 10.8% 13.8% 2.7% 3.9% 7.3% 10.8% 14.3% DB 12.2% 11.7% 12.1% 12.1% 12.5% 3.2% 4.2% 7.2% 10.8% 13.8% WB 7.1% 7.4% 8.0% 9.5% 11.4% 2.8% 3.9% 7.1% 10.7% 14.1% DM 21.3% 24.4% 27.1% 35.1% 43.8% 7.6% 12.9% 24.9% 36.0% 45.2% WM 6.0% 6.9% 7.6% 9.6% 11.9% 2.7% 3.9% 6.9% 10.1% 13.3% DR 17.8% 20.5% 22.8% 30.0% 37.8% 6.0% 9.9% 19.1% 28.4% 36.9% WR 6.0% 6.8% 7.6% 9.6% 12.1% 2.9% 4.1% 7.2% 10.6% 13.9% DP 9.2% 10.7% 11.4% 13.7% 16.9% 2.1% 2.2% 2.4% 2.8% 4.5% WP 3.5% 3.5% 3.6% 3.2% 3.4% 2.2% 2.5% 3.6% 4.7% 5.3% 31