Probabilistic Bisection Search for Stochastic Root Finding Rolf - PowerPoint PPT Presentation

Probabilistic Bisection Search for Stochastic Root Finding Rolf Waeber Peter I. Frazier Shane G. Henderson Operations Research & Information Engineering Cornell University, Ithaca, NY Research supported by AFOSR YIP FA9550-11-1-0083, NSF CMMI 1200315

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Shameless Commerce www.simopt.org Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 2/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Problem X* g(x) 0 0 1 • Consider a function g : [ 0 , 1 ] → R . • Assumption: There exists a unique X ∗ ∈ [ 0 , 1 ] such that • g ( x ) > 0 for x < X ∗ , • g ( x ) < 0 for x > X ∗ . Goal: Find X ∗ ∈ [ 0 , 1 ] . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 3/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Problem X* Y n (X n ) 0 0 1 • Consider a function g : [ 0 , 1 ] → R . • Assumption: There exists a unique X ∗ ∈ [ 0 , 1 ] such that • g ( x ) > 0 for x < X ∗ , • g ( x ) < 0 for x > X ∗ . Goal: Find X ∗ ∈ [ 0 , 1 ] . • Can only observe Y n ( X n ) = g ( X n ) + ε n ( X n ) , where ε n ( X n ) is a conditionally independent noise sequence with zero mean (median). Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 3/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Problem X* Y n (X n ) 0 0 1 • Consider a function g : [ 0 , 1 ] → R . • Assumption: There exists a unique X ∗ ∈ [ 0 , 1 ] such that • g ( x ) > 0 for x < X ∗ , • g ( x ) < 0 for x > X ∗ . Goal: Find X ∗ ∈ [ 0 , 1 ] . • Can only observe Y n ( X n ) = g ( X n ) + ε n ( X n ) , where ε n ( X n ) is a conditionally independent noise sequence with zero mean (median). Decisions: • Where to place samples X n for n = 0 , 1 , 2 , . . . • How to estimate X ∗ after n iterations. Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 3/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Applications • Simulation optimization: • g ( x ) as a gradient • Finance: • Pricing American options • Estimating risk measures • Computer science: • Edge detection • Image detection and tracking Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 4/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Approximation [Robbins and Monro, 1951] X* g(x) 0 0 1 1. Choose an initial estimate X 0 ∈ [ 0 , 1 ] ; 2. Select a tuning sequence ( a n ) n ≥ 0, � ∞ n = 0 a 2 n < ∞ , and � ∞ n = 0 a n = ∞ . (Example: a n = d / n for d > 0.) 3. X n + 1 = Π [ 0 , 1 ] ( X n + a n Y n ( X n )) , where Π [ 0 , 1 ] is the projection to [ 0 , 1 ] . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Approximation [Robbins and Monro, 1951] X* g(x) 0 0 1 1. Choose an initial estimate X 0 ∈ [ 0 , 1 ] ; 2. Select a tuning sequence ( a n ) n ≥ 0, � ∞ n = 0 a 2 n < ∞ , and � ∞ n = 0 a n = ∞ . (Example: a n = d / n for d > 0.) 3. X n + 1 = Π [ 0 , 1 ] ( X n + a n Y n ( X n )) , where Π [ 0 , 1 ] is the projection to [ 0 , 1 ] . Stochastic approximation is fragile . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Isotonic Regression 1. Simulate at selected points in the interval ( 0 , 1 ) 2. Minimize a sum of squared deviations from the sample values 3. Subject to a monotonicity constraint 4. Estimate root from regression function 5. Add points as necessary Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 6/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Isotonic Regression 1. Simulate at selected points in the interval ( 0 , 1 ) 2. Minimize a sum of squared deviations from the sample values 3. Subject to a monotonicity constraint 4. Estimate root from regression function 5. Add points as necessary Computationally intensive if warm starts are not possible. Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 6/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding A Different Approach What about a bisection algorithm? X* g(x) 0 0 1 • Deterministic bisection algorithm will fail almost surely. • Need to account for the noise. Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 7/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding The Probabilistic Bisection Algorithm Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 8/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding The Probabilistic Bisection Algorithm [Horstein, 1963] • Input: Z n ( X n ) := sign ( Y n ( X n )) . • Assume a prior density f 0 on [ 0 , 1 ] . n = 0, X n = 0.5, Z n (X n ) = −1 X* 2 f n (x) 1 0 0 1 X n Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding The Probabilistic Bisection Algorithm [Horstein, 1963] • Input: Z n ( X n ) := sign ( Y n ( X n )) . • Assume a prior density f 0 on [ 0 , 1 ] . n = 0, X n = 0.5, Z n (X n ) = −1 n = 1, X n = 0.38462, Z n (X n ) = −1 X* X* 2 2 f n (x) f n (x) 1 1 0 0 0 1 0 1 X n X n Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding The Probabilistic Bisection Algorithm [Horstein, 1963] • Input: Z n ( X n ) := sign ( Y n ( X n )) . • Assume a prior density f 0 on [ 0 , 1 ] . n = 0, X n = 0.5, Z n (X n ) = −1 n = 1, X n = 0.38462, Z n (X n ) = −1 X* X* 2 2 f n (x) f n (x) 1 1 0 0 0 1 0 1 X n X n n = 2, X n = 0.29586, Z n (X n ) = 1 X* 2 f n (x) 1 0 0 1 X n Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding The Probabilistic Bisection Algorithm [Horstein, 1963] • Input: Z n ( X n ) := sign ( Y n ( X n )) . • Assume a prior density f 0 on [ 0 , 1 ] . n = 0, X n = 0.5, Z n (X n ) = −1 n = 1, X n = 0.38462, Z n (X n ) = −1 X* X* 2 2 f n (x) f n (x) 1 1 0 0 0 1 0 1 X n X n n = 2, X n = 0.29586, Z n (X n ) = 1 n = 3, X n = 0.36413, Z n (X n ) = 1 X* X* 2 2 f n (x) f n (x) 1 1 0 0 0 1 0 1 X n X n Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Revisited X* g(x) 0 0 1 � sign ( g ( X n )) with probability p ( X n ) , Z n ( X n ) = − sign ( g ( X n )) with probability 1 − p ( X n ) . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Revisited 1 X* p(x) X* g(x) 0 0.5 0 0 1 0 1 � sign ( g ( X n )) with probability p ( X n ) , Z n ( X n ) = − sign ( g ( X n )) with probability 1 − p ( X n ) . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Revisited 1 X* p(x) X* g(x) 0 0.5 0 0 1 0 1 � sign ( g ( X n )) with probability p ( X n ) , Z n ( X n ) = − sign ( g ( X n )) with probability 1 − p ( X n ) . • The probability of a correct sign p ( · ) depends on g ( · ) and the noise ( ε n ) n . Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32

Waeber, Frazier, Henderson Probabilistic Bisection Search for Stochastic Root Finding Stochastic Root-Finding Revisited 1 X* p(x) X* g(x) p 0 0.5 0 0 1 0 1 � sign ( g ( X n )) with probability p ( X n ) , Z n ( X n ) = − sign ( g ( X n )) with probability 1 − p ( X n ) . • The probability of a correct sign p ( · ) depends on g ( · ) and the noise ( ε n ) n . • Stylized Setting: • p ( · ) is constant. Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32