Foundations*for*Stochastic* Systems* Sriram*Sankaranarayanan* - PowerPoint PPT Presentation

Foundations*for*Stochastic* Systems* Sriram*Sankaranarayanan* University*of*Colorado,*Boulder*

Joint*Work*with* Aleksandar*Chakarov*

Stochastic*Systems* Discrete*Time,** Continuous*Time,** Finite*State* Finite*State* S 1 * 0.9* 0.9* S 1 * 0.8* 0.8* S 0 * S 0 * S 3 * S 2 * S 3 * S 2 * 0.2* 0.2* 0.1* Discrete*Time,** Continuous*Time,** Infinite*State* infinite*State* x 0 = F ( x , w ) d x = f ( x , t ) dt + g ( x , t ) dw

Analyzing*Stochastic*Systems* Random* Outputs* Inputs* Stochastic*System* Goal:*Bounds*on*probability*that*the*system* satisfies*properties.*

Insulin*Infusion* Meals,*Physical* Activity** Blood*Glucose* Glucose** Sensor* Insulin* Noise*+*Delays* Delays*+**Set*Failures* InsulinQInfusion* Insulin* Infusion* Control* Pump* Failure*Probability*<*10 Q6 * [Cameron*et*al.,*DallaQMan*et*al.,*Doyle*et*al.,*Hovorka*et*al.]*

Analysis*of*Stochastic*Systems** Statistical*Guarantees* Mathematical*Guarantees* Model* Statistical* Prob.*Abstract* Deductive* “MonteQCarlo”* Checking* Model** Interpretation* Techniques* Simulations* Checking* PRISM:** McIver+Morgan* [Rubenstein*+*Kroese,* Kwiatkowska*et*al.* Chakarov+S* Younes+Simmons,*Jha*et*al,** Clarke*+*Zuliani,*Legay*et*al,*…]* Monniaux** Cousot*+*Monerau**

Rest*of*the*Talk* • An*Illustrative*(Toy)*Example* – Dubins*Vehicle*on*a*Tarmac* • Concentration*of*Measure* • (Super)*Martingales* • Synthesizing*Super*Martingales* • Concluding*Thoughts*

Example:*Dubins*Vehicle*with* Steering*Errors* θ y 0 y + 0 . 1 θ = Disturb* Dubin’s* Vehicle* θ 0 0 . 99 θ + w = w ∼ Uniform ( − 0 . 01 , 0 . 01) Feedback* SteeringAngle* y ≥ 1 ∧ θ > 0 ( x, y, θ ) y ≤ − 1 ∧ θ < 0

Intermediate*Distributions* n=95* Histogram* of*y*values* n=500* for*Dubins* Vehicle* n=1000* Monniaux,*Kwiatkowska*et* al,**Mardziel*et*al,* S*et*al.,**Abate*et*al.,* Xiu*and*Karandiakis,…*

Deductive*Approach* Deduce*facts*about*the*distributions.* Without*approximating*it.* Martingale* y 0 y + 0 . 1 θ = θ 0 0 . 99 θ + w = y + 10 θ w ∼ Uniform ( − 0 . 01 , 0 . 01) Prajna+Jadbabaie+Pappas’04* McIver+Morgan’06* Steinhardt+Tedrake’13* Chakarov+S’13,’14*

Martingale:*Background* Stochastic*Process* X 0 , X 1 , X 2 , X 3 , . . . Martingale* E ( X n +1 | X n , . . . , X 0 ) = X n E ( X 4 | x 3 , . . . , x 0 ) = x 3 X* x 2 x 3 x 0 x 1 X 4 Time*

Super*Martingales* Martingale* E ( X n +1 | X n ) = X n Super*Martingale* E ( X n +1 | X n ) ≤ X n

Martingale:*Example* Martingale* y 0 y + 0 . 1 θ = θ 0 0 . 99 θ + w = y + 10 θ w ∼ Uniform ( − 0 . 01 , 0 . 01) θ n +1 y n +1 E ( y n +1 + 10 θ n +1 | y n , θ n ) = y n + 0 . 1 θ n + 10(0 . 99 θ n + E ( w n )) = y n + 10 θ n

Azuma’s*Inequality* “Martingales*do*not*stray*too* far*from*their*starting*values”* X n t* X 0 Pr ( X n − X 0 ≥ t ) ⇣ ⌘ t 2 Pr ( X n − X 0 ≥ t ) ≤ exp − 2 nC 2 Azuma’67,*Hoeffding’63*

Application*of*Azuma*Inequality* Probability*that*system*enters*failure* set*within*first*N*steps.* Safe*Set* Initial* Dist.* 1. Find*a*(super)*martingale*f(x)* 2. Bound*f(x)*for*failure*set* 3. Pr*(Enter*Failure*Set)*<=*Pr*(Martingale*exceeds*bound).*

Dubin’s*Car* M : y + 10 θ y ≥ 1 ∧ θ > 0 ( x, y, θ ) y ≤ 1 ∧ θ < 0 failure ⇒ | M | > 1 Azuma*Inequality* Pr ( failure ) ≤ 0 . 013 Bound*

y x Road%Width% Azuma%Bound% MC%Estimate%(10 5 %sims)% 0*(no*failures*seen)* [;1,1]% <=*0.013* [;1.5,1.5]% <=*2.7x10 Q5 * 0*(no*failures*seen)* 0*(no*failures*seen)* [;2,2]% <=*4.2x10 Q9 * [;2.5,2.5]% <=*5.4x10 Q14 * 0*(no*failures*seen)*

Discovering*Martingales* 1. Fix*a*desired*form*for*the*(super)*martingale.* c 1 y + c 2 θ + c 3 y 2 + c 4 y θ + c 5 θ 2 2. Encode*the*conditions*for*being*a* martingale.* 1. Linear*Systems:*Farkas*Lemma*(dualization)* 2. Polynomial*Systems:*SumQOfQSquares* Programming* 3. Bernstein*Polynomials* 3. Solve*to*obtain*(super)*martingales*

Discovering*Super*Martingales* x + 0 . 1(1 − 1 2 θ 2 ) := x := y + 0 . 1 θ y := 0 . 99 θ + 0 . 1 w θ 2 . 985 n + 150 θ 2 − − 2 . 985 x Martingale 10 θ + y Martingale 2000 θ y − 199 n + 100 y 2 + 1990 x Martingale SuperMartingale 49 n − 500 x SuperMartingale 1000 θ − n SuperMartingale 10 x − n − n − 1000 θ SuperMartingale

Beyond*Martingales* E ( X n | X n − 1 ) ≤ X n − 1 Super*Martingales*     X 1 ,n X 1 ,n − 1 Expectation* E  ≤ M     . . Invariants* . .    . . X m,n X m,n − 1 Nonnegative* Matrix* Abstract*Interpretation** techniques*for*discovering* Chakarov*+*S’*2014* Expectation*Invariants*

Concluding*Thoughts* • Martingales*+*Concentration*of*Measure:** – Prove*bounds*on*extremely*rare*events.* – Depends*critically*on*finding*the*“right”*martingale.* – Promising*approaches*[*Previous*Talk* ! *]* • Continuous*Time*Systems:* – Theory*extends*naturally.* – Different*kinds*of*concentration*of*measure* inequalities .* – * [*Prajna+Jadbabaie+Pappas,Steinhardt+Tedrake,**Platzer]*

Support* This*work*was*supported*in*part* by*the*US*National*Science** Foundation*under*award*#s** CNSQ1320069,*CNSQ0953941*and** CNSQ1016994.**All*opinions*expressed* are*those*of*the*speaker*and*not** necessarily*of*NSF.* *