1 Formal Methods for Probabilistic Systems Annabelle McIver Carroll Morgan • Source-level program logic • Meta-theorems for loops • Examples • Relational operational model • Almost-certain termination • “Herman’s Graph” • Probabilistic variant rule • Termination of Herman’s Graph • Herman’s Ring • Termination of Herman’s Ring • Expected time to stability; comparison with model-checking ( PRISM )
2 A variation on Herman’s Ring : “Herman’s Graph” T Herman. Probabilistic self-stabilization. Inf. Proc. Lett. 35(2):63-67, 1990.
3 Herman’s Graph 1/ 4 1/ 3 1/ 6 1/ 4 On every step, each processor decides probabilistically to which neighbour its tokens will go... ...including possibly itself. All (four, in this case) probabilities must be non-zero.
4 Herman’s Graph The decisions are made synchronously .
5 Herman’s Graph And then the moves are made.
6 Herman’s Graph Choose; move.
7 Herman’s Graph Choose; move.
8 Herman’s Graph Choose; move.
9 Herman’s Graph Because all tokens are together, they will now circulate as a group.
10 Herman’s Graph Like this...
11 Herman’s Graph Thus the system is stable . What is the probability that the system will become stable eventually, no matter where it begins? Eventual stability for Herman’s Graph is almost certain .
12 The (probabilistic) variant rule for loops CC Morgan. Proof rules for probabilistic loops. Proc. 3rd BCS Refinement Workshop. Springer, 1996. http://ewic.bcs.org/conferences/1996/refinement/ papers/paper10.htm S Hart, M Sharir and A Pnueli. Termination of probabilistic concurrent programs. TOPLAS 5:356-380, 1983.
13 page 55 If the invariant Inv is true at the beginning of the loop body, then it’s true at the end.
14 page 55 The variant V is bounded below and above — that is, it takes only finitely many values.
15 page 55 The variant V is strictly decreased, by the loop body, with some non-zero probability.
16 The (probabilistic) variant rule for loops at least , then with probability � the variant If the guard will G is true, the strictly invariant Inv decrease. holds, and the variant V has some value N ,
17 The (probabilistic) variant rule for loops at least , then with probability � the variant If the guard will G is true, the strictly invariant Inv decrease. holds, and the variant V has some value N ,
18 The (probabilistic) variant rule for loops For finite-state systems: Choose some integer-valued variant function of the state. If you can show that from every non-terminated state there is a non-zero probability of strict decrease of that variant in the very next step, then you have shown that termination will eventually occur, from any initial state, with probability one. This a consequence of our “paradoxical” Zero-One Law for probabilistic processes: if from every state the probability of eventual termination is bounded away from zero, then in fact that probability is one.
19 What is the variant for Herman’s Graph ?
20 The variant for Herman’s Graph It is the size of the smallest connected subgraph containing all tokens.
21 Herman’s Graph: the variant It is the size of the smallest connected subgraph containing all tokens. The value of the variant is 3 in this example.
22 Herman’s Graph: the variant With probability pqr p tokens will move as shown... ... and pqr is strictly q greater than zero... r ...so that the variant will decrease, from 3 to 2, with non � zero probability . Q.E.D. This variant suffices because it has non-zero probability of decrease on each iteration.
23 Herman’s Ring The system comprises a number of processes, connected in a ring: it is a special case of the graph .
24 Herman’s Ring Normally, a single token will circulate around the ring...
25 Herman’s Ring But occasionally a hardware or software error causes extra tokens to appear.
26 Herman’s Ring ... like this.
27 Herman’s Ring Because of Herman’s clever underlying encoding, however, there can only ever be an odd number of tokens. If extra ones appear, how do we get rid of them?
28 Herman’s algorithm On every “tick”, each token-holding processor flips a coin: if heads , the token is kept; if tails , it is passed downstream. Colliding tokens are annihilated.
29 Herman’s algorithm For example, we might have this, with heads probability 1/ 8, that is heads , heads , tails... heads tails
30 Herman’s algorithm For example, we might have this, with keep probability 1/ 8, that is keep , keep , pass... keep pass
31 Herman’s algorithm keeping ...and so a token moves along... keeping passing
32 Herman’s algorithm kept ...but, afterwards, there are still three of them. kept passed
33 Herman’s algorithm pass Suppose this time, we get pass , keep , pass ... keep pass
34 Herman’s algorithm passing ...again probability 1/ 8... passing keeping
35 Herman’s algorithm passed ...and there is a collision... annihilated
36 Herman’s algorithm ...so that the ring becomes stable once more.
37 Herman’s algorithm ...so that the ring becomes stable once more.
38 Herman’s algorithm ...so that the ring becomes stable once more.
39 Herman’s algorithm Herman’s algorithm has the property that no matter how the ring is perturbed (provided the number of tokens remains odd), it is guaranteed with probability 1 to return “automatically” to a stable state in which there is only one token. In that sense, it is “self-repairing”. How do we prove this is so? And how long does it take for stabilisation to occur?
40 Herman’s proof of eventual stabilisation http://www.cs.uiowa.edu/ftp/selfstab/H90.ps.gz , pp. 6-7.
41 A very short proof of eventual stabilisation We get a much shorter proof — Choose as probabilistic variant the essentially “one line” — by using length of the smallest consecutive the same technique as before, span of ring segments that contains where the hard work has been all tokens; apply Lemma 2.7.1. packaged up in a theorem that can be used over and over. Q.E.D.
42 Expected time to stabilisation The probabilistic variant can also be used to estimate the expected time to stabilisation. The variant effectively performs a random walk on the integers between 0 and N -1. When it is zero, stabilisation has occurred. It is known from probability theory that the expected time for a random walker to move N steps in the same direction is of order N squared. That is thus an upper bound on how long it takes for stabilisation to occur. AK McIver and CC Morgan. An elementary proof that Herman’s Ring is � (N2) . http://web.comlab.ox.ac.uk/oucl/research/areas/probs/ bibliography.html#HR04
43 Herman’s Ring is � (N2) We know already that eventual convergence is assured with probability one. But how long does it take?
44 Herman’s Ring is � (N2) — using program logic Write the ring as a small looping pGCL program, with an extra “counting” variable k initialised to zero and incremented on each iteration; the loop guard is “there is more than one token”; determine the expected final value of k . In principle...
45 Herman’s Ring is � (N2) — using program logic multi-way probabilistic choice k := 0; In practice the program � do n � 0 is rather messy, and the calculations complex — if 0 < n < N -1 � n := | n -1 @ 1/ 4 if one can find the | n @ 1/ 2 invariant at all! | n +1 @ 1/ 4 � n := | n -1 � n = N -1 @ 1/ 4 Instead we abstract , | n @ 3/ 4 using as inspiration the fi ; same variant n that n : � n ; showed eventual termination. k := k +1 od same as n := n -1 1/ 4 � n := n
46 Herman’s Ring is � (N2) — using program logic Part of the abstraction k := 0; however is that we do � do n � 0 not know exactly what if 0 < n < N -1 � n := | n -1 the effect of other @ 1/ 4 collisions might be; | n @ 1/ 2 | n +1 @ 1/ 4 that is represented by � n := | n -1 � n = N -1 @ 1/ 4 the demonic possible | n @ 3/ 4 decrease of the fi ; maximum separation n ; n : � n ; and it is the problem k := k +1 with applying standard od Markov methods .
47 Herman’s Ring is � (N2) — using program logic k := 0; � do n � 0 if 0 < n < N -1 � n := | n -1 @ 1/ 4 | n @ 1/ 2 | n +1 @ 1/ 4 � n := | n -1 � n = N -1 n = 4 @ 1/ 4 | n @ 3/ 4 fi ; n : � n ; k := k +1 od
48 Herman’s Ring is � (N2) — using program logic k := 0; � do n � 0 if 0 < n < N -1 � n := | n -1 @ 1/ 4 | n @ 1/ 2 | n +1 @ 1/ 4 4 � n := | n -1 � n = N -1 @ 1/ 4 | n @ 3/ 4 fi ; n : � n ; k := k +1 od
49 Herman’s Ring is � (N2) — using program logic k := 0; � do n � 0 if 0 < n < N -1 � n := | n -1 @ 1/ 4 | n @ 1/ 2 | n +1 @ 1/ 4 3 � n := | n -1 � n = N -1 @ 1/ 4 | n @ 3/ 4 fi ; n : � n ; k := k +1 od
Recommend
More recommend
