7 th Central and Eastern European Software Engineering Conference in Russia - CEE-SECR 2011 October 31 – November 3, Moscow Software Reliability Estimation Based on Static Error Detection M. Moiseev, M. Glukhikh , A. Karpenko, H. Richter
Importance of Software Reliability Analysis Errors can lead Modern software to disasters contains errors Software Reliability Analysis Error detection should be organized Software Reliability Estimation Based on Static 2 Error Detection
Known Approaches Architecture- Heuristics Dynamic based approach approaches approach Development Program process metrics Software Reliability Estimation Based on Static 3 Error Detection
Known Approaches – Program Metrics � Based on simple code properties, such as � number of statements � number of conditions � number of loops � number of functions � ... Software Reliability Estimation Based on Static 4 Error Detection
Known Approaches – Development Process Metrics � Based on development process properties, such as � duration of development � number & qualification of developers � number & qualification of testers � methodology used � automation tools used Software Reliability Estimation Based on Static 5 Error Detection
Known Approaches – Others � Runtime � Based on failures observed at run-time � Architecture-based � Based on known reliability of program components Software Reliability Estimation Based on Static 6 Error Detection
Our Approach � Based on source code static analysis � Delivers � Ranking of errors (based on failure probability) � Reliability characteristics � Limitations � Single-threaded C programs � Error types � uninitialized variable use � incorrect pointer dereference � pointer out of bounds Software Reliability Estimation Based on Static 7 Error Detection
Features of Our Approach � Analysis of a program model � Analysis of all possible execution paths � Advantages � Reliability estimations is based on real errors � Results are applicable for any exploitation conditions � Makes debugging more effective � Drawbacks � Does not consider quantitative time � Does not consider normal program exploitation � Execution path probability estimation � False positives problem Software Reliability Estimation Based on Static 8 Error Detection
Program Classes Programs Server Computational Software Reliability Estimation Based on Static 9 Error Detection
Reliability characteristics used � Computational programs � Probability of whole program successful execution P( ∞ ) � Server programs � Probability of n statements successful execution P(n) � Mean executed statement n number before failure Software Reliability Estimation Based on Static 10 Error Detection
Algorithms Model building State determination Error detection Error ranking Reliability estimation Software Reliability Estimation Based on Static 11 Error Detection
Program Model Features � Control flow graph � Three-operand assignment form A = B op C � If and Phi statements If Phi If Phi Software Reliability Estimation Based on Static 12 Error Detection
State Determination Algorithms � State representation � Control flow analysis � Statement analysis � Sequential � If statement analysis � Phi statement analysis � Loop analysis � Interprocedural analysis Software Reliability Estimation Based on Static 13 Error Detection
Program State Representation � Based on objects, values, and probabilities { ( ) } = Q o v p � set of triples , , j k jk ( ) P Q � state probability � Object values � intervals � pointers � resource descriptors Software Reliability Estimation Based on Static 14 Error Detection
Probability normalization � Control flow normalization ( ) ( ) ∑ ∑ in = out P Q P Q j j ( ) ( ) in out ∀ ∈ ∈ Q Input s Q Output s j j � State normalization ( ) ( ) ∑ ∀ ∃ ∈ ⇒ = o o v p Q p P Q : , , j j k jk jk ( ) ∀ ∈ o v p Q , , j k jk Software Reliability Estimation Based on Static 15 Error Detection
Sequential Statement Analysis 1 1 b b , 1 , , , 2 , 2 2 ( ( ) ) b , 1 .. 2 , 1 , 1 1 Q in = = c c , 3 , , , 4 , , ( ( ) ) c , 3 .. 6 , 1 ,... 4 4 1 1 c c , 5 , , , 6 , ,... 4 4 a = b + c 1 1 a a , 4 , , , 5 , , 8 4 Q out = { ( ( ) ) } 1 1 = a a a , 4 .. 8 , 1 ,... , 6 , , , 7 , , 4 4 1 a , 8 , ,... 8 Software Reliability Estimation Based on Static 16 Error Detection
If Statement Analysis � True and false combination consideration ( ) ∑ ∏ = true P Q p , jk ( ) true ∈ o v p ∈ c c C , , j k jk ( ) ∑ ∏ false = P Q p . jk ( ) false ∈ ∈ o v p c c C , , j k jk � Normalization of state probabilities � Normalization of non-affected triples probabilities Software Reliability Estimation Based on Static 17 Error Detection
If Statement Analysis Example � 172 combinations where a < b � 28 combinations where a >= b ( ( ) ) a , 1 .. 10 , 1 , Q in = ( ( ) ) b , 4 .. 23 , 1 ,... false true Q Q ( ( ) ) ( ( ) ) a a , 4 .. 10 , 0 . 14 , 1 .. 10 , 0 . 86 ( ( ) ) ( ( ) ) b b , 4 .. 10 , 0 . 14 , 4 .. 23 , 0 . 86 � Normalization: 0.86 for true, 0.14 for false Software Reliability Estimation Based on Static 18 Error Detection
Phi Statement Analysis � Identical triples are added together ( ) ( ) ∀ ∈ ∈ ⇒ in in o v o v p Q o v r Q , : , , , , , j k j k jk j k jk 1 2 ( ) + ∈ out o v p r Q , , j k jk jk ( ) ( ) ( ) out = in + in P Q P Q P Q � Control flow normalization 1 2 In 1 In 2 Phi Software Reliability Estimation Based on Static 19 Error Detection
Error Detection � Based on incorrect values in state ( ) o v p � uninitialized variable use , , j noninit k ( ) o v p � pointer dereference , , j noninit k ( ) ( ) o v p o v p , , , , j invalid k j null k ( ( ) ) o o offset p , , , � out of bounds i j j k ( ) ≤ offset < sizeof o 0 � correct if j j � otherwise error is detected Software Reliability Estimation Based on Static 20 Error Detection
Error Inhibition (obj, valid, p1) (obj, invalid, p2) P(Q)=p1+p2 obj use (obj, valid, p1) P(Q)=p1 Software Reliability Estimation Based on Static 21 Error Detection
Error Ranking � Errors are sorted according to probability of occurrence � Most dangerous errors can be corrected first � Probabilities are summarized for same errors in the same statement Software Reliability Estimation Based on Static 22 Error Detection
Overall reliability estimation � probability of successful execution ( ) ∑ = P n P Q ( ) end statements � probability of n statements successful execution ( ) ∑ = P n P Q ( ) n statements executed � mean executed statements number before failure n ( ) ∑ max = − + ⋅ n P n P n n ( ) ( 1 ) = n 0 Software Reliability Estimation Based on Static 23 Error Detection
Implementation � AEGIS static analyzer � analysis of C/C++ source code � interval, points to, resource analysis � loop & interprocedural analysis � spread range of program errors detected � Results � error ranking table � P(n) table � P( ∞ ) � mean executed statements number before failure Software Reliability Estimation Based on Static 24 Error Detection
Experiments made � Purpose � Testing of our approach � Debugging example Test programs Students' Real-world projects projects (embedded software) Software Reliability Estimation Based on Static 25 Error Detection
Sample of reliability analysis while (!(feof(f))) // 0.5 { i = t = 0; // Failure in one of three cases prov(&t, strlen(st), st); } � Probability of successful execution is 0.75 = 0.5 + 0.5 * 0.33 + 0.5 * 0.33 2 + ... Software Reliability Estimation Based on Static 26 Error Detection
Amount of errors in real-world projects More than 500 errors, 2/3 of considered types � Density about 0.8/1KLOC � 100 80 60 rror number 40 E 20 0 A B C D E F G H I J K L Projec t name Software Reliability Estimation Based on Static 27 Error Detection
Distribution of error number 175 150 125 100 rror number 75 E 50 25 0 1.E -06 1.E -05 1.E -04 1.E -03 1.E -02 1.E -01 0,25 0,5 1,0 E rror probability Software Reliability Estimation Based on Static 28 Error Detection
Recommend
More recommend
