Outline Analysis of Algorithms Definition Aggregate Method - PowerPoint PPT Presentation

Outline Analysis of Algorithms � Definition � Aggregate Method � Accounting Method Amortized Analysis � Potential Method � Examples : Binary Counter and Dynamic Table http://www.cp.eng.chula.ac.th/faculty/spj Cost of A Sequence of Operations Amortized Analysis � Let operation A requires Θ ( n ) cost in worst-case � The worst-case cost for any sequence of m operations � Calling A m times costs Θ ( m n ) ? � Average performance of each operation in worst � Not necessary : it may cost O( m n ) case (no probability is involved). � Sometimes worst cases do not happen consecutively in a sequence of calls worst time for a sequence of m ops amortized time = � Actual worst-case cost may be ο ( m n ) m http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Example Example � Given a list of n elements � Given a list of n elements � Sort this list m times using InsertionSort � Sort this list m times using SelectionSort � First time : worst-case Θ ( n 2 ) � First time : worst-case Θ ( n 2 ) � 2 nd - m th times : worst-case Θ ( n ) � 2 nd - m th times : worst-case Θ ( n 2 ) � Total worst-case time : Θ ( n 2 + mn ) � Total worst-case time : Θ ( mn 2 ) � Amortized time : Θ ( n 2 / m + n ) � Amortized time : Θ ( n 2 ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Disjoint Sets Amortized Analysis time � Union : Union by rank : O(1) � Find : Path compression : O( log n ) Σ worst � A sequence of m Union and Find operations Σ amortized perform on n elements : O( m log * n ) � Amortized cost : O( log * n ) Σ actual 2 2 2 log* 2 4 = m number of operations http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Amortized Analysis Techniques Aggregate Method � Aggregate Method � Compute the worst-case time T ( m ) in total of a sequence of m operations � Accounting Method � Amortized cost = T ( m ) / m � Potential Method � Amortized cost of each operation is the same even when there are several types of operations in the sequence. http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Accounting Method Potential Method � Assign (guess) a potential function for the entire � Assign (guess) amortized cost for each operation data stucture � If an operation�s actual cost is less than its � Potential energy increases if amortized cost amortized cost, the difference is assigned to exceeds actual cost specfic objects in the data structure as credit � Potential energy decreases if amortized cost is � Credit can be used later for operations whose less than actual cost actual cost exceeds their amortized cost � Amortized cost = Actual cost + ∆ ( potential ) � Correct if the credit is nonnegative at all times � Correct if potential is never less than its initial http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Binary Counter Increment Binary Counter � a k -bit binary counter A[0..k-1] Increment( A[0..k-1] ) { � A[k-1] : MSB, A[0] : LSB i = 0 while ( i < k and A[i] = 1 ) � count upward from 0 m times A[i] = 0 i = i+1 � 0 0 0 0 if ( i < k ) � 0 0 0 1 A[i] = 1 } � 0 0 1 0 � 0 0 1 1 Actual cost, Worst cost, Amortized cost = ? � 0 1 0 0 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Actual Cost Increment : Worst Cost � Actual cost is linear in the number of bits flipped � A [ 0.. k -1 ] : k bits Θ ( k ) � Worst when all bits are flipped � 0 0 0 0 Actual cost � 0 0 0 1 1 � 1 1 1 1 � 0 0 1 0 2 � 0 0 0 0 � 0 0 1 1 1 � m Increment s on an initially zero k-bit counter � 0 1 0 0 3 takes time O( mk ) � 0 1 0 1 1 � 0 1 1 0 2 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Aggregate Method Increment : Accounting Method � Tighten the worst-case cost of m Increment s � Assign 2-baht amortized cost for an Increment � A [0] flips every time : m times � 1 baht for flipping a 0-bit to 1-bit � A [1] flips every other time : m /2 times � A [2] flips every fourth time : m /4 times ... � 1 baht kept at the 1-bit for later flipping back to 0 : m /2 i times � A [ i ] flips every fourth time � The total number of flips in m Increment s is  lg m  ∞ 1 m ∑ < 2 m = O ( m ) ∑ m = i i 0 2 0 2 i i = = � Amortized cost = O( m ) / m = O(1) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 0 1 0 0 0 0 0 0 0 1 0 1 Σ ( actual cost ) Σ ( actual cost ) 1 st Increment http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 2 2+0 1 1 0 0 0 0 0 0 0 1 1+1 2 nd Increment Σ ( actual cost ) Σ ( actual cost ) 1 st Increment http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 2+1 2+2 1 1 0 0 0 1 0 0 0 1+2 1+2 2 nd Increment 2 nd Increment Σ ( actual cost ) Σ ( actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 2+2+1 2+2+2 1 1 1 1 1 1 1 0 0 0 0 1+2+1 1+2+1 3 rd Increment 3 rd Increment Σ ( actual cost ) Σ ( actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 2+2+2+0 2+2+2+0 1 1 0 0 0 0 0 0 0 1+2+1+1 1+2+1+2 4 th Increment 4 th Increment Σ ( actual cost ) Σ ( actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method Σ ( amortized cost ) Σ ( amortized cost ) 2+2+2+1 2+2+2+2 1 1 0 0 1 0 0 0 0 1+2+1+3 1+2+1+3 4 th Increment 4 th Increment Σ ( actual cost ) Σ ( actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method � Each Increment costs at most two bahts since at Σ ( amortized cost ) most one bit is set to �1� 2+2+2+2 � No negative credit happens 1 1 0 0 0 � Σ (amortized cost) ≥ Σ (actual cost) at all times • Increment is O(1) amortized time 1+2+1+3 4 th Increment Σ ( actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Potential Method Potential Function � Let Φ i be potential function of the data structure � Potential energy increases if a i > c i after the i th operation � Potential energy decreases if a i < c i � Let c i be the actual cost of the i th operation Φ i = Φ i -1 + ( a i - c i ) � � Let a i be the amortized cost of the i th operation � a i = c i + Φ i - Φ i -1 � Σ a i = Σ c i + Σ Φ i - Σ Φ i -1 � = Σ c i + Φ m - Φ 0 � Σ a i ≥ Σ c i if Φ m ≥ Φ 0 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Potential Method Increment : Potential Method � Let Φ i = b i , Φ 0 = 0, Φ i ≥ Φ 0 � Let b i be the number of 1�s in the counter after after the i th Increment � Φ i - Φ i -1 = b i - b i -1 � Suppose that the i th operation resets r i bits ≤ ( b i -1 - r i + 1) - b i -1 � c i ≤ r i + 1 0 1 0 1 1 b 11 = 3 � = 1 - r i b i ≤ b i -1 - r i + 1 0 1 1 0 0 b 12 = b 11 - r 12 + 1 � a i = c i + Φ i - Φ i -1 = 3 - 2 + 1 = 2 ≤ ( r i + 1) + 1 - r i � c 12 = 3 � = 2 = O(1) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Analysis Loop Dy namic Table Guess � Table that can expand or contract as needed Solve Table_Insert( T, x ) if ( T.size = 0 ) Σ a i ≥ Σ c i ? T = CreateTable( 1 ) N if ( T.num = T.size ) S = CreateTable( 2*T.size ) Y insert all items of T into S O( n ) freeTable( T ) Is a i acceptable ? T = S N insert x into Table[T] Y O( n ) End http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Dynamic Table : Sequence of Inserts Worst Case of n Insertions � A sequence of n Table_Insert on initially � Aggregate Method empty table � Observe that c i = i if i -1 is an exact power of 2, � Double the table size when inserting into the full otherwise c i = 1 n   lg n j ∑ table c ≤ + ∑ 2 n i � Amortized cost is 3 = O(1) 1 i = j 0 = � Worst case of Table_Insert is O( n )  lg  1 n + ≤ + 2 − 1 n � Worst case of the sequence is O( n 2 ) 2 < n + n � Not tight : table expansion occurs infrequently 3 = n http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj