Outline Analysis of Algorithms Definition Aggregate Method - PDF document

Outline Analysis of Algorithms � Definition � Aggregate Method � Accounting Method Amortized Analysis � Potential Method � Examples : Binary Counter and Dynamic Table http://www.cp.eng.chula.ac.th/faculty/spj Cost of A Sequence of Operations Amortized Analysis � Let operation A requires � ( n ) cost in worst-case � The worst-case cost for any sequence of m operations � Calling A m times costs � ( m n ) ? � Average performance of each operation in worst � Not necessary : it may cost O( m n ) case (no probability is involved). � Sometimes worst cases do not happen consecutively in a sequence of calls worst time for a sequence of m ops amortized time = � Actual worst-case cost may be � ( m n ) m http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Example Example � Given a list of n elements � Given a list of n elements � Sort this list m times using InsertionSort � Sort this list m times using SelectionSort � First time : worst-case � ( n 2 ) � First time : worst-case � ( n 2 ) � 2 nd - m th times : worst-case � ( n ) � 2 nd - m th times : worst-case � ( n 2 ) � Total worst-case time : � ( n 2 + mn ) � Total worst-case time : � ( mn 2 ) � Amortized time : � ( n 2 / m + n ) � Amortized time : � ( n 2 ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Disjoint Sets Amortized Analysis time � Union : Union by rank : O(1) � Find : Path compression : O( log n ) �� worst � A sequence of m Union and Find operations �� amortized perform on n elements : O( m log * n ) � Amortized cost : O( log * n ) �� actual 2 2 2 log* 2 4 � m number of operations http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Amortized Analysis Techniques Aggregate Method � Aggregate Method � Compute the worst-case time T ( m ) in total of a sequence of m operations � Accounting Method � Amortized cost = T ( m ) / m � Potential Method � Amortized cost of each operation is the same even when there are several types of operations in the sequence. http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Accounting Method Potential Method � Assign (guess) a potential function for the entire � Assign (guess) amortized cost for each operation data stucture � If an operation�s actual cost is less than its � Potential energy increases if amortized cost amortized cost, the difference is assigned to exceeds actual cost specfic objects in the data structure as credit � Potential energy decreases if amortized cost is � Credit can be used later for operations whose less than actual cost actual cost exceeds their amortized cost � Amortized cost = Actual cost + � ( potential ) � Correct if the credit is nonnegative at all times � Correct if potential is never less than its initial http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Binary Counter Increment Binary Counter � a k -bit binary counter A[0..k-1] Increment( A[0..k-1] ) { � A[k-1] : MSB, A[0] : LSB i = 0 while ( i < k and A[i] = 1 ) � count upward from 0 m times A[i] = 0 i = i+1 � 0 0 0 0 if ( i < k ) � 0 0 0 1 A[i] = 1 } � 0 0 1 0 � 0 0 1 1 Actual cost, Worst cost, Amortized cost = ? � 0 1 0 0 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Actual Cost Increment : Worst Cost � Actual cost is linear in the number of bits flipped � A [ 0.. k -1 ] : k bits � ( k ) � Worst when all bits are flipped � 0 0 0 0 Actual cost � 0 0 0 1 1 � 1 1 1 1 � 0 0 1 0 2 � 0 0 0 0 � 0 0 1 1 1 � m Increment s on an initially zero k-bit counter � 0 1 0 0 3 takes time O( mk ) � 0 1 0 1 1 � 0 1 1 0 2 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Aggregate Method Increment : Accounting Method � Tighten the worst-case cost of m Increment s � Assign 2-baht amortized cost for an Increment � A [0] flips every time : m times � 1 baht for flipping a 0-bit to 1-bit � A [1] flips every other time : m /2 times � A [2] flips every fourth time : m /4 times ... � 1 baht kept at the 1-bit for later flipping back to 0 : m /2 i times � A [ i ] flips every fourth time � The total number of flips in m Increment s is � lg m � � 1 m � 2 m � O ( m ) � � m � i i 0 2 0 2 i i � � � Amortized cost = O( m ) / m = O(1) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 0 1 0 0 0 0 0 0 0 1 0 1 ����� actual cost ) 1 st ����� actual cost ) Increment http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 2 2+0 1 1 0 0 0 0 0 0 0 1 1+1 2 nd Increment ����� actual cost ) ����� actual cost ) 1 st Increment http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 2+1 2+2 1 1 0 0 0 1 0 0 0 1+2 1+2 2 nd Increment 2 nd Increment ����� actual cost ) ����� actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 2+2+1 2+2+2 1 1 1 1 1 1 1 0 0 0 0 1+2+1 1+2+1 3 rd Increment 3 rd Increment ����� actual cost ) ����� actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 2+2+2+0 2+2+2+0 1 1 0 0 0 0 0 0 0 1+2+1+1 1+2+1+2 4 th Increment 4 th Increment ����� actual cost ) ����� actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Accounting Method Increment : Accounting Method ����� amortized cost ) ����� amortized cost ) 2+2+2+1 2+2+2+2 1 1 0 0 1 0 0 0 0 1+2+1+3 1+2+1+3 4 th Increment 4 th Increment ����� actual cost ) ����� actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj

Increment : Accounting Method Increment : Accounting Method � Each Increment costs at most two bahts since at ����� amortized cost ) most one bit is set to �1� 2+2+2+2 � No negative credit happens 1 1 0 0 0 � � (amortized cost) � �� (actual cost) at all times • Increment is O(1) amortized time 1+2+1+3 4 th Increment ����� actual cost ) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Potential Method Potential Function � Let � i be potential function of the data structure � Potential energy increases if a i > c i after the i th operation � Potential energy decreases if a i < c i � Let c i be the actual cost of the i th operation � i = � i -1 + ( a i - c i ) � � Let a i be the amortized cost of the i th operation � a i = c i + � i - � i -1 � �� a i = �� c i + �� � i - �� � i -1 � = �� c i + � m - � 0 � �� a i � �� c i if � m ���� 0 http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj Increment : Potential Method Increment : Potential Method � Let � i = b i , � 0 = 0, � i � � 0 � Let b i be the number of 1�s in the counter after after the i th Increment � � i - � i -1 = b i - b i -1 � Suppose that the i th operation resets r i bits �� ( b i -1 - r i + 1) - b i -1 � c i ��� r i + 1 0 1 0 1 1 b 11 = 3 � = 1 - r i b i � b i -1 - r i + 1 0 1 1 0 0 b 12 = b 11 - r 12 + 1 � a i = c i + � i - � i -1 = 3 - 2 + 1 = 2 � ( r i + 1) + 1 - r i � c 12 = 3 � = 2 = O(1) http://www.cp.eng.chula.ac.th/faculty/spj http://www.cp.eng.chula.ac.th/faculty/spj