Amortized Analysis Carola Wenk Slides by courtesy of Charles - PowerPoint PPT Presentation

CMPS 6610 – Fall 2018 Amortized Analysis Carola Wenk Slides by courtesy of Charles Leiserson with additions by Carola Wenk CMPS 6610 Algorithms 1

Dynamic tables Task: Store a dynamic set in a table/array. Elements can only be inserted, and all inserted elements are stored in one contiguous part in the array. The table should be as small as possible, but large enough so that it won’t overflow. Problem: We may not know the proper size in advance! Solution: Dynamic tables. IDEA: Whenever the table overflows, “grow” it by allocating (via malloc or new ) a new, larger table. Move all items from the old table into the new one, and free the storage for the old table. CMPS 6610 Algorithms 2

Example of a dynamic table 1. I NSERT 1 2. I NSERT overflow CMPS 6610 Algorithms 3

Example of a dynamic table 1. I NSERT 1 2. I NSERT overflow CMPS 6610 Algorithms 4

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 CMPS 6610 Algorithms 5

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT overflow CMPS 6610 Algorithms 6

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT overflow CMPS 6610 Algorithms 7

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT CMPS 6610 Algorithms 8

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT 3 4. I NSERT 4 CMPS 6610 Algorithms 9

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT 3 4. I NSERT 4 5. I NSERT overflow CMPS 6610 Algorithms 10

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT 3 4. I NSERT 4 5. I NSERT overflow CMPS 6610 Algorithms 11

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT 3 4. I NSERT 4 5. I NSERT CMPS 6610 Algorithms 12

Example of a dynamic table 1. I NSERT 1 2. I NSERT 2 3. I NSERT 3 4. I NSERT 4 5. I NSERT 5 6 6. I NSERT 7 7. I NSERT CMPS 6610 Algorithms 13

Worst-case analysis Consider a sequence of n insertions. The worst-case time to execute one insertion is  ( n ). Therefore, the worst-case time for n insertions is n ꞏ  ( n ) =  ( n 2 ). NO! In fact, the worst-case cost for n insertions is only  ( n )  ( n 2 ). Let’s see why. CMPS 6610 Algorithms 14

Tighter analysis Let c i = the cost of the i th insertion i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 c i CMPS 6610 Algorithms 15

Tighter analysis Let c i = the cost of the i th insertion 1 + cost to double array size = i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 1 1 1 1 1 1 1 1 1 1 c i ? ? ? ? ? ? ? ? ? ? CMPS 6610 Algorithms 16

Tighter analysis Let c i = the cost of the i th insertion 1 + cost to double array size = i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 1 1 1 1 1 1 1 1 1 1 c i 0 1 2 0 4 0 0 0 8 0 CMPS 6610 Algorithms 17

Tighter analysis Let c i = the cost of the i th insertion 1 + cost to double array size = i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 1 1 1 1 1 1 1 1 1 1 c i 1 2 3 1 5 1 1 1 9 1 0 1 2 0 4 0 0 0 8 0 CMPS 6610 Algorithms 18

Tighter analysis (continued) n   c Cost of n insertions i  i 1    log( n 1 )    j n 2  j 0  3 n .   ( n ) Thus, the average cost of each dynamic-table operation is  ( n )/ n =  (1). CMPS 6610 Algorithms 19

Amortized analysis An amortized analysis is any strategy for analyzing a sequence of operations: • compute the total cost of the sequence, OR • amortized cost of an operation = average cost per operation, averaged over the number of operations in the sequence • amortized cost can be small, even though a single operation within the sequence might be expensive CMPS 6610 Algorithms 20

Amortized analysis Even though we’re taking averages, however, probability is not involved! • An amortized analysis guarantees the average performance of each operation in the worst case . CMPS 6610 Algorithms 21

Types of amortized analyses Three common amortization arguments: • the aggregate method, • the accounting method, • the potential method. We’ve just seen an aggregate analysis. The aggregate method, though simple, lacks the precision of the other two methods. In particular, the accounting and potential methods allow a specific amortized cost to be allocated to each operation. CMPS 6610 Algorithms 22

Accounting method • Charge i th operation a fictitious amortized cost ĉ i , where $1 pays for 1 unit of work ( i.e. , time). • This fee is consumed to perform the operation, and • any amount not immediately consumed is stored in the bank for use by subsequent operations. • The bank balance must not go negative! We must ensure that n n    c c ˆ i i   i 1 i 1 for all n . • Thus, the total amortized costs provide an upper bound on the total true costs. CMPS 6610 Algorithms 23

Accounting analysis of dynamic tables Charge an amortized cost of ĉ i = $3 for the i th insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Example: $0 $0 $0 $0 $2 $2 $2 $2 overflow CMPS 6610 Algorithms 24

Accounting analysis of dynamic tables Charge an amortized cost of ĉ i = $3 for the i th insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Example: overflow $0 $0 $0 $0 $0 $0 $0 $0 CMPS 6610 Algorithms 25

Accounting analysis of dynamic tables Charge an amortized cost of ĉ i = $3 for the i th insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Example: $0 $0 $0 $0 $0 $0 $0 $0 $2 $2 $2 CMPS 6610 Algorithms 26

Accounting analysis (continued) Key invariant: Bank balance never drops below 0. Thus, the sum of the amortized costs provides an upper bound on the sum of the true costs. i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 c i 1 2 3 1 5 1 1 1 9 1 ĉ i 2 3 3 3 3 3 3 3 3 3 * bank i 1 2 2 4 2 4 6 8 2 4 *Okay, so I lied. The first operation costs only $2, not $3. CMPS 6610 Algorithms 27

Potential method I DEA : View the bank account as the potential energy ( à la physics) of the dynamic set. Framework: • Start with an initial data structure D 0 . • Operation i transforms D i –1 to D i . • The cost of operation i is c i . • Define a potential function  : { D i }  , such that  ( D 0 ) = 0 and  ( D i )  0 for all i . • The amortized cost ĉ i with respect to  is defined to be ĉ i = c i +  ( D i ) –  ( D i –1 ). CMPS 6610 Algorithms 28

Understanding potentials ĉ i = c i +  ( D i ) –  ( D i –1 ) potential difference  i • If  i > 0, then ĉ i > c i . Operation i stores work in the data structure for later use. • If  i < 0, then ĉ i < c i . The data structure delivers up stored work to help pay for operation i . CMPS 6610 Algorithms 29

The amortized costs bound the true costs The total amortized cost of n operations is n n          c c ( D ) ( D ) ˆ  i i i i 1   i 1 i 1 Summing both sides. CMPS 6610 Algorithms 30

The amortized costs bound the true costs The total amortized cost of n operations is n n          c c ( D ) ( D ) ˆ  i i i i 1   i 1 i 1 n       c ( D ) ( D ) i n 0  i 1 The series telescopes. CMPS 6610 Algorithms 31

The amortized costs bound the true costs The total amortized cost of n operations is n n          c c ( D ) ( D ) ˆ  i i i i 1   i 1 i 1 n       c ( D ) ( D ) i n 0  i 1 n   c since  ( D n )  0 and i  ( D 0 ) = 0.  i 1 CMPS 6610 Algorithms 32

Potential analysis of table doubling Define the potential of the table after the i th insertion by  ( D i ) = 2 i – size i = 2 i – 2  log i  . (Assume that 2  log 0  = 0.)  log i  0 1 2 2 3 3 3 3 4 4 i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 c i 1 2 3 1 5 1 1 1 9 1 ĉ i 2 3 3 3 3 3 3 3 3 3 *  ( D i ) bank i 1 2 2 4 2 4 6 8 2 4   i 1 0 2 -2 2 2 2 -6 2

Potential analysis of table doubling • Potential after i th insertion is  ( D i ) = 2 i – size i • Immediately before an expansion, i = size i , and thus  ( D i )= i . • After an expansion, i – 1 = size i / 2, and thus  ( D i ) = 0 + 2 after insertion of one element.  log i  0 1 2 2 3 3 3 3 4 4 i 1 2 3 4 5 6 7 8 9 10 size i 1 2 4 4 8 8 8 8 16 16 c i 1 2 3 1 5 1 1 1 9 1 ĉ i 2 3 3 3 3 3 3 3 3 3 *  ( D i ) bank i 1 2 2 4 2 4 6 8 2 4  ( D i ) –  ( D i ) 1 0 2 -2 2 2 2 -6 2