Part II Course Goals and Overview Nikita Borisov (UIUC) CS/ECE 374 - PowerPoint PPT Presentation

Part II Course Goals and Overview Nikita Borisov (UIUC) CS/ECE 374 15 Fall 2019 15 / 33

High-Level Questions Computation, formally. 1 Is there a formal definition of a computer? 1 Is there a “universal” computer? 2 Algorithms 2 What is an algorithm? 1 What is an e ffi cient algorithm? 2 Some fundamental algorithms for basic problems 3 Broadly applicable techniques in algorithm design 4 Limits of computation. 3 Are there tasks that our computers cannot do? 1 How do we prove lower bounds? 2 Some canonical hard problems. 3 Nikita Borisov (UIUC) CS/ECE 374 16 Fall 2019 16 / 33

Course Structure Course divided into three parts: Basic automata theory: finite state machines, regular languages, 1 hint of context free languages/grammars, Turing Machines Algorithms and algorithm design techniques 2 Undecidability and NP-Completeness, reductions to prove 3 intractability of problems Nikita Borisov (UIUC) CS/ECE 374 17 Fall 2019 17 / 33

Goals 1 Algorithmic thinking Learn/remember some basic tricks, algorithms, problems, ideas 2 Understand/appreciate limits of computation (intractability) 3 Appreciate the importance of algorithms in computer science 4 and beyond (engineering, mathematics, natural sciences, social sciences, ...) Nikita Borisov (UIUC) CS/ECE 374 18 Fall 2019 18 / 33

History Nikita Borisov (UIUC) CS/ECE 374 19 Fall 2019 19 / 33

History Muhammad ibn Musa al-Khwarizmi (c.780–c.850) Nikita Borisov (UIUC) CS/ECE 374 19 Fall 2019 19 / 33

Text on Algebra Nikita Borisov (UIUC) CS/ECE 374 20 Fall 2019 20 / 33

Algorithm Description If some one says: “You divide ten into two parts: multiply the one by itself; it will be equal to the other taken eighty-one times.” Computation: You say, ten less a thing, multiplied by itself, is a hundred plus a square less twenty things, and this is equal to eighty-one things. Separate the twenty things from a hundred and a square, and add them to eighty-one. It will then be a hundred plus a square, which is equal to a hundred and one roots. Nikita Borisov (UIUC) CS/ECE 374 21 Fall 2019 21 / 33

Algorithm Description If some one says: “You divide ten into two parts: multiply the one by itself; it will be equal to the other taken eighty-one times.” Computation: You say, ten less a thing, multiplied by itself, is a hundred plus a square less twenty things, and this is equal to eighty-one things. Separate the twenty things from a hundred and a square, and add them to eighty-one. It will then be a hundred plus a square, which is equal to a hundred and one roots. (10 − x ) 2 = 81 x x 2 − 20 x + 100 = 81 x x 2 + 100 = 101 x Nikita Borisov (UIUC) CS/ECE 374 21 Fall 2019 21 / 33

Models of Computation vs Computers Model of Computation: an “idealized mathematical construct” 1 that describes the primitive instructions and other details Computer: an actual “physical device” that implements a very 2 specific model of computation Nikita Borisov (UIUC) CS/ECE 374 22 Fall 2019 22 / 33

First Computer Babbage’s analytical engine—designed in 1837, never built. Nikita Borisov (UIUC) CS/ECE 374 23 Fall 2019 23 / 33

First Program Ada Lovelace’s “Note G” describing how to calculate Bernouilli numbers using the analytical engine. Nikita Borisov (UIUC) CS/ECE 374 24 Fall 2019 24 / 33

First Bug! Ada Lovelace’s “Note G” describing how to calculate Bernouilli numbers using the analytical engine. This version contains a bug! Nikita Borisov (UIUC) CS/ECE 374 25 Fall 2019 25 / 33

Models of Computation vs. Computers Models and devices: Algorithms: usually at a high level in a model 1 Device construction: usually at a low level 2 Intermediaries: compilers 3 How precise? Depends on the problem! 4 Physics helps implement a model of computer 5 Physics also inspires models of computation 6 Nikita Borisov (UIUC) CS/ECE 374 26 Fall 2019 26 / 33

Adding Numbers Problem Given two n -digit numbers x and y , compute their sum. Basic addition 3141 + 7798 10939 Nikita Borisov (UIUC) CS/ECE 374 27 Fall 2019 27 / 33

Adding Numbers c = 0 for i = 1 to n do z = x i + y i z = z + c If ( z > 10 ) c = 1 z = z − 10 (equivalently the last digit of z ) Else c = 0 print z End For If ( c == 1) print c Nikita Borisov (UIUC) CS/ECE 374 28 Fall 2019 28 / 33

Adding Numbers c = 0 for i = 1 to n do z = x i + y i z = z + c If ( z > 10 ) c = 1 z = z − 10 (equivalently the last digit of z ) Else c = 0 print z End For If ( c == 1) print c Primitive instruction is addition of two digits 1 Algorithm requires O ( n ) primitive instructions 2 Nikita Borisov (UIUC) CS/ECE 374 28 Fall 2019 28 / 33

Multiplying Numbers Problem Given two n -digit numbers x and y , compute their product. Grade School Multiplication Compute “partial product” by multiplying each digit of y with x and adding the partial products. 3141 × 2718 25128 3141 21987 6282 8537238 Nikita Borisov (UIUC) CS/ECE 374 29 Fall 2019 29 / 33

Time analysis of grade school multiplication Each partial product: Θ ( n ) time 1 Number of partial products: ≤ n 2 Adding partial products: n additions each Θ ( n ) (Why?) 3 Total time: Θ ( n 2 ) 4 Is there a faster way? 5 Nikita Borisov (UIUC) CS/ECE 374 30 Fall 2019 30 / 33

Fast Multiplication n log n · 4 log ∗ n � � Best known algorithm: O by Harvey and van der Hoeven, published in 2018! Conjecture: there exists an O ( n log n ) time algorithm Nikita Borisov (UIUC) CS/ECE 374 31 Fall 2019 31 / 33

Fast Multiplication n log n · 4 log ∗ n � � Best known algorithm: O by Harvey and van der Hoeven, published in 2018! Conjecture: there exists an O ( n log n ) time algorithm We don’t fully understand multiplication! Computation and algorithm design is non-trivial! Nikita Borisov (UIUC) CS/ECE 374 31 Fall 2019 31 / 33

Aside about O -notation Some previous versions of multiplication are still widely used: Karatsuba algorithm O ( n log 2 3 ) [1962] Sch¨ onhage-Strassen (FFT) O ( n log n log log n ) [1971] Why? Nikita Borisov (UIUC) CS/ECE 374 32 Fall 2019 32 / 33

Aside about O -notation Some previous versions of multiplication are still widely used: Karatsuba algorithm O ( n log 2 3 ) [1962] Sch¨ onhage-Strassen (FFT) O ( n log n log log n ) [1971] urer’s algorithm (2007) O ( n 2 O (log ∗ n ) ) Why? F¨ Nikita Borisov (UIUC) CS/ECE 374 32 Fall 2019 32 / 33

Aside about O -notation Some previous versions of multiplication are still widely used: Karatsuba algorithm O ( n log 2 3 ) [1962] Sch¨ onhage-Strassen (FFT) O ( n log n log log n ) [1971] urer’s algorithm (2007) O ( n 2 O (log ∗ n ) ) Why? F¨ onhage-Strassen for numbers greater than 2 2 64 . . . . beats Sch¨ Nikita Borisov (UIUC) CS/ECE 374 32 Fall 2019 32 / 33