Introduction to Complexity Theory Winter 2005/06 Dr. Axel Gromann - PowerPoint PPT Presentation

Introduction to Complexity Theory Winter 2005/06 Dr. Axel Groÿmann Knowledge Representation and Reasoning Group Artificial Intelligence Institute 1

Computational complexity theory Complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem. The most common resources • time (how many steps it takes to solve a problem) • space (how much memory it takes) Other resources, e.g., • how many parallel processors are needed to solve a problem in parallel Computability theory deals with whether a problem can be solved at all, regardless of the resources required. Axel Großmann, CT 2005/06 2

Problems and instances A single problem is an entire set of related questions, where each question is a finite-length string. The FACTORIZE problem • given an integer written in binary • return all of the prime factors of that number A particular question is called an instance . Instance of the FACTORIZE problem • give the factors of the number 15 Axel Großmann, CT 2005/06 3

Decision problems Much of complexity theory deals with decision problems. A decision problem is a problem where the answer is always YES/NO. The IS-PRIME problem • given an integer written in binary • return whether it is a prime number or not Decision problems are often considered because an arbitrary problem can always be reduced to a decision problem. Axel Großmann, CT 2005/06 4

Complexity classes The complexity class P is the set of decision problems that can be solved by a deterministic machine in polynomial time. This class corresponds to an intuitive idea of the problems which can be effectively solved in the worst cases. The complexity class NP is the set of decision problems that can be solved by a non-deterministic machine in polynomial time. This class contains many problems that people would like to be able to solve effectively, including the Boolean satisfiability problem, All the problems in this class have the property that their solutions can be checked effectively. The question of whether P is the same set as NP is the most important open question in theoretical computer science. Axel Großmann, CT 2005/06 5

History of computational complexity The ancient Greeks • The idea of `efficient algorithms' 1936 • Turing developed hist theoretical computational model based on how he perceived mathematicians think. 1940s/50s • Digital computers were developed. • Turing machine proved itself as the right theoretical model for computation, but fails to account for the amount of time or memory needed by a computer. Axel Großmann, CT 2005/06 6

History of computational complexity (2) Mid of 1960's • Hartmanis and Stearns [1965] introduced the idea to measure time and space as a function of the length of the input. • The birth of computational complexity • Edmonds [1965] argued that polynomial-time gives good formalisation of efficient computation. • NP -completeness at the most important concept and its most fundamental ques- tion, whether P = NP 1970's • Cook and Karp showed a large number of combinatorial and logical problems were NP -complete. • Growth of complexity classes • Probabilistic computation Axel Großmann, CT 2005/06 7

History of computational complexity (3) 1980's • Finite models like circuits 1990's • Quantum computers and propositional proof systems Axel Großmann, CT 2005/06 8

Questions and answers that make up complexity theory Complexity theory is concerned with the study of the intrinsic complexity of computational tasks. Its final goals: • absolute answers about specific computational phenomena (determining the complexity of any well-defined class) • answers about the relation between computational phenomena Failure to resolve questions of the absolute type led to flourish- ing of methods for coping with questions of the relative type. Establishing relations between phenomena seems more revealing than making statements about each phenomenon. Axel Großmann, CT 2005/06 9

Questions and answers that make up complexity theory (2) Complexity theory failed to determine the intrinsic complexity of task such as finding a satisfying assignment to a given (satisfiable) propositional formula of finding a 3-colouring of a given (3-colourable) graph. But is has established that these two seemingly different com- putational tasks are computationally equivalent. Axel Großmann, CT 2005/06 10

The P versus NP question Our daily experience is that it is harder to solve a problem than it is to check the correctness of a solution. Is this experience merely a coincidence or does it represent a fundamental fact of life (or a property of the world)? Could you imagine a world in which solving any problem is not significantly harder than checking a solution to it? Would the term `solving a problem' not lose its meaning in such a hypothetical (and impossible in our opinion) world? The denial of the plausibility of such a hypothetical world (in which "solving" is not harder than "checking") is what ` P different than NP ' actually means, where P represents tasks that are efficiently solvable and NP represents tasks for which solutions can be efficiently checked. Axel Großmann, CT 2005/06 11

References Recommended text: • Christos H. Papadimitriou. Computational Complexity, Addison Wesley, 1994 Other useful texts: • Michael Sipser. Introduction to the Theory of Computation, PWS, 1997 • Michael R. Garey and David S. Johnson. Computers and Intractability, W. H. Freeman, 1979 Axel Großmann, CT 2005/06 12

List of topics arranges by weeks Week 1 (18th Nov.): 2 lectures, 1 tutorial • Introduction (Slides) • Languages for specification and the need for formalism. Algorithms, polynomials and polynomial time algorithms. (Ch. 1) ⊲ Graph reachability ⊲ Maximum flow and matching ⊲ Travelling Salesman problem Week 3 (2nd Dec.): 2 lectures, 1 tutorial • Turing machines as formalised algorithm. Complexity classes. Space complexity and nondeterminism (Ch. 2, except for 2.6) ⊲ TM basics ⊲ TM as algorithms ⊲ TM with multiple strings ⊲ Linear speedup ⊲ Space bounds ⊲ Nondeterministic machines Axel Großmann, CT 2005/06 13

List of topics arranges by weeks (2) Week 5 (16th Dec.): 3 lectures, 1 tutorial • Universal machines and undecidability. (Ch. 3) • Propositional logic. The complexity of satisfiability and validity. (Sec. 4.1 and 4.2) ⊲ Universal TM ⊲ Halting problem ⊲ More undecidability ⊲ Boolean logic ⊲ Boolean expressions ⊲ Satisfiability and validity (except for Horn clauses) Week 8 (20th Jan.): 3 lectures, 1 tutorial • Complexity classes. The Hierarchy theorem. (Sec. 7.1 and 7.2) • Reductions and completeness. (Sec. 8.1 and 8.2) ⊲ Complexity classes ⊲ Hierarchy theorem ⊲ Reductions ⊲ Completeness (Cook's theorem) Axel Großmann, CT 2005/06 14

List of topics arranges by weeks (3) Exam period • Written exam on all courses within the Foundations modules Axel Großmann, CT 2005/06 15