Computational Complexity (a quick introduction)
Tiling puzzles Can you write a computer program which, given any finite set T of tile types, . e.g., . decides whether T tiles the infinite N × N grid ? – the finite n × n grid ? – what is the run-time of the algorithm (in terms of | T | and n )? is there a faster algorithm? . . Computational complexity 1
Why complexity? Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness Computational complexity 2
Why complexity? Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness Computational complexity 3
Why complexity? Garey & Johnson, 1979 Computers and Intractability: A Guide to the Theory of NP-Completeness Computational complexity 4
Computation model: Turing machines q two-way infinite read/write tape 0 1 1 tape alphabet (e.g., { 0 , 1 } ) Σ states (initial state q 0 , accepting state q a , rejecting state q r ) Q δ : Σ × Q → 2 Σ × Q ×{− 1 , 0 , 1 } transition relation ( a ′ , q ′ , d ) ∈ δ ( a, q ) : when in state q scanning symbol a , write a ′ on that tape cell, move the head in direction d and enter state q ′ nondeterministic! NDTM deterministic if | δ ( a, q ) | ≤ 1 DTM configuration = contents of the tape + state e.g., 01 q 1 Computational complexity 5
Computations of TMs TMs recognise languages, i.e., sets of words (a word is a sequence of letters) initial configuration : initial state q 0 and input word w written in the first | w | cells (the rest is blank) c 0 δ ( a, q ) = { ( a ′ , q ′ , d ′ ) , ( a ′′ , q ′′ , d ′′ ) } c ′ c ′′ 1 1 computation tree branches may be infinite, lead to the accepting state q a or the rejecting state q r M accepts w if there is a path in the computation tree leading to an accepting configuration a configuration with the accepting state q a L ( M ) is the set of all inputs accepted by M (the language of M ) Computational complexity 6
Determinism v. noneterminism – NDTMs are not more powerful than DTMs (they accept they same languages) DTMs can simulate breadth-first search of NDTMs trees – but NDTMs can be exponentially more efficient than DTMs “Whether this exponential loss is inherent or an artefact of our limited understanding of nondeterminism is the famous P = NP ? problem” (Christos Papadimitriou) NDTMs are not a ‘true’ model of computation; it is a very useful abstraction Other models of computation – All other known deterministic models of computation are polynomially reducible to DTMs Not clear for quantum computers, though Computational complexity 7
The Church–Turing thesis The Church-Turing thesis says that any algorithmic procedure that can be carried out by a human being or by a computer can also be carried out by a Turing machine This is not a theorem that can be proved . Why is that? The Church-Turing thesis establishes a connection between the precise, formal notion of a Turing machine and the intuitive, informal notion of an algorithm. It cannot be proved because it is not, in fact can’t be, stated in a precise, formal way. In can rather be considered as a definition of the term ‘ algorithm ’: an algorithmic procedure is what can be carried out by a Turing machine Computational complexity 8
Evidence for the Church–Turing thesis Why should we believe that the thesis is true? Here are some arguments: (1) Turing machines can do many different kinds of algorithmic procedures: compute functions, decide languages, do iterations, simulate finite au- tomata, simulate Turing machines, etc. In fact, no one has yet found an algorithm that happened to be unimplementable by a Turing machine (2) Extensions with multiple tapes and/or heads, non-determinism, counters, etc. all turned out to have the same computational power as standard Turing machines. (3) All other suggested theoretical models of computation have turned out to be provably equivalent to Turing machines. Computational complexity 9
Can the thesis be disproved? In principle, yes. It can happen in the future that somebody finds some kind of a procedure that looks intuitively algorithmic (not only to the ‘finder’ but to everybody • else), and • it is possible to prove in a precise way that the procedure cannot be imple- mented by a Turing machine. So far no one has yet found a procedure that happened to be widely ac- cepted as an algorithm and unimplementable by a Turing machine at the same time. Computational complexity 10
Time and space complexity classes M runs in time T ( n ) if no computation path takes > T ( | w | ) steps before halting M runs in space S ( n ) if no computation path uses > S ( | w | ) worktape cells DT IME ( T ( n )) = { L ( M ) | DTM M runs in time T ( n ) } NT IME ( T ( n )) = { L ( M ) | NDTM M runs in time T ( n ) } DS PACE ( S ( n )) = { L ( M ) | DTM M runs in space S ( n ) } NS PACE ( S ( n )) = { L ( M ) | NDTM M runs in space S ( n ) } basic inclusions, for T ( n ) ≥ log n DT IME ( T ( n )) ⊆ NT IME ( T ( n )) ⊆ DS PACE ( T ( n )) ⊆ NS PACE ( T ( n )) ⊆ DT IME (2 O ( T ( n )) ) DT IME ( T ( n )) � DT IME ( T ( n ) · log 2 ( T ( n )) , DS PACE ( S ( n )) � DS PACE ( S ( n ) · log( S ( n )) Savitch, 1970 : NS PACE ( S ( n )) ⊆ DS PACE ( S ( n ) 2 ) , for S ( n ) ≥ log n Computational complexity 11
Common complexity classes � DT IME ( n c ) = DT IME ( n O (1) ) P = c ≥ 1 NP = NT IME ( n O (1) ) PS PACE = DS PACE ( n O (1) ) = NS PACE ( n O (1) ) E XP T IME = DT IME (2 n O (1) ) NE XP T IME = NT IME (2 n O (1) ) E XP S PACE = DS PACE (2 n O (1) ) = NS PACE (2 n O (1) ) . . . DT IME ( T ( n )) = CO DT IME ( T ( n )) (i.e., DT IME ( T ( n )) is closed under complement ) where L = { 0 , 1 } ∗ \ L CO DT IME ( T ( n )) = { L | L ∈ DT IME ( T ( n )) } Immerman and Szelepcs´ enyi, 1987 : NS PACE ( S ( n )) = CO NS PACE ( S ( n )) , for S ( n ) ≥ log n What about NT IME ( T ( n )) and CO NT IME ( T ( n )) ? Computational complexity 12
Complexity classes R ELEMENTARY . . . . . . 2E XP T IME E XP S PACE E XP T IME PS PACE NPC P N P O N C P Log Space Log Time Computational complexity 13
P: tractable decision problems Cobham-Edmonds thesis: – P captures decision problems with ‘feasible’ decision procedures is n 100 feasible? In practice, problems in P always have ≤ n 3 or ≤ n 5 time algorithms n 2 typical running time Problems in P – linear programming – calculating the greatest common divisor – circuit value problem – determining if a given number is prime (Agrawal, Kayal, Saxena, 2004) not known whether P-complete, in NC Complexity of factorisation is not known (it is in NP ∩ CO NP): hundreds of machines factored a 232-digit number (RSA-768) over 2 years Computational complexity 14
Boolean satisfiability Boolean formulas are constructed from variables p 1 , p 2 , . . . using Boolean connectives ∧ (and) , ∨ (or) and ¬ (not) Example: ( p 1 ∧ p 2 ) ∨ ¬ p 1 interpretation I maps variables p i onto the truth-values T and F p I – I | if, and only if, = p i i = T – I | if, and only if, = ϕ and I | = ϕ ∧ ψ I | = ψ – I | if, and only if, = ϕ or I | = ϕ ∨ ψ I | = ψ – I | if, and only if, = ¬ ϕ I �| = ϕ a formula ϕ is satisfiable if there is an interpretation I such that I | = ϕ Theorem. SAT = { ϕ | ϕ is satisfiable } is in NP (NDTM guesses an interpretation I and checks in polynomial time that I | = ϕ ) How hard is SAT? Is P = NP or not? the first of the 7 Millennium Prize Problems ($1,000,000) Computational complexity 15
The class NP A language L ⊆ { 0 , 1 } ∗ is in NP if there exists a polynomial p : N → N and a polynomial-time DTM M (called the verifier for L ) such that, for every word x ∈ { 0 , 1 } ∗ , ∃ y ∈ { 0 , 1 } p ( | x | ) such that M ( x, y ) = 1 iff x ∈ L such y is called a certificate for x Exercise: Given a graph G and a number k , decide whether it contains a k -clique It is hard to find a solution to a problem (or to prove a theorem), but it is easy to check whether a solution is correct What if P = NP? – Mathematicians and computer scientists would not be needed – Artificial Intelligence software would be perfect – No privacy in the digital domain (encryption schemes would be easily decodable) Computational complexity 16
NP-completeness Language L is reducible to language L ′ if there is a function f : L → L ′ such that w ∈ L iff f ( w ) ∈ L ′ f has to be effectively computable , e.g., P-computable L is NP-complete if L ∈ NP and every L ′ in NP is P-reducible to L Cook 1971, Levin 1973: SAT is NP-complete Proof. Let L ∈ NP. There is an NDTM M that recognises L and runs in time n c , for some c (and uses ≤ n c cells) . s ( i, j, a ) — cell i contains symbol a at step j h ( i, j, q ) — head is in state q while scanning cell i at step j consider the formulas, for 0 ≤ i, j < n c , a ∈ Σ q ∈ Q , � ( s ( i, j + 1 , a ′ ) ∧ h ( i + d, j + 1 , q ′ )) s ( i, j, a ) ∧ h ( i, j, q ) → ( a ′ ,q ′ ,d ) ∈ δ ( a,q ) s ( i, j, a ) ∧ h ( i ′ , j, q ) → s ( i, j + 1 , a ) , for all i � = i ′ for all a ′ � = a s ( i, j, a ) ∧ s ( i, j, a ′ ) → ⊥ , h ( i, j, q ) ∧ h ( i ′ , j, q ′ ) → ⊥ , for all ( i ′ , q ) � = ( i, q ) Computational complexity 17
Recommend
More recommend
