Announcements Final Exam Dates have been Computational Complexity announced Tuesday, November 11 12:30 – 2:30pm Room 70-2690 Conflicts? Let me know. Announcements Plan for this week Today Reminder: Exam 2 1st half TMs and Languages Will be returned after break 2nd Half Computatbility Course evaluations Wednesday Online Complexity Please complete! Problem session. Plan for remainder of quarter Before We Start Note change Any questions? Next week Tuesday: Cook’s Theorem NP-Complete Problems Thursday Final Exam Review Final problem session 1
Our complete picture: The Turing Machine Recursively Enumerable Motivating idea Recursive Build a theoretical a “human computer” Context Sensitive Likened to a human with a paper and pencil that can solve problems in an algorithmic way Context Free The theoretical machine provides a means to Deterministic Context Free determine: If an algorithm or procedure exists for a given problem Finite What that algorithm or procedure looks like How long would it take to run this algorithm or Regular procedure. The Church-Turing Thesis (1936) Decision Problem Any algorithmic procedure that can be Let’s formalize this a bit carried out by a human or group of A decision problem is a problem that has a humans can be carried out by some yes/no answer Turing Machine” Equating algorithm with running on a TM Example: Turing Machine is still a valid Is a given string x a palindrome (Is x ∈ pal?) computational model for most modern Is a given context free language empty? computers. What makes a good Decision Problem algorithm? Running a decision problem on a TM. Suppose we know that a decision problem is decidable, how do we know The problem must first be encoded that there is a good algorithm that Example: solves the problem? Is a given string x a palindrome (Is x ∈ pal?) x is an instance of the probkem Consider running time on a TM Is a given context free language empty? Actually, consider the complexity of a TM Instance of a problem is a CFG…must be encoded. that can solve the problem. 2
Complexity Units Complexity refers to the rate at which the What does T(n) return? storage or time grows as a function of the A “basic operation” can be defined as a input to the algorithm move a TM makes in accepting / rejecting T(n) = time complexity (amount of time an a string algorithm will take based on input) S(n) = space complexity (amount of space an T(n) = number of TM moves algorithm will take based on input) For the rest of this lecture, we’ll only consider T(n) though the discussion can also be applied to S(n) Asymptotic Analysis Asymptotic Analysis Based on the idea that as the input to Big-O (order of) an algorithm gets large T(n) = O(f(n)) if and only if there are constants c 0 and n 0 such that The complexity will become proportional to a known function. T(n) ≤ c 0 f(n) for all n ≥ n 0 What this really means Notations: Eventually, when the size of the input gets O (Big-O) – upper bounds on the complexity large enough, the upper bounds on the runtime Θ (Big-Theta) – tight bounds on the complexity will be proportional to the function f(n). Algorithm Efficiencies Are you a good witch? So what should be the cutoff between a “good” algorithm and a “bad” algorithm? In the 1960s, Jack Edmonds proposed: A “good” algorithm is one whose running time is a polynomial function of the size of the input Other algorithms are “bad” This definition was adopted: A problem is called tractable if there exists a “good” (polynomial time) algorithm that solves it. A problem is called intractable otherwise. 3
Is this a valid cutoff? The class P The class P contains all decision problems that are decidable by an “algorithm” that runs in polynomial time. Does this define a class of languages? Yes… The set of all problems whose encodings of “yes instances” (a language) is recognized by a TM M M recognizes the above language in Polynomial Time. Recall: These languages are recursive The class P The class P Are there actually problems in P? If we take the Church-Turing Thesis to be true: Most everyday algorithms (e.g. sorting, searching) are in P P is robust: Are there problems not in P: The problems contained in P remain in P even Yes, we know that Self-Accepting is not in if we change our basic model of computation. P since it isn’t even solvable A basic TM A Sun, a Mac, even a PC running Windows Vista! Are there decidable problems not in P? Let’s take a look… Satisfiability (SAT) Satisfiability (SAT) INSTANCE Example of an instance A Logical expression containing (x 1 | x 2 | x 3 ) & (x 4 | !x 5 ) & !x 6 & (x 7 | x 8 ) variables x i logical connectors &, |, and ! One Solution: In conjuction normal form (C 1 & C 2 & C 3 … & C n ) PROBLEM x 1 = true x 5 = false Is there an assignment of truth values to each of x 2 = false x 6 = false the variables such that the expression will x 3 = false x 7 = true evaluate to true. x 4 = false x 8 = false 4
Satisfiability (SAT) Satisfiability (SAT) Naïve algorithm to solve Can we do better? Systematically consider all combinations of No known polynomial algorithm assignments of True and False values for Either one doesn’t exist all variables and test the expression One exists and we haven’t found it yet. In the worst case, for n variables T(n) = O (2 n ) Non-deterministic TM Non-deterministic TM Let’s reconsider the NDTM A non-determininstic TM will accept w if there is at least one sequence of Same as the ordinary TM except: The transition function will return a set of triplets moves: (q, x, D) q 0 w * α p β and p ∈ F For each state / symbol combination, 0 or more transitions can be defined. The machine can “choose” which transition to take. Non- deterministic TM δ : Q x Γ → 2 Q x Γ x {R, L} Non-deterministic TM Non-deterministic TM Simulating a NDTM on a TM Simple backtracking to consider all paths through the TM. One way to think about it… A TM “replicates” itself whenever there is a choice. Each “replication” continues computation along a given path 5
Non-deterministic TM The class NP The class NP contains all decision problems that are If we map all paths that can possibly be decidable by a non-deterministic “algorithm” that taken: runs in polynomial time. For each w accepted, there is at least one accepting Number of paths will be exponential sequence that will run in polynomial time. Example: Does this define a class of languages? -- Yes… Suppose at each configuration there are 2 The set of all problems whose encodings of “yes instances” (a language) is recognized by a NDTM M possible moves. M recognizes the above language in Polynomial Time. Number of paths 2 n Are there problems in NP Well, for one SAT is Satisfiability (SAT) The class NP Running SAT on a non-deterministic TM. Clearly P is a subset of NP Step 1: “Guess” a set of boolean assignments to each variable (this is the non-deterministic part – 2 n different choices) Step 2: Evaluate the truth of the entire expression using the guessed assignment (this part is deterministic) NP Can certainly do each step in polynomial time P Is there something in After 40 years of research…nobody here? knows Reducing one language to Reducing one language to another another Formally (for complexity) Worked well for decidability…Let’s use it here. Basic idea Let L 1 and L 2 be languages over Σ 1 and Σ 2 We say L 1 is polynomial time reducible to Take an encoding of one problem L 2 (L 1 ≤ p L 2 ) if Convert it to another problem we know to be in P There exists a Turing computable function or NP f: Σ 1 * → Σ 2 * such that Conversion must be done in polynomial time! x ∈ L 1 iff f(x) ∈ L 2 f can be computed in polynomial time. 6
Recommend
More recommend
