Turing Machines Lecture 8 1 Course Trajectory We will see - PowerPoint PPT Presentation

Turing Machines Lecture 8 1

Course Trajectory We will see algorithms, what can be done. But what cannot be done? CS 374 2

Computation Problem : 
 Program : 
 To compute a function F that A finitely described process maps each input (a string) to taking a string as input, and an output bit outputting a bit (or not halting) P computes F if for every x, P(x) outputs F(x) and halts 1 ε • A program is a finite bit string 2 0 3 1 • Programs can be enumerated — listed 4 00 5 sequentially — (say, lexicographically) so 01 6 10 that every program appears somewhere in 7 11 the list 8 000 CS 374 9 001 • The set of all programs is countable. 10 010 11 011 3 12 100

Computation Problem : 
 Program : 
 To compute a function F that A finitely described process maps each input (a string) to taking a string as input, and an output bit outputting a bit (or not halting) P computes F if for every x, P(x) outputs F(x) and halts 1 0 ε • A function assigns a bit to each finite string 2 0 0 3 1 1 • Corresponds to an infinite bit string 4 0 00 5 1 01 • The set of all functions is uncountable! 6 1 10 7 0 11 • As numerous as, say, real numbers 
 8 0 000 CS 374 9 1 in [0,1] 001 10 1 010 11 0 011 4 12 1 100

Computation Problem : 
 Program : 
 To compute a function F that A finitely described process maps each input (a string) to taking a string as input, and an output bit outputting a bit (or not halting) P computes F if for every x, P(x) outputs F(x) and halts There are uncountably many But only countably many functions! programs CS 374 Almost every function is uncomputable! (non constructive proof) 5

Course Trajectory We will be looking at what can be computed at all? What cannot be decided (undecidability) CS 374 6

What about a particular problems? • Given program P , input w : w P Will halt or run into an infinite loop? CS 374 Halting problem! 7

Halting Problem Undecidable • Given program P : P P • Write a program that decides if a program halts. • Dual view of program as program and as data. CS 374 8

Alas! There is no program that solves the Halting Problem! Way to view code as data allows diagonalization proof. 
 CS 374 9

Computing What does it mean to compute something? “There is no algorithm for the halting problem” What does it mean for something to be an algorithm formally? Then I can say “For all algorithms…” CS 374 10

A Brief History of Computing • Leibnitz (circa 1600): Believed in universal language for encoding any problem (math, philosophy, religion). • Thought if you properly encode any problem in binary form, there is a way to calculate an answer (run the algorithm) calculemus! CS 374 11

A Brief History of Computing • Babbage (circa 1860): built Difference and Analytical Engine (Add, multiply, etc). • It marks the transition from mechanised arithmetic to fully-fledged general purpose computation. • Hypothesized that any mathematical question can be answered by the analytical engine, suitably encoded. CS 374 12

A Brief History of Computing • Hilbert (circa 1900): What can we prove in mathematical world? • Proving and computing almost identical, proof= trace the program. • The Entscheidungsproblem (descision problem): is there an algorithm that takes as input a statement of a first-order logic and answers "Yes" or "No" according to whether the statement is universally valid , i.e., valid in every structure satisfying the axioms. CS 374 13

A Brief History of Computing • Gödel: no! there is no such algorithm • Church: better way to prove it with functional programming CS 374 14

Alan Turing • British mathematician – cryptanalysis during WWII – arguably, father of AI, CS Theory – several books, movies • Mathematically defined computation – Invented Turing Machines at 23 (1936). Turing machines can compute everything that is computable. He proved that The Halting Problem has no general CS 374 algorithm (it is not possible to decide whether a turing machine will ever halt) 15

Computation • Computers were people at that time! • The way people do math is write-erase-throw away. • Turing proposed to abstract this process. CS 374 16

Turing Machine CS 374

Turing Machine Finite alphabet Read Write Move +1 or -1 Halt condition CS 374

Turing Machine Finite number of internal states Finite alphabet Read Was designed as a model of human computation but it models computers as Write we know them Move +1 or -1 Halt condition CS 374 Internal state (finite number)

Why do we care? • I need a model of computation that is simple enough to convince you that a TM can take as input a description of a TM and simulate it. • Python interpreter in Python? Harder to explain Python semantics • Will be able to define “what can we compute?” CS 374

TM for Decision Problems M = ( Q, Σ , Γ , B, δ , q start, , q accept , q reject ) : Γ is a finite tape alphabet. - B or ⌷ is the blank symbol (special symbol) - Σ is a finite input alphabet Σ ⊆ Γ \ B Q is a finite set of states q start ∈ Q is the initial state q accept , q reject ∈ Q accept/reject states CS 374 Or maybe run forever 21

TM for Decision Problems M = ( Q, Σ , Γ , B, δ , q start, , q accept , q reject ) : Γ is a finite tape alphabet. - B or ⌷ is the blank symbol (special symbol) - Σ is a finite input alphabet Σ ⊆ Γ \ B Q is a finite set of states q start ∈ Q is the initial state q accept , q reject ∈ Q accept/reject states CS 374 Or maybe run forever Transition function: δ : Q × Γ (read) → Q × Γ (write) × { L, R } 22

Turing Machine pink 0 1 0 1 1 tape = string in Γ * followed by infinite stream of ⌷ we will treat 011111=011111 ⌷ ⌷ ⌷ ⌷… configuration = state, string (content of tape), and integer (position of tape). (Q,x,i) ∈ Q x Γ *x N CS 374 e.g. (pink, 01011, 5)


 
 
 Configuration = ID (Instantaneous Description) Contains all necessary information to capture the 
 “current configuration of the computation” state, tape-contents & head-location a: symbol TM is about to read Easy-to-read notation: ( q,xay,i) 
 x ∈ Γ * : tape contents left of the head 
 q ∈ Q : state 
 y ∈ Γ * : tape contents at & right of the head 
 (till last non-blank) CS 374 Initial ID: ( q start , ⟨ input ⟩ ,0) 24

Relations ⇒ & ⇒ * on IDs ID 1 ⇒ ID 2 iff ID 1 evolves into ID 2 in one step. e.g., if δ ( q, a i ) = ( q ʹ ,b, L ) , then 
 ( q,a 1 a 2 … a i -1 a i a i +1 … a n, i) ⇒ (q ʹ ,a 1 a 2 … a i -2 a i -1 b a i +1 … a n ,i-1) current ID next ID ⇒ * i s the reflexive & transitive closure of ⇒ CS 374 Thus, ID 1 ⇒ * ID 2 iff M , when run from ID 1 , reaches ID 2 after some finite number (0 or more) of moves 25

Accept/Reject 0 1 0 1 1 In DFAs it was clear when to accept an input string or reject it. TM can: • go back and forth over the input • overwrite the input CS 374 • write on the tape way past the end of the input • need an explicit state


 Definition of Acceptance M accepts w iff ( q start , w, 0) ⇒ * (q accept , x,i) 
 for some x ∈ Γ * Note that M is allowed to accept w without scanning all of w L ( M ) = { w | M accepts w } M does not accept w if starting from the ID q start w : 1. M halts in q reject , or 
 2. M crashes (head moves off the tape), or 
 CS 374 3. M never stops 27

Deciding/Recognizing a Language L ( M ) = { w | M accepts w } is called the language recognized by M M decides L ( M ) if on input w ∉ L , M halts in q reject If a TM decides the language it recognizes, 
 then, on every input, it halts in q accept or q reject Easy to change “crashes” to rejects But turns out the we can’t avoid infinite CS 374 executions! (can’t tell if it is going to be infinite) 28

Deciding/Recognizing a Language L ( M ) = { w | M accepts w } is called the language recognized by M M decides L ( M ) if on input w ∉ L , M halts in q reject Fundamental questions of computability: Recursively 
 Which languages are recognizable? Enumerable 
 Language Which languages are decidable? Recursive 
 CS 374 Language 29

Example Input alphabet : Σ = {0,1} Tape alphabet : Γ = {0,1,0 ʹ , 1 ʹ , _} What does this TM do? 0 / 0, R 1 / 1, R _ / 0, L 0 / 0 ʹ , R 0 / 0, L q 3 1 / 1 ʹ , R 1 / 1, L 0 ʹ / 0, R q 2 q 1 0 / 0, R _ / _, L 1 ʹ / 1, R 1 / 1, R CS 374 q 4 _ / 1, L

Example 0 0 1 _ _ _ _ _ _ _ _ q 1 Input alphabet : Σ = {0,1} 0 ʹ 0 1 _ _ _ _ _ _ _ _ 0 ʹ 0 ʹ 1 Tape alphabet : Γ = {0,1,0 ʹ , 1 ʹ , _} _ _ _ _ _ _ _ _ 0 ʹ 0 ʹ 1 ʹ _ _ _ _ _ _ _ _ q 2 0 ʹ 0 ʹ 1 ʹ _ _ _ _ _ _ _ _ 0 / 0, R 1 / 1, R q 4 0 ʹ 0 ʹ 1 _ _ _ _ _ _ _ _ q 2 0 ʹ 0 ʹ 1 1 _ _ _ _ _ _ _ _ / 0, L 0 ʹ 0 ʹ 1 1 _ _ _ _ _ _ _ 0 / 0 ʹ , R 0 / 0, L q 3 1 / 1 ʹ , R 1 / 1, L 0 ʹ 0 1 1 _ _ _ _ _ _ _ q 3 0 ʹ 0 1 1 _ _ _ _ _ _ _ 0 ʹ / 0, R 0 ʹ 0 q 2 1 1 _ _ _ _ _ _ _ q 1 0 / 0, R _ / _, L 1 ʹ / 1, R 1 / 1, R q 2 0 ʹ 0 1 1 0 _ _ _ _ _ _ CS 374 0 ʹ 0 1 1 0 _ _ _ _ _ _ 0 ʹ 0 1 1 0 _ _ _ _ _ _ q 4 _ / 1, L 0 ʹ 0 1 1 0 _ _ _ _ _ _