Resource-Bounded Computation Previously: can something be done? - PowerPoint PPT Presentation

Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources? Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w  * within O(t(|w|)) steps / time. Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w  * while never using more than O(s(|w|)) space / tape cells.

Complexity Classes Def: DTIME(t(n))={L | L is decidable within time O(t(n)) by some deterministic TM} Def: NTIME(t(n))={L | L is decidable within time O(t(n)) by some non-deterministic TM} Def: DSPACE(s(n))={L | L decidable within space O(s(n)) by some deterministic TM} Def: NSPACE(s(n))={L | L decidable within space O(s(n)) by some non-deterministic TM}

Time is Tape Dependent Theorem: The time depends on the # of TM tapes. Idea: more tapes can enable higher efficiency. Ex: {0 n 1 n | n>0} is in DTIME(n 2 ) for 1-tape TM’s, and is in DTIME(n) for 2- tape TM’s. Note: For multi- tape TM’s, input tape space does not “count” in the total space s(n). This enables analyzing sub-linear space complexities.

Space is Tape Independent Theorem: The space does not depend on the # tapes. Proof: 1 1 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 Idea : Tapes can be “interlaced” space -efficiently: Note: This does not asymptotically increase the overall space (but can increase the total time). Theorem: A 1-tape TM can simulate a t(n)-time- bounded k-tape TM in time O(k  t 2 (n)).

Space-Time Relations Theorem : If t(n) < t’(n) " n>1 then: DTIME(t(n))  DTIME(t’(n )) NTIME(t(n))  NTIME(t’(n)) Theorem : If s(n) < s’(n) " n>1 then: DSPACE(s(n))  DSPACE(s’(n )) NSPACE(s(n))  NSPACE(s’(n)) Example: NTIME(n)  NTIME(n 2 ) Example : DSPACE(log n)  DSPACE(n)

Examples of Space & Time Usage Let L 1 ={0 n 1 n | n>0}: For 1- tape TM’s: L 1  DTIME(n 2 ) L 1  DSPACE(n) L 1  DTIME(n log n) For 2- tape TM’s: L 1  DTIME(n) L 1  DSPACE(log n)

Examples of Space & Time Usage Let L 2 = S * L 2  DTIME(n) Theorem: every regular language is in DTIME(n) L 2  DSPACE(1) Theorem: every regular language is in DSPACE(1) L 2  DTIME(1) Let L 3 ={w$w | w in S *} L 3  DTIME(n 2 ) L 3  DSPACE(n) L 3  DSPACE(log n)

Special Time Classes Def: P =  DTIME(n k ) " k>1 P  deterministic polynomial time Note: P is robust / model-independent Def: NP =  NTIME(n k ) " k>1 NP  non-deterministic polynomial time Theorem: P  NP Conjecture: P = NP ? Million $ question!

Other Special Space Classes Def: PSPACE =  DSPACE(n k ) " k>1 PSPACE  deterministic polynomial space Def: NPSPACE =  NSPACE(n k ) " k>1 NPSPACE  non-deterministic polynomial space Theorem: PSPACE  NPSPACE (obvious) Theorem: PSPACE = NPSPACE (not obvious)

Other Special Space Classes Def: EXPTIME =  DTIME(2 n k ) " k>1 EXPTIME  exponential time Def: EXPSPACE =  DSPACE(2 n k ) " k>1 EXPSPACE  exponential space Def: L = LOGSPACE = DSPACE(log n) Def: NL = NLOGSPACE = NSPACE(log n)

Space/Time Relationships Theorem: DTIME(f(n))  DSPACE(f(n)) Theorem: DTIME(f(n))  DSPACE(f(n) / log(f(n))) Theorem: NTIME(f(n))  DTIME(c f(n) ) for some c depending on the language. Theorem: DSPACE(f(n))  DTIME(c f(n) ) for some c, depending on the language. Theorem [Savitch]: NSPACE(f(n))  DSPACE(f 2 (n)) Corollary: PSPACE = NPSPACE Theorem: NSPACE(n r )  DSPACE(n r+ e ) " r>0, e >0

The Extended Chomsky Hierarchy 2 S * Decidable Presburger arithmetic EXPSPACE ? H H EXPTIME Not finitely describable Turing PSPACE EXPSPACE-complete =RE degrees Context sensitive LBA PSPACE-complete QBF EXPTIME-complete Go Not Recognizable NP NP-complete SAT P a n b n c n Recognizable Context-free ww R Det. CF a n b n Regular a* Finite {a,b}

Time Complexity Hierarchy Theorem: for any t(n)>0 there exists a decidable language L  DTIME(t(n)). Juris Hartmanis Richard Stearns  No time complexity class contains all the decidable languages, and the time hierarchy is  !  There are decidable languages that take arbitrarily long times to decide! Note: t(n) must be computable & everywhere defined Proof: (by diagonalization) Fix lexicographic orders for TM’s: M 1 , M 2 , M 3 , . . . Interpret TM inputs i  Σ * as encodings of integers: a=1, b=2, aa=3, ab=4, ba=5, bb=6, aaa =7, …

Time Complexity Hierarchy (proof) Define L={i | M i does not accept i within t(i) time} Note: L is decidable (by simulation) Q: is L  DTIME(t(n)) ? Assume (towards contradiction) L  DTIME(t(n)) i.e., $ a fixed K  N such that Turing machine M K decides L within time bound t(n) i If M i accepts i within t(i) time then Reject else Accept M K  decides / accepts L

Time Complexity Hierarchy (proof) K If M K accepts K within t(K) time then Reject else Accept M K  decides / accepts L Consider whether K  L: K  L  M K must accept K within t(K) time  M K must reject K  K  L K  L  M K must reject K within t(K) time  M K must accept K  K  L So (K  L)  (K  L), a contradiction!  Assumption is false  L  DTIME(t(n))

Time Hierarchy (another proof) Consider all t(n)-time- bounded TM’s on all inputs: … i = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 w i  Σ * = a aa ab ba bb aaa aab aba abb baa bab bbabbbaaaa … b M 1 (i)  √  √ √   √ √ √     √  … M 2 (i)    √   √   √  √ √    … M 3 (i)  √ √ √  √    √ √ √ √  √  … M 4 (i)  √  √  √ √  √ √   √    … M 5 (i)   √ √   √ √  √ √ √ √  √  … . . .   M’( i)  √ √ √ . . . is t(n) time-bounded. But M’ computes a different function than any M j  Contradiction!

“Lexicographic order.”

Space Complexity Hierarchy Theorem: for any s(n)>0 there exists a decidable language L  DSPACE(s(n)). Juris Hartmanis Richard Stearns  No space complexity class contains all the decidable languages, and the space hierarchy is  !  There are decidable languages that take arbitrarily much space to decide! Note: s(n) must be computable & everywhere defined Proof: (by diagonalization) Fix lexicographic orders for TM’s: M 1 , M 2 , M 3 , . . . Interpret TM inputs i  Σ * as encodings of integers: a=1, b=2, aa=3, ab=4, ba=5, bb=6, aaa =7, …

Space Complexity Hierarchy (proof) Define L={i | M i does not accept i within t(i) space} Note: L is decidable (by simulation;  -loops?) Q: is L  DSPACE(s(n)) ? Assume (towards contradiction) L  DSPACE(s(n)) i.e., $ a fixed K  N such that Turing machine M K decides L within space bound s(n) i If M i accepts i within t(i) space then Reject else Accept M K  decides / accepts L

Space Complexity Hierarchy (proof) K If M K accepts K within s(K) space then Reject else Accept M K  decides / accepts L Consider whether K  L: K  L  M K must accept K within s(K) space  M K must reject K  K  L K  L  M K must reject K within s(K) space  M K must accept K  K  L So (K  L)  (K  L), a contradiction!  Assumption is false  L  DSPACE(s(n))

Space Hierarchy (another proof) Consider all s(n)-space- bounded TM’s on all inputs: … i = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 w i  Σ * = a aa ab ba bb aaa aab aba abb baa bab bbabbbaaaa … b M 1 (i)  √  √ √   √ √ √     √  … M 2 (i)    √   √   √  √ √    … M 3 (i)  √ √ √  √    √ √ √ √  √  … M 4 (i)  √  √  √ √  √ √   √    … M 5 (i)   √ √   √ √  √ √ √ √  √  … . . .   M’( i)  √ √ √ . . . is s(n) space-bounded. But M’ computes a different function than any M j  Contradiction!

Savitch’s Theorem Theorem: NSPACE(f(n))  DSPACE (f 2 (n)) Walter Savitch Proof: Simulation: idea is to aggressively conserve and reuse space while sacrificing (lots of) time. Consider a sequence of TM states in one branch of an NSPACE(f(n))-bounded computation: Computation time / length is bounded by c f(n) (why?) We need to simulate this branch and all others too! Q: How can we space-efficiently simulate these? A: Use divide-and-conquer with heavy space-reuse!

Savitch’s Theorem Pick a midpoint state along target path: Walter Savitch Verify it is a valid intermediate state by recursively solving both subproblems. Iterate for all possible midpoint states! The recursion stack depth is at most log(c f(n) )=O(f(n)) Each recursion stack frame size is O(f(n)).  total space needed is O(f(n) * f(n))=O(f 2 (n)) Note : total time is exponential (but that’s OK).  non-determinism can be eliminated by squaring the space: NSPACE(f(n))  DSPACE (f 2 (n))

Savitch’s Theorem Corollary: NPSPACE = PSPACE NPSPACE =  NSPACE(n k ) Walter Savitch Proof:   DSPACE(n 2k ) k>1 =  DSPACE(n k ) k>1 k>1 = PSPACE i.e., polynomial space is invariant with respect to non-determinism! Q: What about polynomial time? A: Still open! (P=NP)