CS137: Things weve seen Electronic Design Automation Add two N-bit - PDF document

CS137: Things we’ve seen Electronic Design Automation • Add two N-bit numbers in O(log(N)) time on O(N) processors (gates) • Sort N elements in O(log(N)) time on O(N) processors Day 13: February 8, 2006 • Evaluate an FSM on N inputs in O(log(N)) NC time on O(N) processors • Find the I’th element in a collection of N items in O(log 2 (N)) time on O(N) processors • Compute issuable instructions in O(log(N)) time with O(N) hardware 1 2 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Complexity Class If we use enough space… • Exponential space: P=NP • What are the complexity classes for – NTM runs in time f(N) parallelism? – Use 2 f(N) PEs • Suggested not all tasks have perfect – Each evaluates with a different choice area-time tradeoffs sequence • How well can we parallelize problems? – Prefix on completion – Differentiate things which parallelize well – Solve problem in f(N) time – …things that don’t parallelize so well • Of course, ignores 3-space wire delays 3 4 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon NC • So, we really want to know, how fast • Class of problems that can be: something can be run with a – Computed in polylogarithimic time “reasonable” number of processor • Polynomial in log k (N) (amount of hardware) • E.g. 3log 2 (N)+2log(N)+234 – Using polynomial hardware • NC for Nick’s Class – Named after Nick Pippenger 5 6 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon 1

All in NC Open Question • Can do • NC ?= P – Add two N-bit numbers in O(log(N)) time on O(N) • Are all Polynomial Time algorithms processors (gates) computable in parallel – Sort N elements in O(log(N)) time on O(N) processors – Polylog time – Evaluate an FSM on N inputs in O(log(N)) time on – Polynomial processors O(N) processors – Find the I’th element in a collection of N items in O(log 2 (N)) time on O(N) processors • Suspected they are not – Compute issuable instructions in O(log(N)) time – More at end with O(N) hardware 7 8 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Transitive Closure Basic Sequential Algorithm • N=|V| • Given a Graph: G=(V,E) • Think of M=N×N connectivity matrix for G • Compute G*=(V,E*) • M 2 =G 2 • E* contains an edge e=(V i ,V j ) • M 2 [i,j]=OR (all k) (M[i,k] & M[k,j]) – Iff there is a path from V i to V j in G • M 2n [i,j]=OR (all k) (M n [i,k] & M n [k,j]) • M N represents G N =G* • Transitive Closure ∈ NC • Compute in log steps – O(N 3 log(N)) 9 10 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Parallel Algorithm All Pairs Shortest Paths • Use N 3 processor • Given a Graph: G=(V,E) – Edge weight on each edge e ∈ E • N processors per element M n [i,j] • Compute G’=(V,E’) • N 2 processors to compute all elements of M n • E’ contains an edge e’=(V i ,V j ) • Group of N processors for M n [i,j] perform an – Iff there is a path from V i to V j in G associative reduce � O(log(N)) time – Edge weight is shortest path from V i to V j in G • Still takes log(N) steps to compute M N • O(log 2 (N)) with N 3 processors � in NC • All Pairs Shortest Path ∈ NC – Slight modification on transitive closure – [this construct may be weak?] 11 12 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon 2

Basic Sequential Algorithm (Same) Parallel Algorithm • As before • Use N 3 processor – N=|V| • N processors per element M n [i,j] – Think of M=N×N connectivity matrix for G • N 2 processors to compute all elements of M n – M 2 =G 2 • Change • Group of N processors for M n [i,j] perform an – OR to MIN associative reduce � O(log(N)) time – & to + • Still takes log(N) steps to compute M N • So • O(log 2 (N)) with N 3 processors � in NC – M 2 [i,j]=OR (all k) (M[i,k] & M[k,j]) – Becomes: M 2 [i,j]=MIN (all k) (M[i,k] + M[k,j]) – [this construct may be weak?] • M N represents G N =G’ 13 14 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon NL ⊆ NC NL • Complexity class • Theorem from Borodin: – Computations that can be computed using – If A is accepted by a NDTM using space logarithmic space on a Non-Deterministic S(n) ≥ log 2 (n), Turing Machine – then there is a d>0 such that: • Similarly L DEPTH A (n) ≤ d×S(n) 2 . – logspace on Deterministic TM – [Depth here = circuit depth = time] – Addition ∈ L • For NL • Certainly: L ⊆ NL – S(n)=log 2 (n) � Depth(n) ≤ d×log 2 (n) 15 16 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Borodin Construction (Idea) Borodin States • State is bounded • What states can the NDTM be in? – At most s S(N) values on tape • Can construct the graph of all states • s=size of symbol set – This will only take polynomial hardware – Head of TM at most S(N) positions • Compute transitive closure on graph – q states for FSM – N locations for input tape head – O(log 2 (N)) • Total: states=N×q×S(N)×s S(N) • Use associative reduce to extract – For S(N)=log(N) solution • N×q×log(N) ×s log(N) =qN (log(s)+1) log(N) – O(log(N)) – Number of states polynomial in N 17 18 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon 3

Build Graph Transitive Closure • Construct graph |V|=# states • Transitive Closure with O(|V| 3 ) PEs • M[i,j]=1 iff move from configuration i to j – Still polynomial in N • If V i is a state that corresponds to the input – |V|=N×q×log(N) ×s log(N) =qN (log(s)+1) log(N) head being on square k – O(|V| 3 ) ⊆ O(N 3(log(s)+2) ) – M[i,j] “enabled” iff move from i to j only when kth input is 1 and inputs is 1. • In log 2 (N) time – M[i,j] “enabled” iff move from i to j only when kth – O(log 2 (|V|)) ⊆ O( [log(N (log(s)+2) )] 2 ) input is 0 and input is 0. – Can just be a set of AND’s initially setting up the – O([log(s)+2] 2 ×log 2 (N))=O(log 2 (N)) initial connectivity matrix M 19 20 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Converse Holds Extract Result • Borodin • OR reduce on Reachable states – If A is in DEPTH((S(n)) for S(n) ≥ log(n) – Can reach an accepting state for TM? – Then A is in DSPACE(S(n)) – Recursive evaluation of gate value • w/ compact stack representation • Therefore: NL ⊆ NC • Specialized for S(n)=log(n) – If A is in NC, then A is in L – NC ⊆ L • Know L ⊆ NL … just showed NL ⊆ NC • NL = NC 21 22 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Context Free Languages P-Complete • Can recognize all context free • There are languages that are P-Complete languages in NC – i.e. if could show these were in NC • PDA ⊆ NC – Then would show NC=P • E.g. TM simulation 23 24 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon 4

Complexity Roundup Physical Realism • In NC • Unknown: • All rely on reductions in log(N) time – FA – P=NC (P=NL) • With 3D space, speed of light – PDA – …there are no log(N) time reductions – L • Maybe notion of 3-space parallelizable? – NL – Run in O(N 1/3 ) time – O(N) processors • Cannot talk to more than O(N) in O(N 1/3 ) time 25 26 CALTECH CS137 Winter2006 -- DeHon CALTECH CS137 Winter2006 -- DeHon Admin • Friday/Monday:?? • Q: requests – what’s missing? • Project: two things due end of next week – Sequential implementation – Proposed plan of attack 27 CALTECH CS137 Winter2006 -- DeHon 5