Computability of the Zero-Error capacity with Kolmogorov Oracle - PowerPoint PPT Presentation

Computability of the Zero-Error capacity with Kolmogorov Oracle Holger Boche 1 and Christian Deppe 2 Technical University of Munich 1 Chair of Theoretical Information Technology 2 Institute for Communications Engineering ISIT 21-26 June 2020 Work was supported by Germany’s Excellence Strategy - EXC 2092 CASA - 390781972 (Boche) and BMBF through grant 16KIS1005 (Deppe)

Content • Motivation - Status Quo • Zero-Error Capacity • Shannon Capacity • Computability • Kolmogorov Oracle • Computability of C 0 with Kolmogorov Oracle • Zuiddam’s Characterization of C 0 and Algorithmic Computability Holger Boche (TUM) 2

Motivation - Status Zero-Error Capacity C 0 • Introduced by Shannon in 1956, no formula for "computing" C 0 is known today. • "Folklore theorem": Computation of C 0 is a "hard problem" (To compute the independence number of the strong product of the confusability graph is NP hard, but this result implies nothing about Turing computability of C 0 or Θ ). BUT: • It is unknown if C 0 or the Shannon capacity for graphs Θ is Turing computable. • It is even unknown if Θ( G ) is a computable number for all graphs G . Plan: Investigate Turing computability in the framework of Turing with Oracle methods and analyze: • Turing computability of Θ . • Algorithmic behavior of Zuiddams recent characterization of Θ . Holger Boche (TUM) 3

Zero-Error Capacity Definition A discrete memoryless channel (DMC) is a triple ( X , Y , W ) , where X is the finite input alphabet, Y is the finite output alphabet, and W ( y | x ) with x ∈ X , y ∈ Y is a stochastic matrix. The probability for a sequence y n ∈ Y n to be received if x n ∈ X n was sent is defined by n � W n ( y n | x n ) = W ( y j | x j ) . j = 1 Holger Boche (TUM) 4

Zero-Error Capacity • Two sequences x n and x ′ n of size n of input variables are distinguishable by a receiver if the vectors W n ( ·| x n ) and W n ( ·| x ′ n ) are orthogonal. • That means if W n ( y n | x n ) > 0 then W n ( y n | x ′ n ) = 0 and if W n ( y n | x ′ n ) > 0 then W n ( y n | x n ) = 0. • We denote by M ( W , n ) the maximum cardinality of a set of mutually orthogonal vectors among the W n ( ·| x n ) with x n ∈ X n . The zero-error capacity of W is: log 2 M ( W , n ) C 0 ( W ) = lim inf n n →∞ Holger Boche (TUM) 5

Shannon Capacity of a Graph • Shannon introduced a simple graph G W . • In this graph two letters/vertices x and x ′ are connected, if one could be confused with the other. • Therefore, the size of the maximum independent set α ( G W ) is the maximum number of 1-letter messages which can be sent without danger of confusion. • The definition is extended to words of length n by α ( G ⊠ n W ) . Theorem (Shannon 1956) 2 C 0 ( W ) = Θ( G W ) = lim 1 n →∞ α ( G ⊠ n W ) n . No "computable" formula for Θ( G W ) is known. Question: Is there an algorithm, that takes G and error µ as inputs and computes the number Θ( G ) with precision µ ? Holger Boche (TUM) 6

Ahlswede’s Question In his paper 1970 Rudolf Ahlswede wrote “One would like to have a “reasonable” formula for C 0 , which does not “depend on an infinite product space.” Such a formula is unknown. An answer as: for given d there exists a k = k ( d ) such that N ( nk , 0 ) = ( N ( k , 0 )) n ” could be considered “reasonable”.” N ( n , 0 ) denotes the maximal N for for which a zero-error code for n exists. Thus, the definition of N ( n , 0 ) matches our definition of M ( W , n ) for a fixed channel W . If Ahlswede’s “resonable formula” were correct, we would of course have achieved Turing computability immediately. Ahlswede’s question about a reasonable formula can be interpreted in the weakest form as a question about Turing computablility. Holger Boche (TUM) 7

Turing Computability • The concept of a Turing machine is a mathematical model of an abstract machine that manipulates symbols on a strip of tape according to certain given rules. • It can simulate any given algorithm and therewith provides a simple but very powerful model of computation. • Turing machines have no limitations on computational complexity,unlimited computing capacity and storage, and execute programs completely error-free. • They provide fundamental performance limits for digital computers and they are the ideal concept to decide whether or not a function (here the zero-error capacity) is effectively computable. Holger Boche (TUM) 8

Computability We would like to make statements about the computability of the zero-error capacity. This capacity is generally a real number. Therefore, we first define when a real number is computable. Definition A sequence of rational numbers { r n } n ∈ N 0 is called a computable sequence if there exist recursive functions a , b , s : N 0 → N 0 with b ( n ) � = 0 for all n ∈ N 0 and r n = ( − 1 ) s ( n ) a ( n ) b ( n ) , n ∈ N 0 . A real number x is said to be computable if there exists a computable sequence of rational numbers { r n } n ∈ N 0 such that | x − r n | < 2 − n for all n ∈ N 0 . We denote the set of computable real numbers by R c . Holger Boche (TUM) 9

Algorithmic Computability • We have a representation for C 0 ( W ) and Θ( G ) as the limit of a monotonic convergent sequence 1 . • We do not have an effective estimate of the rate of convergence. • If we want to calculate the first bits of the binary representation of the number Θ( G ) even for a fixed graph G, this is not possible with only the result of Shannon. • An approach would now be, e.g. for the decimal representation, to derive the best possible lower bound for Θ( G ) from achievability part and derive a good upper bound for Θ( G ) from an approach for the inverse part. • If the two bounds match for the first L decimal places, then we have determined Θ( G ) for the first L decimal places. 1 Note: There are well known and simple examples of computable monotonic increasing and boundes sequences of rational numbers such that the limit points are not computable numbers. Holger Boche (TUM) 10

Semi-Decidable Definition A subset G 1 ⊂ G is called semi-decidable, if there is a Turing machine TM 1 with the state “stop”, such that TM 1 ( G ) stops if and only if G ∈ G 1 . Therefore TM 1 has only one stop state. If TM 1 does not stop, it computes forever. A subset G 1 ⊂ G is called decidable, if G 1 and G c 1 are semi decidable. Lemma � � Let λ ∈ R c , λ ≥ 0 . Then the set G ( λ ) := G ∈ G : Θ( G ) > λ is semi decidable. Holger Boche (TUM) 11

Kolmogorov Oracle Definition Let i be an enumeration of simple graphs and u N 0 an optimal recursive listing of the set of natural numbers (Schorr). The Kolmogorov oracle O K , G ( · ) is a function from N 0 to the power set of the set of graphs that produces a list � � O K , G ( n ) := G : C u G ( G ) ≤ n for each n ∈ N 0 , where C u G ( G ) := min { k : i ( u N 0 ( k )) = G } . According to our definition of graphs and the set G with the listing i , this is the same as the listing O K , N 0 of the natural numbers k with C u N 0 ( k ) ≤ n . Holger Boche (TUM) 12

Computability with Kolmogorov Oracle • We say that TM can use the oracle O K , G if for every n ∈ N 0 , on input n the Turing machine gets the list O K , G ( n ) . • With TM ( O K , G ) we denote a Turing Machine that has access to the Oracle O K , G . Theorem Let λ ∈ R c , λ > 0 , then the set G ( λ ) is decidable with a Turing machine TM ∗ ( O K , G ) . This means there exists a Turing machine TM ∗ ( O K , G ) , such that the set G ( λ ) is computable with this Turing machine with oracle. Holger Boche (TUM) 13

Alon’s Question Corollary Let λ ∈ R c , λ ≥ 0 . Then, the set { G : Θ( G ) ≤ λ } is semi-decidable for Turning machines with oracle O K , N 0 , oracle O K , G , respectively. 1. Noga Alon has asked 2006 if the set { G : Θ( G ) ≤ λ } is semi-decidable. We gave a positive answer to this question if we can include the oracle. 2. We do not know if C 0 is computable concerning TM ( O K , G ) . Holger Boche (TUM) 14

Effective Converse Theorem The Shannon capacity and thus the function Θ is Turing computable if and only if there is a computable sequence { F N } N ∈ N 0 of computable functions F N : G → R c , so that the following conditions apply: 1. For all N ∈ N 0 holds F N ( G ) ≥ Θ( G ) for all G ∈ G . 2. lim N →∞ F N ( G ) = Θ( G ) for all G ∈ G . Holger Boche (TUM) 15

Zuiddam’s Characterization • In order to prove the computability of C 0 or Θ , we need computable converses in the sense of previous Theorem. • Therefore, the characterization of Zuiddam use the functions from the asymptotic spectrum of graphs is interesting. • We examine this approach with regard to computability. Holger Boche (TUM) 16