Towards a Theory of Information Flow -Transducers in the Finitary - PowerPoint PPT Presentation

Information Flow Spencer Mathews Towards a Theory of Information Flow ǫ -Transducers in the Finitary Process Soup Dynamics Single State Soup Evolution Channel Capacity Spencer Mathews Partitioning Metamachine Department of Computer Science Conclusion and Complexity Sciences Center University of California at Davis June 1, 2010

Goals Information Flow Spencer Mathews ǫ -Transducers Dynamics Analyze model of evolutionary self-organization in terms of Single State Soup information flow. Evolution Channel Measure channel capacity of elementary ǫ -Transducers. Capacity Develop functional partitioning based on this and language Partitioning properties. Metamachine Conclusion

Outline ǫ -Transducers 1 Information Flow Dynamics 2 Spencer Single State Soup Mathews Evolution ǫ -Transducers Channel Capacity 3 Dynamics Partitioning 4 Single State Soup Metamachine Evolution 5 Channel Conclusion 6 Capacity Partitioning Metamachine Conclusion

ǫ -machines Information Flow Spencer ǫ -machines Defined Mathews T = { S , T } ǫ -Transducers S is a set of causal states Dynamics T is the set of transitions between them: T ( s ) ij , s ∈ A Single State Soup Evolution Channel Capacity ǫ -machine Properties Partitioning All of their recurrent states form a single strongly Metamachine connected component. Conclusion Transitions are deterministic . S is minimal : an ǫ -machine is the smallest causal representation of the transformation it implements.

Transducers Information Flow Spencer Mathews We interpret the symbols labeling the transitions in the ǫ -Transducers alphabet A as consisting of two parts: an input symbol Dynamics Single State that determines which transition to take from a state and Soup Evolution an output symbol which is emitted on taking that Channel transition. Capacity Partitioning Transducers implement functions: Metamachine Character to character Conclusion Input string to output string Map sets to sets (languages)

Set of Single State Machines Information 0|1 1|0 Flow 0|0 0|1 0|0 1|0 0|0 Spencer Mathews 1 2 3 4 5 ǫ -Transducers Dynamics 1|0 Single State Soup 1|0 0|1 1|1 1|1 Evolution 0|1 0|0 1|1 0|0 0|1 Channel Capacity Partitioning 6 7 8 9 10 Metamachine Conclusion 1|1 1|1 1|1 1|1 1|0 0|1 1|1 1|0 1|0 0|1 0|0 1|0 0|0 0|1 0|0 11 12 13 14 15

Transducer Composition Information Flow Spencer Mathews ǫ -Transducers Dynamics Create new mapping, input language of one machine Single State Soup Evolution becomes input language of another Channel Not commutative Capacity Partitioning Possible exponential growth in number of states. Metamachine Conclusion

Interaction Network Information Flow Spencer Mathews ǫ -Transducers Dynamics Single State Interaction Matrix G ( k ) Soup Evolution � 1 Channel if T k = T j ◦ T i G ( k ) = Capacity ij 0 otherwise Partitioning Metamachine Conclusion

Single State Transition Matrix Information Flow Spencer   T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 Mathews T 0 T 0 T 0 T 1 T 1 T 1 T 1 T 2 T 2 T 2 T 2 T 3 T 3 T 3 T 3     ǫ -Transducers T 1 T 2 T 3 T 1 T 1 T 3 T 3 T 2 T 3 T 2 T 3 T 3 T 3 T 3 T 3     T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 Dynamics     Single State T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15   Soup   Evolution T 4 T 8 T 12 T 1 T 5 T 9 T 13 T 2 T 6 T 10 T 14 T 3 T 7 T 11 T 15     T 5 T 10 T 15 T 1 T 5 T 11 T 15 T 2 T 7 T 10 T 15 T 3 T 7 T 11 T 15 Channel     Capacity T 0 T 0 T 0 T 4 T 4 T 4 T 4 T 8 T 8 T 8 T 8 T 12 T 12 T 12 T 12     Partitioning  T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 T 11 T 12 T 13 T 14 T 15     T 0 T 0 T 0 T 5 T 5 T 5 T 5 T 10 T 10 T 10 T 10 T 15 T 15 T 15 T 15  Metamachine    T 1 T 2 T 3 T 5 T 5 T 7 T 7 T 10 T 11 T 10 T 11 T 15 T 15 T 15 T 15  Conclusion    T 4 T 8 T 12 T 4 T 4 T 12 T 12 T 8 T 12 T 8 T 12 T 12 T 12 T 12 T 12      T 5 T 10 T 15 T 4 T 5 T 14 T 15 T 8 T 13 T 10 T 15 T 12 T 13 T 14 T 15     T 4 T 8 T 12 T 5 T 5 T 13 T 13 T 10 T 14 T 10 T 14 T 15 T 15 T 15 T 15   T 5 T 10 T 15 T 5 T 5 T 15 T 15 T 10 T 15 T 10 T 15 T 15 T 15 T 15 T 15

Interaction Network Information Flow Spencer Mathews ǫ -Transducers Dynamics Interaction Graph G Single State Soup Evolution Nodes correspond to ǫ -transducers Channel Capacity If T C = T B ◦ T A , place directed edge connecting node T A , Partitioning to T C , labeled with the transforming machine T B . Metamachine Conclusion

Single State Interaction Network G Information Flow T_6 T_9 Spencer T_6 T_6 Mathews T_13 T_9 T_9 T_7 ǫ -Transducers T_14 T_7 T_13 T_11 Dynamics T_4 T_7 T_9T_13 T_11 T_14 T_13 T_9T_13 T_10 T_3 Single State Soup Evolution T_8 T_1 T_6 T_14 T_6 T_7 T_10 T_15 T_6 T_14 T_6 T_7 T_12 T_1 Channel T_12 T_2 T_4 T_11 T_9T_11 T_15 T_2 T_10 T_10 T_2 T_14 T_9T_11 T_4 T_12 T_5 T_2 Capacity Partitioning T_3 T_8 T_5 T_3 T_1 T_7 T_11 T_15 T_8 T_10 T_15 T_3 T_7 T_11 T_8 T_10 T_4 T_3 T_8 Metamachine T_1 T_1T_5T_9T_13 T_12 T_5 T_2 T_1 T_15 T_12 T_13 T_14 T_13 T_14 T_10 T_8T_9T_10T_11 T_15 T_12 T_1 T_1 T_5 T_12 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_8 Conclusion T_14 T_6 T_10 T_4 T_5 T_6 T_7 T_2 T_3 T_5 T_4 T_12 T_2 T_8 T_10 T_13 T_14 T_15 T_4 T_5 T_2 T_3 T_7 T_11 T_15 T_4 T_5 T_1 T_15 T_3 T_7 T_11 T_1 T_4 T_5 T_2 T_8T_9T_10T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_14 T_5 T_6 T_7 T_4 T_2 T_6 T_10 T_4 T_1T_5T_9T_13 T_2 T_12 T_13 T_14 T_15 T_8 T_10 T_2 T_8 T_10 T_12 T_13 T_14 T_15 T_1 T_4 T_5 T_3 T_7 T_11 T_15 T_6 T_10 T_14 T_4 T_5 T_6 T_7 T_2 T_3 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_5 T_1T_5T_9T_13 T_8 T_8T_9T_10 T_11

Population Dynamics Information Flow Spencer Mathews ǫ -Transducers Dynamics Single State Soup Evolution Population P Channel N individuals Capacity Partitioning Metamachine Conclusion

Population Dynamics Information Flow Spencer Mathews A single replication is determined through compositions and ǫ -Transducers replacements in a two-step sequence: Dynamics Single State Soup Construct ǫ -machine T C by forming the composition 1 Evolution T C = T B ◦ T A from T A and T B randomly selected from Channel Capacity the population and minimizing. Partitioning Replace a randomly selected ǫ -machine, T D , with T C . 2 Metamachine Note that there is no imposed notion of fitness Conclusion nor spatial component.

Relaxation to Steady State (Single State Soup of Size 100,000) Information Flow 0.25 T 15 Spencer Mathews 0.2 ǫ -Transducers Dynamics Single State Soup 0.15 Evolution T 3 , T 5 , T 10 , T 12 Channel p Capacity 0.1 Partitioning T 1 , T 2 , T 4 , T 8 Metamachine Conclusion 0.05 T 7 , T 11 , T 13 , T 14 T 6 , T 9 0 0 2 4 6 8 10 t/N

Communication Channel Information Flow Spencer Mathews ǫ -Transducers Dynamics Discrete Channel [Cover and Thomas, 2006] Single State Soup Evolution ( X , p ( y | x ) , Y ): Channel Capacity ( X and Y ) are finite sets Partitioning p ( y | x ) are probability mass functions Metamachine Conclusion

Channel Capacity Information Flow Spencer Mathews Cover and Thomas [Cover and Thomas, 2006] define the ǫ -Transducers channel capacity of a discrete memoryless channel as: Dynamics Single State Soup C = max p ( x ) I ( X ; Y ) (1) Evolution Channel Capacity This capacity specifies the highest rate, in bits, at which Partitioning Metamachine information may be reliably transmitted through the channel, Conclusion and Shannon’s second theorem states that his rate is achievable in practice.

Mutual Information Information Flow Spencer Mathews ǫ -Transducers Dynamics Remember that mutual information is the reduction in Single State Soup uncertainty in one random variable due to knowledge of Evolution Channel another. Capacity I ( X ; Y ) = H ( Y ) − H ( Y | X ) (2) Partitioning Metamachine Conclusion

Discrete Noiseless Channel Information Flow Spencer Mathews ǫ -Transducers Dynamics Single State Soup Discrete Noiseless Channel Evolution One to one correspondence between input and output Channel Capacity symbols Partitioning Metamachine Conclusion

Languages Information Flow Spencer Mathews ǫ -Transducers Dynamics Partitioning based on language L Single State Soup Evolution L in = Σ ∗ Channel L out = Σ ∗ Capacity Partitioning union of these – possibility for positive channel capacity Metamachine Conclusion

Single State Channel Capacities Information Flow 1 Spencer Mathews ǫ -Transducers 0.5 Dynamics Single State Soup 0 Evolution 1 Channel Capacity bits 0.5 Partitioning Metamachine 0 Conclusion 1 0.5 0 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 P(X)

Machines of Maximal Channel Capacity Information Flow 1 Spencer Mathews ǫ -Transducers Dynamics Single State Soup Evolution Channel Capacity bits 0.5 Partitioning Metamachine Conclusion 0 0 0.5 1 P(X)

Partially Noisy Channels Information Flow 1 Spencer Mathews ǫ -Transducers Dynamics Single State Soup Evolution Channel Capacity bits 0.5 Partitioning Metamachine Conclusion 0 0 0.5 1 P(X)