complete monotonicity conjecture of heat equation
play

Complete Monotonicity Conjecture of Heat Equation MDDS, SJTU, 2019 - PowerPoint PPT Presentation

How to Solve Gaussian Interference Channel Complete Monotonicity Conjecture of Heat Equation MDDS, SJTU, 2019 Fan Cheng Shanghai Jiao Tong University chengfan@sjtu.edu.cn From 2008 to 2019 2 2 2 , =


  1. How to Solve Gaussian Interference Channel Complete Monotonicity Conjecture of Heat Equation MDDS, SJTU, 2019 Fan Cheng Shanghai Jiao Tong University chengfan@sjtu.edu.cn

  2. From 2008 to 2019

  3. πœ– 2 2πœ–π‘¦ 2 𝑔 𝑦, 𝑒 = πœ– πœ–π‘’ 𝑔(𝑦, 𝑒) β„Ž π‘Œ + π‘’π‘Ž 𝑍 = π‘Œ + π‘’π‘Ž β„Ž π‘Œ = βˆ’βˆ« 𝑦log𝑦 d𝑦 π‘Ž ∼ π’ͺ(0,1) β–‘ A new mathematical theory on Gaussian distribution β–‘ Its application on Gaussian interference channel β–‘ History, progress, and future

  4. Outline β–‘ History of β€œSuper -H ” Theorem β–‘ Boltzmann equation, heat equation β–‘ Shannon Entropy Power Inequality β–‘ Complete Monotonicity Conjecture β–‘ How to Solve Gaussian Interference Channel

  5. Study of Heat πœ– 2 πœ–π‘’ 𝑔 𝑦, 𝑒 = 1 πœ– πœ–π‘¦ 2 𝑔(𝑦, 𝑒) 2 β–‘ The history begins with the work of Joseph Heat transfer Fourier around 1807 β–‘ In a remarkable memoir, Fourier invented both Heat equation and the method of Fourier analysis for its solution

  6. Information Age Gaussian Channel: π‘Ž 𝑒 ∼ π’ͺ(0, 𝑒) X and Z are mutually independent. The p.d.f of X is g(x) 𝑒 is the convolution of X and π‘Ž 𝑒 . 𝑍 𝑍 𝑒 ≔ π‘Œ + π‘Ž 𝑒 The probability density function (p.d.f.) of 𝑍 𝑒 (π‘§βˆ’π‘¦) 2 1 𝑔(𝑧; 𝑒) = ∫ 𝑕(𝑦) 𝑓 2𝑒 2πœŒπ‘’ πœ– 2 πœ–π‘’ 𝑔(𝑧; 𝑒) = 1 πœ– πœ–π‘§ 2 𝑔(𝑧; 𝑒) A mathematical theory of communication, 2 Bell System Technical Journal. The p.d.f. of Y is the solution to the heat equation, and vice versa. Gaussian channel and heat equation are identical in mathematics.

  7. Ludwig Boltzmann Boltzmann formula: 𝑇 = βˆ’π‘™ 𝐢 ln𝑋 Gibbs formula: 𝑇 = βˆ’π‘™ 𝑐 βˆ‘ π‘ž 𝑗 lnπ‘ž 𝑗 𝑗 Boltzmann equation: 𝑒𝑔 𝑒𝑒 = (πœ–π‘” πœ–π‘’) force + (πœ–π‘” πœ–π‘’) diff + (πœ–π‘” Ludwig Eduard Boltzmann πœ–π‘’) coll 1844-1906 Vienna, Austrian Empire H-theorem: 𝐼(𝑔(𝑒)) is nonβˆ’decreasing

  8. β€œSuper H - theorem” for Boltzmann Equation A function is completely monotone (CM) iff all the signs of its derivatives are alternating in +/-: +, -, +, - ,…… (e.g., 1/𝑒, 𝑓 βˆ’π‘’ ) β–‘ McKean’s Problem on Boltzmann equation (1966): β–‘ 𝐼(𝑔(𝑒)) is CM in 𝑒, when 𝑔 𝑒 satisfies Boltzmann equation β–‘ False, disproved by E. Lieb in 1970s β–‘ the particular Bobylev-Krook-Wu explicit solutions, this β€œtheorem” holds true for π‘œ ≀ 101 and H. P. McKean, NYU. breaks downs afterwards National Academy of Sciences

  9. β€œSuper H - theorem” for Heat Equation β–‘ Heat equation: Is 𝐼(𝑔(𝑒)) CM in 𝑒 , if 𝑔(𝑒) satisfies heat equation β–‘ Equivalently, is 𝐼(π‘Œ + π‘’π‘Ž) CM in t? β–‘ The signs of the first two order derivatives were obtained Failed to obtain the 3 rd and 4 th . (It is easy to compute the β–‘ derivatives, it is hard to obtain their signs) β€œThis suggests that……, etc., but I could not prove it” -- H. P. McKean C. Villani, 2010 Fields Medalist

  10. Claude E. Shannon and EPI Central limit theorem Capacity region of Gaussian broadcast channel Capacity region of Gaussian Multiple-Input Multiple-Output broadcast channel Uncertainty principle All of them can be proved by Entropy Power Inequality (EPI) β–‘ Entropy power inequality (Shannon 1948): For any two independent continuous random variables X and Y, 𝑓 2β„Ž(𝒀+𝒁) β‰₯ 𝑓 2β„Ž(𝒀) + 𝑓 2β„Ž(𝒁) Equality holds iff X and Y are Gaussian β–‘ Motivation: Gaussian noise is the worst noise β–‘ Impact: A new characterization of Gaussian distribution in information theory β–‘ Comments: most profound! (Kolmogorov)

  11. Entropy Power Inequality β–‘ Shannon himself didn’t give a proof but an explanation, which turned out to be wrong β–‘ The first proof is given by A. J. Stam (1959), N. M. Blachman (1966) β–‘ Research on EPI Generalization, new proof, new connection. E.g., Gaussian interference channel is open, some stronger β€œEPI’’ should exist. β–‘ Stanford Information Theory School: Thomas Cover and his students: A. El Gamel, M. H. Costa, A. Dembo, A. Barron (1980- 1990) β–‘ After 2000, Princeton && UC Berkeley Heart of Shannon theory

  12. Ramification of EPI Gaussian perturbation: β„Ž(π‘Œ + π‘’π‘Ž) Shannon EPI πœ– Fisher Information: 𝐽 π‘Œ + π‘’π‘Ž = πœ–π‘’ β„Ž(π‘Œ + π‘’π‘Ž)/2 Fisher Information is decreasing in 𝑒 Fisher information inequality (FII): 𝑓 2β„Ž(π‘Œ+ π‘’π‘Ž) is concave in 𝑒 1 1 1 𝐽(π‘Œ+𝑍) β‰₯ 𝐽(π‘Œ) + 𝐽(𝑍) Status Quo: FII can imply EPI and all its generalizations. Tight Young’s inequality Many network information problems remain open even π‘Œ + 𝑍 𝑠 β‰₯ 𝑑 π‘Œ π‘ž 𝑍 π‘Ÿ the noise is Gaussian. --Only EPI is not sufficient

  13. Remarks 𝑒 ) is concave in 𝑒 ο‚‘ Costa’s EPI: 𝑓 2β„Ž(𝑍 ο‚‘ Derived the first two derivatives by very involved calculus (1986) ο‚‘ IT society did not know McKean’s paper until 2014 ο‚‘ Log-Sobolev inequality ο‚‘ A. Dembo gave a very simple proof via FII (1987) ο‚‘ C. Villani simplified Costa’s calculus (2006) ο‚‘ The first two derivatives are not commonly used in network information theory ο‚‘ In geometry, mathematician need the first derivative to estimate the speed of convergence. However, information theorists are not interested ο‚‘ Relation with CLT

  14. Where our journey begins  Shannon Entropy power inequality Information theorists get lost in the past 70 years  Fisher information inequality  β„Ž(π‘Œ + π‘’π‘Ž)  is CM β„Ž 𝑔 𝑒  When 𝑔(𝑒) satisfied Boltzmann equation, disproved Mathematician ignored it  When 𝑔(𝑒) satisfied heat equation, unknown  We even don’t know what CM is!  Raymond introduced this paper to me in 2008  I made some progress with Chandra Nair in 2011 (MGL)  Complete monotonicity (CM) was discovered in 2012  The third derivative in 2013 (Key breakthrough)  The fourth order in 2014  Recently, CM οƒ  GIC

  15. Motivation Motivation: to find some inequalities to obtain a better rate region; e.g., the 𝑱 𝒀+ 𝒖𝒂 𝒇 βˆ’π’– 𝒂) , the concavity of convexity of π’Š(𝒀 + , etc. 𝒖 β€œAny progress?” It is widely believed that there should be no β€œ Nope …” new EPI except Shannon EPI and FII. Observation: 𝑱(𝒀 + 𝒖𝒂) is convex in 𝒖 πœ– π‘’π‘Ž β‰₯ 0 ( de Bruijn , 1958) 𝐽 π‘Œ + π‘’π‘Ž = 2πœ–π‘’ β„Ž π‘Œ + 𝐽 (1) = πœ– π‘’π‘Ž ≀ 0 (McKean1966, Costa 1985) πœ–π‘’ 𝐽 π‘Œ + Could the third one be determined?

  16. Discovery Observation: 𝑱(𝒀 + 𝒖𝒂) is convex in 𝒖 1 1  β„Ž π‘Œ + 𝑒 . 𝐽 is CM: +, -, +, - … π‘’π‘Ž = 2 ln 2πœŒπ‘“π‘’, 𝐽 π‘Œ + π‘’π‘Ž =  If the observation is true, the first three derivatives are: +, -, +  Q: Is the 4 th order derivative -? Because π‘Ž is Gaussian ! If so, then…  The signs of derivatives of β„Ž(π‘Œ + π‘’π‘Ž) are independent of π‘Œ . Invariant!  Exactly the same problem in McKean’s 1966 paper My own opinion: β€’ A new fundamental result on Gaussian distribution β€’ Invariant is very important in mathematics β€’ In mathematics, the more beautiful, the more powerful β€’ Very hard to make any progress To convince people, must prove its convexity

  17. Challenge Let π‘Œ ∼ 𝑕(𝑦) οƒ˜ β„Ž 𝑍 𝑒 = βˆ’βˆ« 𝑔(𝑧, 𝑒) ln 𝑔(𝑧, 𝑒) 𝑒𝑧 : no closed-form expression except for some special 𝑕 𝑦 . οƒ˜ 𝑔(𝑧, 𝑒) satisfies heat equation. 2 𝑔 οƒ˜ 𝐽 𝑍 1 𝑒 = ∫ 𝑔 𝑒𝑧 2 2 οƒ˜ 𝐽 1 𝑍 𝑔 𝑔 2 1 𝑒 = βˆ’βˆ« 𝑔 βˆ’ 𝑒𝑧 𝑔 2 οƒ˜ So what is 𝐽 (2) ? (Heat equation, integration by parts)

  18. Challenge (cont’d) 𝑱 It is trivial to calculate derivatives. It is not generally obvious to prove their signs.

  19. Breakthrough Integration by parts: ∫ 𝑣𝑒𝑀 = 𝑣𝑀 βˆ’ ∫ 𝑀𝑒𝑣 First breakthrough since McKean 1966

  20. GCMC Gaussian complete monotonicity conjecture (GCMC): 𝒖𝒂) is CM in 𝒖 𝑱(𝒀 + Conjecture 2: 𝐦𝐩𝐑𝑱(𝒀 + 𝒖𝒂) is convex in 𝒖 A general form: number partition. Hard to determine the coefficients. Hard to find 𝛾 𝑙,π‘˜ !

  21. Remarks: C. Villani showed the work of H. P. McKean to us. G. Toscani cited our work within two weeks:  the consequences of the evolution of the entropy and of its subsequent derivatives along the solution to the heat equation have important consequences.  Indeed the argument of McKean about the signs of the first two derivatives are equivalent to the proof of the logarithmic Sobolev inequality. Gaussian optimality for derivatives of differential entropy using linear matrix inequalities X. Zhang, V. Anantharam, Y. Geng - Entropy, 2018 - mdpi.com β€’ A new method to prove signs by LMI β€’ Verified the first four derivatives β€’ For the fifth order derivative, current methods cannot find a solution

  22. Complete monotone function 𝜈 ∞ 𝑓 βˆ’π‘’π‘¦ π‘’πœˆ(𝑦) 𝑔 𝑒 = ΰΆ± 0 How to construct 𝜈(𝑦) ? A new expression for entropy involved special functions in mathematical physics Herbert R. Stahl, 2013

Recommend


More recommend