Convex Functions (II) Lijun Zhang zlj@nju.edu.cn - PowerPoint PPT Presentation

Convex Functions (II) Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj

Outline  The Conjugate Function  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities  Summary

Outline  The Conjugate Function  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities  Summary

Conjugate Function � Its conjugate function is  ∗ � �∈��� � ∗ ∗  ∗ is always convex 

Conjugate Function � Its conjugate function is  ∗ � �∈��� �

Conjugate examples  Affine function  ∗  �∈𝐒 ∗ ∗   Negative logarithm  ∗  �∈𝐒 �� ∗ ∗  ��

Conjugate examples  Exponential �  ∗ �  �∈𝐒 ∗ ∗  �  Negative entropy  ∗  �∈𝐒 � ∗ ∗ ��� 

Conjugate examples  Inverse  ∗  �∈𝐒 �� ∗ ∗ �/�  �  Strictly convex quadratic function � � �  �� � � ∗ � �  � �∈𝐒 � � ∗ � ∗ � ��  �

Conjugate examples  Log-determinant � ��  �� ∗  � �∈𝐓 �� � ∗ ∗ ��  ��  Indicator function � is not  � � necessarily convex ∗ �  � �∈� ∗ is the support function of the set  �

Conjugate examples  Norm � with dual norm  ∗ ∗ �  �∈𝐒 � ∗ ∗  ∗  Norm squared � with dual norm � �  ∗ � � ∗ � �  � �∈𝐒 � � ∗ � ∗ �  ∗ �

Basic properties  Fenchel’s inequality ∗ ∗ �  ∗ �  �∈𝐒 � � � �  �� � � � � � � �� � �  Conjugate of the conjugate ∗∗ is convex and closed 

Basic properties  Differentiable functions  𝑔 is convex and differentiable, dom 𝑔 � 𝐒 � ∗ �  �∈𝐒 � ∗ � ∗  ∗� ∗� ∗ ∗ ∗ ∗   𝑦 ∗ � 𝛼 �� 𝑔 𝑧

Basic properties  Scaling with affine transformation  � ∗ ∗ � ��� is nonsingular �  ∗ ∗ �� � �� ∗ � ∗  Sums of independent functions � are convex  � � � ∗ ∗ ∗ � �

Outline  The Conjugate Function  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities  Summary

Quasiconvex functions  Quasiconvex  𝑔: 𝐒 � → 𝐒  𝑇 � � 𝑦 ∈ dom 𝑔 𝑔 𝑦 � 𝛽�, ∀𝛽 ∈ 𝐒 is convex

Quasiconvex functions  Quasiconvex  𝑔: 𝐒 � → 𝐒  𝑇 � � 𝑦 ∈ dom 𝑔 𝑔 𝑦 � 𝛽�, ∀𝛽 ∈ 𝐒 is convex  Quasiconcave  �𝑔 is quasiconvex ⇒ 𝑔 is quasiconcave  Quasilinear  𝑔 is quasiconvex and quasiconcave ⇒ 𝑔 is quasilinear

Examples  Some example on  Logarithm: on ��  Ceiling function ：  Linear-fractional function � � ��� �  � � ��� � � ��� �  � � ��� � � ��� � is convex � � ��� is Quasilinear

Basic properties  Jensen’s inequality for quasiconvex functions is quasiconvex is convex and  𝑔 𝜄𝑦 � 1 � 𝜄 𝑧 � max 𝑔 𝑦 , 𝑔 𝑧

Basic properties  Condition 𝑔 is quasiconvex ⇔ its restriction to any line  intersecting its domain is quasiconvex  Quasiconvex functions on A continuous function 𝑔: 𝐒 → 𝐒 is quasiconvex ⇔  one of the following conditions holds 𝑔 is nondecreasing • 𝑔 is nonincreasing • ∃𝑑 ∈ dom 𝑔, ∀𝑢 ∈ dom 𝑔, 𝑢 � 𝑑, 𝑔 is • nonincreasing, and 𝑢 � 𝑑, 𝑔 is nondecreasing

Differentiable quasiconvex functions  First-order conditions 𝑔 is differentiable  𝑔 is quasiconvex ⇔ dom 𝑔 is convex, ∀𝑦, 𝑧 ∈  dom 𝑔, 𝑔 𝑧 � 𝑔 𝑦 ⇒ 𝛼𝑔 𝑦 � 𝑧 � 𝑦 � 0 It is possible that 𝛼𝑔 𝑦 � 0 , but 𝑦 is not a  global minimizer of 𝑔 .  Second-order conditions 𝑔 is twice differentiable  ∀𝑦 ∈ dom 𝑔, ∀𝑧 ∈ 𝐒 � , 𝑧 � 𝛼𝑔 𝑦 � 0 ⇒ 𝑧 � 𝛼 � 𝑔 𝑦 𝑧 �  0 ⇒ 𝑔 is quasiconvex

Operations that preserve quasiconvexity  Nonnegative weighted maximum � is quasicovex, 𝑥 � � 0 ⇒ 𝑔 �  𝑔 � � is quasiconvex max�𝑥 � 𝑔 � , … , 𝑥 � 𝑔  𝑕 𝑦, 𝑧 is quasiconvex in 𝑦 for each �𝑥 𝑧 𝑔 𝑦, 𝑧 � is 𝑧, 𝑥 𝑧 � 0 ⇒ 𝑔 𝑦 � sup �∈� quasiconvex

Operations that preserve quasiconvexity  Composition 𝑕: 𝐒 � → 𝐒 is quasiconvex, ℎ: 𝐒 → 𝐒 is  nondecreasing ⇒ 𝑔 � ℎ ◦ 𝑕 is quasiconvex 𝑔: 𝐒 � → 𝐒 is quasiconvex ⇒ 𝑕 𝑦 � 𝑔�𝐵𝑦 � 𝑐� is  quasiconvex 𝑔: 𝐒 � → 𝐒 is quasiconvex ⇒ 𝑕 𝑦 � 𝑔� ���� � � ��� � is  quasiconvex, dom 𝑕 � �𝑦|𝑑 � 𝑦 � 𝑒 � 0, �𝐵𝑦 � 𝑐�/�𝑑 � 𝑦 � 𝑒� ∈ dom 𝑔�  Minimization 𝑔�𝑦, 𝑧� is quasicovex in 𝑦 and 𝑧, 𝐷 is a convex set  �∈� 𝑔�𝑦, 𝑧� is quasiconvex ⇒ 𝑕 𝑦 � inf

Outline  The Conjugate Function  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities  Summary

Log-concave and log-convex functions  Definition � is  concave (convex) is log-concave (convex)  Condition � is log-  concave � ���

Examples � � is  log-concave � is log-  �� convex, is log-concave � � �� � /� is log-concave  �� �� � ��� �� is log-convex for  � ��� � � and �� � are log-concave on �� 

Properties  Twice differentiable log convex/concave functions  𝑔 is twice differentiable, dom 𝑔 is convex  𝛼 � log 𝑔�𝑦� � � � � � 𝛼 � 𝑔 𝑦 � � � � 𝛼𝑔 𝑦 𝛼𝑔 𝑦 �  𝑔 is log convex ⇔ 𝑔 𝑦 𝛼 � 𝑔 𝑦 ≽ 𝛼𝑔 𝑦 𝛼𝑔 𝑦 �  𝑔 is log concave ⇔ 𝑔 𝑦 𝛼 � 𝑔 𝑦 ≼ 𝛼𝑔 𝑦 𝛼𝑔 𝑦 �

Outline  The Conjugate Function  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities  Summary

Convexity with respect to a generalized inequality -convex  � is a proper cone with associated  generalized inequality � � is -convex if �  � � � is − convex if  om �

Examples  Componentwise Inequality �  � � is convex with respect to �  componentwise inequality  Each � is a convex function

Examples  Matrix Convexity � is convex with respect to �  matrix inequality 𝑔 𝜄𝑦 � 1 � 𝜄 𝑧 ≼ 𝜄𝑔 𝑦 � 1 � 𝜄 𝑔�𝑧� ��� is matrix convex �  � is matrix convex on �� � for  or , and matrix concave for

Convexity with respect to generalized inequalities  Dual characterization of -convexity A function 𝑔 is (strictly) 𝐿 -convex ⇔ For every  𝑥 ≽ � ∗ 0 , the real-valued function 𝑥 � 𝑔 is (strictly) convex in the ordinary sense.  Differentiable -convex functions A differentiable function 𝑔 is 𝐿 -convex ⇔ dom 𝑔  is convex, ∀𝑦, 𝑧 ∈ dom 𝑔, 𝑔 𝑧 ≽ � 𝑔 𝑦 � 𝐸𝑔 𝑦 𝑧 � 𝑦 A differentiable function 𝑔 is strictly 𝐿 -convex  ⇔ dom 𝑔 is convex, ∀𝑦, 𝑧 ∈ dom 𝑔, 𝑦 � 𝑧, 𝑔 𝑧 ≻ � 𝑔 𝑦 � 𝐸𝑔�𝑦��𝑧 � 𝑦�

Convexity with respect to generalized inequalities  Composition theorem 𝑕: 𝐒 � → 𝐒 � is 𝐿 -convex, ℎ: 𝐒 � → 𝐒 is convex, and  � (the extended-value extension of ℎ ) is 𝐿 - ℎ nondecreasing ⇒ ℎ ◦ 𝑕 is convex.  Example 𝑕: 𝐒 ��� → 𝐓 � , 𝑕 𝑌 � 𝑌 � 𝐵𝑌 � 𝐶 � 𝑌 � 𝑌 � 𝐶 � 𝐷 is  convex, where 𝐵 ≽ 0, 𝐶 ∈ 𝐒 ��� and 𝐷 ∈ 𝐓 � ℎ: 𝐓 � → 𝐒, ℎ 𝑍 � � log det��𝑍� is convex and  � increasing on dom ℎ � �𝐓 �� 𝑔 𝑌 � � log det���𝑌 � 𝐵𝑌 � 𝐶 � 𝑌 � 𝑌 � 𝐶 � 𝐷�� is  convex on dom 𝑔 � �𝑌 ∈ 𝐒 ��� |𝑌 � 𝐵𝑌 � 𝐶 � 𝑌 � 𝑌 � 𝐶 � 𝐷 ≺ 0�

Monotonicity with respect to a generalized inequality � is a proper cone with  associated generalized inequality � � is -nondecreasing if  � � is -increasing if  �

Summary  The Conjugate Function  Definitions, Basic properties  Quasiconvex Functions  Log-concave and Log-convex Functions  Convexity with Respect to Generalized Inequalities