Zero-Convex Functions, Perturbation Resilience, and Subgradient - PowerPoint PPT Presentation

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence infinitely many sets are allowed Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence infinitely many sets are allowed general control sequence (beyond cyclic and almost cyclic) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence infinitely many sets are allowed general control sequence (beyond cyclic and almost cyclic) Convergence: Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence infinitely many sets are allowed general control sequence (beyond cyclic and almost cyclic) Convergence: global and weak, sometimes also strong Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

Main results: a schematic description Introducing and discussing in a quite detailed way the class of zero-convex functions , a rich class of functions holding a promising potential Discussing the SSP method for solving the CFP in a general setting: zero-convex functions domain: closed and convex subset of a real Hilbert space Certain perturbations are allowed without losing convergence infinitely many sets are allowed general control sequence (beyond cyclic and almost cyclic) Convergence: global and weak, sometimes also strong Computational simulations: for a problem in molecular biology Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 7 / 24

The class of zero-convex functions Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 8 / 24

The class of zero-convex functions Definition H is a real Hilbert space. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 8 / 24

The class of zero-convex functions Definition H is a real Hilbert space. Ω ⊆ H is nonempty and convex. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 8 / 24

The class of zero-convex functions Definition H is a real Hilbert space. Ω ⊆ H is nonempty and convex. Given g : Ω → R , its 0-level-set is g ≤ 0 = { x ∈ Ω | g ( x ) ≤ 0 } . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 8 / 24

The class of zero-convex functions Definition H is a real Hilbert space. Ω ⊆ H is nonempty and convex. Given g : Ω → R , its 0-level-set is g ≤ 0 = { x ∈ Ω | g ( x ) ≤ 0 } . g is said to be zero-convex at the point y ∈ Ω if there exists a vector t ∈ H (called a 0 -subgradient of g at y ) satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 8 / 24

0 -convex functions (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential the Clarke subdifferential Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential the Clarke subdifferential the Quasi-subdifferential Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential the Clarke subdifferential the Quasi-subdifferential Mordukhovich’s Subdifferential Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential the Clarke subdifferential the Quasi-subdifferential Mordukhovich’s Subdifferential etc. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions (Cont.) The set of all 0-subgradients of g at y is called the zero-subdifferential of g at y and denoted by ∂ 0 g ( y ). A function g satisfying ∀ x ∈ g ≤ 0 . g ( y ) + � t , x − y � ≤ 0 for all y ∈ Ω will be called 0 -convex . Other notions of subdifferentials exist in the literature, e.g., the standard subdifferential the Clarke subdifferential the Quasi-subdifferential Mordukhovich’s Subdifferential etc. Our one seems to be new. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 9 / 24

0 -convex functions: geometric illustration Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 10 / 24

0 -convex functions: geometric illustration The hyperplane M = { x ∈ H : � t , x − y � = − g ( y ) } separates g ≤ 0 and y : Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 10 / 24

0 -convex functions: geometric illustration The hyperplane M = { x ∈ H : � t , x − y � = − g ( y ) } separates g ≤ 0 and y : Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 10 / 24

Zero-convex functions: main characterization Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 11 / 24

Zero-convex functions: main characterization Proposition If g is zero-convex, then its zero-level-set g ≤ 0 is convex. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 11 / 24

Zero-convex functions: main characterization Proposition If g is zero-convex, then its zero-level-set g ≤ 0 is convex. If g ≤ 0 is closed and convex, then g is zero-convex. In fact, we have a formula for the 0-subgradients using separating hyperplanes. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 11 / 24

Zero-convex functions: examples Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

Zero-convex functions: examples Example Any convex function g : R n → R Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

Zero-convex functions: examples Example Any convex function g : R n → R Example Any nonpositive function g is 0-convex at each y with t = 0. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

Zero-convex functions: examples Example Any convex function g : R n → R Example Any nonpositive function g is 0-convex at each y with t = 0. Example Any lower semiconrinuous quasiconvex function is zero-convex. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

Zero-convex functions: examples Example Any convex function g : R n → R Example Any nonpositive function g is 0-convex at each y with t = 0. Example Any lower semiconrinuous quasiconvex function is zero-convex. Such functions frequently appear in generalized convexity theory. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

Zero-convex functions: examples Example Any convex function g : R n → R Example Any nonpositive function g is 0-convex at each y with t = 0. Example Any lower semiconrinuous quasiconvex function is zero-convex. Such functions frequently appear in generalized convexity theory. In particular, certain quadratic functions in subsets of R m (economics) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 12 / 24

0 -convex functions: additional examples (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 13 / 24

0 -convex functions: additional examples (Cont.) Example Multivariate polynomials: Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 13 / 24

0 -convex functions: additional examples (Cont.) Example Multivariate polynomials: e.g., g : R 2 → R defined by g ( x 1 , x 2 ) = x 2 1 + x 2 2 − x 4 1 x 4 2 + x 6 1 x 6 2 / 4 − 0 . 3 . This g is zero-convex but not quasiconvex. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 13 / 24

0 -convex functions: additional examples (Cont.) Example Multivariate polynomials: e.g., g : R 2 → R defined by g ( x 1 , x 2 ) = x 2 1 + x 2 2 − x 4 1 x 4 2 + x 6 1 x 6 2 / 4 − 0 . 3 . This g is zero-convex but not quasiconvex. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 13 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. the distance d ( p , A ) between p and A is positive. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. the distance d ( p , A ) between p and A is positive. g : Ω → R is defined by g ( x ) := d ( x , p ) − d ( x , A ) ∀ x ∈ Ω . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. the distance d ( p , A ) between p and A is positive. g : Ω → R is defined by g ( x ) := d ( x , p ) − d ( x , A ) ∀ x ∈ Ω . g is zero-convex but usually not quasiconvex Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. the distance d ( p , A ) between p and A is positive. g : Ω → R is defined by g ( x ) := d ( x , p ) − d ( x , A ) ∀ x ∈ Ω . g is zero-convex but usually not quasiconvex g ≤ 0 is the Voronoi cell of p with respect to A . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

0 -convex functions: additional examples (Cont.) Example The Voronoi function: p ∈ Ω and A ⊆ H are given. the distance d ( p , A ) between p and A is positive. g : Ω → R is defined by g ( x ) := d ( x , p ) − d ( x , A ) ∀ x ∈ Ω . g is zero-convex but usually not quasiconvex g ≤ 0 is the Voronoi cell of p with respect to A . Remark: Voronoi diagrams appear in numerous places in science and technology and have diverse applications. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 14 / 24

The algorithm Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 15 / 24

The algorithm Algorithm The Sequential Subgradient Projections (SSP) Method with Perturbations Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 15 / 24

The algorithm Algorithm The Sequential Subgradient Projections (SSP) Method with Perturbations Initialization: x 0 ∈ Ω is arbitrary. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 15 / 24

The algorithm Algorithm The Sequential Subgradient Projections (SSP) Method with Perturbations Initialization: x 0 ∈ Ω is arbitrary. Iterative Step: g i ( n ) ( x n )  � � P Ω x n − λ n � t n � 2 t n + b n , if g i ( n ) ( x n ) > 0 ,  x n +1 = if g i ( n ) ( x n ) ≤ 0 , x n ,  Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 15 / 24

The algorithm (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 16 / 24

The algorithm (Cont.) λ n = relaxation parameters ∈ ( ǫ 1 , 2 − ǫ 2 ), Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 16 / 24

The algorithm (Cont.) λ n = relaxation parameters ∈ ( ǫ 1 , 2 − ǫ 2 ), t n = 0-subgradients ∈ ∂ 0 g i ( n ) ( x n ) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 16 / 24

The algorithm (Cont.) λ n = relaxation parameters ∈ ( ǫ 1 , 2 − ǫ 2 ), t n = 0-subgradients ∈ ∂ 0 g i ( n ) ( x n ), b n = error terms. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 16 / 24

Algorithm: geometric illustration when Ω = H Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 17 / 24

Algorithm: geometric illustration when Ω = H M n =an arbitrary separating (closed) hyperplane between x n and g ≤ 0 i ( n ) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 17 / 24

Algorithm: geometric illustration when Ω = H M n =an arbitrary separating (closed) hyperplane between x n and g ≤ 0 i ( n ) , m n =the projection of x n on M n . Then: x n +1 = (1 − λ n ) x n + λ n m n + b n . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 17 / 24

Algorithm: geometric illustration when Ω = H M n =an arbitrary separating (closed) hyperplane between x n and g ≤ 0 i ( n ) , m n =the projection of x n on M n . Then: x n +1 = (1 − λ n ) x n + λ n m n + b n . Figure: Illustration when 0 < λ n < 1 and Ω = H . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 17 / 24

The algorithm (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 18 / 24

The algorithm (Cont.) Control Sequence: more general than cyclic and almost cyclic Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 18 / 24

Conditions for convergence Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 19 / 24

Conditions for convergence Condition � � g ≤ 0 C = C j = � = ∅ . j j ∈ J Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 19 / 24

Conditions for convergence Condition � � g ≤ 0 C = C j = � = ∅ . j j ∈ J Condition Each function g j is 0-convex, uniformly continuous on closed and bounded subsets, and weakly sequential lower semicontinuous. Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 19 / 24

Conditions for convergence (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 20 / 24

Conditions for convergence (Cont.) Condition For a fixed M > d ( x 0 , C ), the following inequality is satisfied ǫ 1 ǫ 2 h 2 � � n � b n �≤ min M , , ∀ n ∈ N , 2(5 M + 4 h n ) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 20 / 24

Conditions for convergence (Cont.) Condition For a fixed M > d ( x 0 , C ), the following inequality is satisfied ǫ 1 ǫ 2 h 2 � � n � b n �≤ min M , , ∀ n ∈ N , 2(5 M + 4 h n ) where � g i ( n ) ( x n ) / � t n � , if g i ( n ) ( x n ) > 0 , h n = 0 , if g i ( n ) ( x n ) ≤ 0 . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 20 / 24

Conditions for convergence (Cont.) Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 21 / 24

Conditions for convergence (Cont.) Condition There exists a K > 0 such that � t n � ≤ K for all n ∈ N . Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 21 / 24

Conditions for convergence (Cont.) Condition There exists a K > 0 such that � t n � ≤ K for all n ∈ N . Holds in many cases (examples mentioned in the paper). Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 21 / 24

The convergence theorem Censor, Reem (Haifa, Technion) 0-convex, perturbation, subgrad. proj. July 2016 22 / 24