Minimizing Submodular Functions Satoru Iwata (RIMS, Kyoto - PowerPoint PPT Presentation

Minimizing Submodular Functions Satoru Iwata (RIMS, Kyoto University)

Outline • Submodular Functions Examples Discrete Convexity • Submodular Function Minimization Min-Max Theorem Combinatorial Algorithms • Applications • Conclusion

Submodular Functions : V Finite Set → ∀ , ⊆ V : 2 R X Y V f + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) f X f Y f X Y f X Y • Cut Capacity Functions • Matroid Rank Functions • Entropy Functions X Y V

Cut Capacity Function ∑ κ = Cut Capacity ( ) { ( ) | : leaving } X c a a X ≥ ( ) 0 c a t s X Max Flow Value ＝ Min Cut Capacity

) Y Whitney (1935) ( | ρ X ≤ Matroid Rank Functions | ≤ ) ) X Submodular X ( ρ ( ρ ⇒ , V Y ⊆ ⊆ X : ρ ∀ X ] U Matrix Rank Function X , U [ A rank V X X = ) = ( ρ A

Entropy Functions Information φ = ( ) 0 h Sources ( X ) : h Entropy of the Joint Distribution + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) h X h Y h X Y h X Y X ≥ 0 Conditional Mutual Information

Positive Definite Symmetric Matrices X φ = ( ) 0 f = ( ) log det [ ] f X A X [ X ] = A A Ky Fan’s Inequality + ≥ ∩ + ∪ ( ) ( ) ( ) ( ) f X f Y f X Y f X Y Extension of the Hadamard Inequality ∏ ≤ det A A ii ∈ i V

) Y ∪ V X ( f + ) Y Y Discrete Convexity ∩ X ( f ≥ ) Y ( f + X ) X ( f Convex Function y x

Discrete Convexity Lovász (1983) V = { , , } a b c ˆ { c , } b ： Linear Interpolation f ˆ { , } a b f ： Convex { b } { c } f ： Submodular { a } φ Discrete Convex Analysis Murota (2003)

Submodular Function Minimization φ = ( ) 0 f Assumption: X Minimization Evaluation Algorithm Oracle ( X ) f ⊆ min{ ( ) | } ? f Y Y V Minimizer

Submodular Function Minimization Grötschel, Lovász, Schrijver (1981, 1988) Ellipsoid Method γ 5 ( log ) Cunningham (1985) O n M γ + γ 7 8 7 ( ) O n n ( log ) O n n Iwata, Fleischer, Fujishige (2000) Schrijver (2000) Fleischer, Iwata (2000) Iwata (2002) Fully Combinatorial Iwata (2003) Orlin (2007) γ + γ + 4 5 5 6 ( ) (( ) log ) O n n O n n M Iwata, Orlin (2009)

Base Polyhedra = → R R V { | } x V ∑ = ( ) ( ) x Y x v ∈ v Y Submodular Polyhedron = ∈ ∀ ⊆ ≤ R V ( ) { | , , ( ) ( )} P f x x Y V x Y f Y Base Polyhedron = ∈ = ( ) { | ( ), ( ) ( )} B f x x P f x V f V

Edmonds (1970) Greedy Algorithm Shapley (1971) ( v ) L v v v v 1 2 n v ∈ = − − ( ) V ( ) ( ( )) ( ( ) { }) y v f L v f L v v : y Extreme Base ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ L 1 0 0 ( ) ( ( )) y v f L v 1 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ O M 1 1 ( ) ( ( )) y v f L v ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = 2 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ M M O M M 0 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ L ⎣ ⎦ ( ) ( ( )) 1 1 1 ⎣ ⎦ ⎣ ⎦ y v f L v n n

Min-Max Theorem Edmonds (1970) Theorem − = ∈ min ( ) max{ ( ) | ( )} f Y x V x B f ⊆ Y V − = ( ) : min{ 0 , ( )} x v x v − ≤ ≤ ( ) ( ) ( ) x V x Y f Y

Combinatorial Approach y L ∈ ( f ) Extreme Base B Convex Combination x ∑ = λ L y L ∈ Λ x L Cunningham (1985) γ 6 ( log ) O n M nM = max | ( ) | M f X X ⊆ V

Combinatorial Approach x ∑ = λ L y L L ∈ Λ L : y Extreme Base L ≤ ∀ ∈ ( ) 0 , x v v T ≥ ∀ ∉ ( ) 0 , x v v T T ∴ = = ∀ ∈ Λ ( ) ( ) x T f T ( ) ( ), . y L T f T L − = = ( ) ( ) ( ) x V x T f T : T Minimizer

Distance Labeling x ∑ = λ L y u v L L ∈ Λ L : y Extreme Base L Labeling u → ∈ Λ Ζ : ( ) d L V L ≤ ⇒ = ∀ ∈ Λ ( ) 0 ( ) 0 , . x u d u L L ⇒ ≤ p ( ) ( ). u v d u d v L L L − ≤ ∀ ∈ Λ ∀ ∈ | ( ) ( ) | 1 , , , . d u d u L K u V L K

Distance Labeling = ∈ Λ ( ) : min{ ( ) | } d u d u L min L Gap of Level k v ∃ ∈ = , ( ) . v V d v k min ∀ ∈ ≠ − , ( ) 1 . v V d v k min < ≥ k k ≥ ⇒ ∉ ∀ . f ( ) , : d v k v Y Y Minimizer of min

Distance Labeling T ≠ φ ⇒ X \ T = < ∪ ( ) ( ) ( ) f T x T x X T v ≤ ∪ ( ). f X T ∴ > ∩ ( ) ( ). f X f X T < ≥ k k ≥ ⇒ ∉ ∀ . f ( ) , : d v k v Y Y Minimizer of min

Iteration η δ = η = ∈ : : max{ ( ) | } x v v V 4 n μ − μ > δ ∀ ∈ Find such that | ( ) | , . x v v V η δ δ 0 μ = ∈ ∈ Λ > μ : min{ ( ) | , , ( ) } l d u u V L x u L ∈ ∈ Λ = Select and such that ( ). u V L l d u L

New Permutation μ η 0 L μ New_Permut ation ( , , ) ∉ l ⎧ ( ) ( ) d v v R = L ⎨ ( ) : d v ′ + ∈ L ⎩ ( ) 1 ( ) = ∈ = d v v R { | , ( ) } S v v V d v l L L = ∈ > μ = { | , ( ) }, \ R v v S x v Q S R L L ′ Q R

Push Operation ′ Push ( , ) L L μ η 0 ⎧ ⎫ − μ ( ) x v β = ∈ ≠ ⎨ ⎬ : min | , ( ) ( ) v S y v y v ′ − L L ⎩ ( ) ( ) ⎭ y v y v ′ L L α = λ α = λ β : min{ , } Saturating L L α = β λ = α Nonsaturating ′ : L λ = λ − α L : L

Potential Function ∑ + Φ = 2 ( ) ( ) x x v + = ( ) max{ ( ), 0 } x v x v ∈ v V x ′ x Nonsaturating Push Moves to ′ ⇒ Φ − Φ ≥ Φ 3 ( ) ( ) ( ) / 16 x x x n Φ x ≤ 2 Initially, ( ) . nM 3 ( log ) After Nonsaturating Pushes, O n nM Φ x < 2 ( ) 1 / . n η < η = ∈ 1 n / . : max{ ( ) | } x v v V

Algorithm Termination → Z V : 2 f η = ∈ : max{ ( ) | } x v v V 1 η < n ⇒ : V Maximal Minimizer − ≥ > − = − Q ) ( ) ( ) ( ) 1 ( ) 1 . f X x V x V f V Λ = 3 | | ( log ) O n nM

Running Time Bound ∑∑ Γ Λ = − ( ) [ ( )] n d L v ∈ Λ ∈ L v V Γ ( Λ A Saturating Push Decreases ). | Λ A Nonsaturating Push Increases by One | 2 Γ ( Λ . n and by at Most ) Γ ( Λ 5 ( log ) ) Total Increase of O n nM 5 ( log ) # Saturating Pushes O n nM γ 6 ( log ) O n nM Running Time

Improvements γ 6 ( log ) • A Simple Algorithm O n nM • A Faster Weakly Polynomial Algorithm γ + 4 5 (( ) log ) O n n nM • A Strongly Polynomial Algorithm γ + 5 6 (( ) log ) O n n n • A Fully Combinatorial Algorithm γ + 7 8 (( ) log ) O n n n

The Minimum-Norm Base 2 Minimize x ∗ ∈ x subject to ( ) x B f Fujishige (1984) Theorem ∗ : opt. sol. x ∗ = < : { | ( ) 0 } X v x v Minimal Minimizer − ∗ v = ≤ : { | ( ) 0 } X v x Maximal Minimizer 0 Nagano (2007) Remark ∑ ( ( )) g x v The minimum-norm base minimizes ∈ ( f ) in for any convex function B v V . g

Evacuation Problem （ Dynamic Flow ） T Hoppe, Tardos (2000) S ( a ) : c Capacity τ ( a ) : Transit Time ( v ) : b Supply/Demand X ∩ ( X ) : o T \ S X Maximum Amount of Flow from to . ≤ ∀ ⊆ ∪ ( ) ( ), b X o X X S T Feasible

X R ) Y Multiterminal Source Coding , X ( H ) ( X H ) Y | X ( H ) ) ) Y X Y ( Y R , X | H Y ( ( H H Decoder Slepian, Wolf (1973) X Y R R Encoder Encoder Y X

Multiclass Queueing Systems Service Time Arrival Interval Server μ λ Service Rate Arrival Rate i i Multiclass M/M/1 Control Policy Preemptive

Performance Region : s Expected Staying Time of a Job in j j ∑ ρ = λ μ ρ < : , 1 i i i i : s Achievable ∈ V i s j : s R Y ∑ ρ μ i i ∑ ρ ≥ ∀ ⊆ ∈ i X , S X V ∑ − ρ i i 1 ∈ i X i ∈ i X Coffman, Mitrani (1980) s i

A Class of Submodular Functions ∈ R Itoko & I. (2005) V , , x y z + : h Nonnegative, Nondecreasing, Convex = − X ⊆ ( ) ( ) ( ) ( ( )) f X z X y X h x X ( ) V Submodular ρ = = ρ ∑ i : y ρ μ : z S μ i i i i i i ∑ i ρ ≥ ∀ ⊆ ∈ i X , S X V ∑ 1 − ρ i i 1 = ρ = 1 : ( ) : ∈ x h x i X i − i i ∈ x i X

Zonotope in 3D z = ( ) ( ( ), ( ), ( )) w X x X y X z X = ⊆ conv { ( ) | } Z w X X V Zonotope ~ = − ( , , ) ( ) f x y z z yh x ⊆ min{ ( ) | } f X X V ~ = min{ ( , , ) | ( , , ) : Lower Extreme Point of } f x y z x y z Z ~ ( , , ) f x y z Remark: is NOT concave!

Line Arrangement β α + β = x y z i i i α Enumerating All the Cells Topological Sweeping Method 2 ( ) O n Edelsbrunner, Guibas (1989)