Reducing the Arity in Unbiased Black-Box Complexity Benjamin Doerr , - PowerPoint PPT Presentation

Reducing the Arity in Unbiased Black-Box Complexity Benjamin Doerr , Carola Winzen Max-Planck-Institut für Informatik, Saarbrücken May 02, 2012 Supported by a Google Fellowship in Randomized Algorithms

Rem inder: Black-Box Com plexity  Allows an abstract view on Randomized Search Heuristics y Algorithm A f ( x , f ( x )) f ( y ) Black-Box = “Oracle”  # queries until an optimum is queried for the first time? Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Rem inder: Black-Box Com plexity Algorithm A f ( x , f ( x )) Black-Box = ( y , f ( y )) “Oracle” Runtim e of A for f : T ( A , f ) Expected number of function evaluations (=calls to the oracle) until an optimal solution is queried for the first time Runtim e of A for F : sup f ∈ F T ( A , f ) Worst runtime of A among all functions f (Unrestricted) Black-Box Com plexity of F : inf A sup f ∈ F T ( A , f ) Best worst-case runtime among all algorithms A Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Drawbacks of the Unrestricted Black-Box Model  NP-hard problems with low black-box complexity  Max-Clique  PARTITION  .....  Most classical test functions have “too low black-box complexities” �  OneMax: Θ ��� � instead of Θ � log � Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Masterm ind-Version of OneMax  Black-Box chooses � ∈ 0,1 �  Algorithm guesses x ∈ 0,1 �  Black-Box answers �� � ��� OneMax Algorithm A Black-Box = 2 “Oracle” Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Drawbacks of the Unrestricted Black-Box Model  NP-hard problems with low black-box complexity  Max-Clique  PARTITION  .....  Most classical test functions have “too low black-box complexities” �  OneMax: Θ ��� � instead of Θ � log �  LeadingOnes: O � log �/log log � instead of Θ � �  ...  Early end of black-box m odels?? Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

The Unbiased Black-Box Model  Revival of black-box studies by Lehre and Witt  Observation: search heuristics sample unbiasedly x Algorithm A f Black-Box = “Oracle” sampled unbiasedly, i.e., in a “fair” way Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

The Unbiased Black-Box Model  Revival of black-box studies by Lehre and Witt  Observation: search heuristics sample unbiasedly  Treat all bit positions and bit values in a “fair way”  Intuitively: “Flip the 5 th bit” “Flip all zeros”  Formally:  queries must be sampled from an unbiased distribution; i.e., a distribution D(.| ..) that satisfies for all x, y � , … . , y � , w and all � ∈ � �  � � � � , … , � � � � � ⨁ � � � ⨁ � , … , � � ⨁ �  � � � � , … , � � � � ���� ��� � �, … , ��� � � Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model Unary Model [LW Gecco 20 10 ] Unary unbiased BBC of OneMax is Ω�� log �� Matched by many unary RSH  Randomized Local Search  (1+1) EA  ( μ � λ ) EAs ( μ, λ� constant Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model Unary Model [LW Gecco 20 10 ] Unary unbiased BBC of OneMax is Ω�� log �� Arities � � � � 2 [DJKLWW Foga 20 11] k-ary unbiased BBC of OneMax is O��/ log ��  Suggests that k-ary RSH are more powerful than unary oned  Not known to be matched by k-ary RSH Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

OneMax in the Unbiased Model Unary Model [LW Gecco 20 10 ] Unary unbiased BBC of OneMax is Ω�� log �� Arities � � � � 2 [DJKLWW Foga 20 11] k-ary unbiased BBC of OneMax is O��/ log �� Arities log � � � � 2 [DW Gecco 20 12] k-ary unbiased BBC of OneMax is O��/�� "log � -arity is as powerful as unrestrictedness” Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Why Should You Care? Nice mathematics  1. 2. Techniques can be applied to other problems 3. Hope: find RSH that are provably faster than basic algorithms Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques 1. Derandomized version of random sam pling technique Probabilistic method at its best: Make cn / log n random guesses For all y, z ∈ 0,1 � with y � z there exits an index i ∈ t such that �� � � � � �� � � � w.h.p. There exists a string-distinguishing sequence � � , … , � � of � � ��/ log � strings Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques 1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string In the FOGA paper, we could handle only blocks of length k , yielding the O( n / log k ) bound Cut board into blocks of length 2 � • Apply random guessing technique to these blocks • • There are �/2 � blocks of length 2 � • Random guessing: O( 2 � / log 2 � )= O( 2 � / k ) guesses each • Total number of guesses: �/2 � O( 2 � / k ) = O( n / k ) Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques 1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string 3. Sim ulating unrestrictedness Using k-ary operators, we can access 2 k -1 bits in an almost unrestricted fashion Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Key techniques 1. Derandomized version of random sam pling technique 2. Block-w ise identification of target string 3. Sim ulating unrestrictedness 4. Storing the fitness values creating unrestricted block “random guess” storing Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

Sum m ary & Future Work Main Result For log � � � � 2 , the k-ary unbiased BBC of OneMax is O��/�� Future Work  Lower bounds!!  Cross-over based algorithms that match the upper bound?  Other BB-models for higher arity algorithms  ..... Winzen: Reducing the Arity in Unbiased Black-Box Complexity.

References  Unrestricted Black-Box Model [DJW06] Droste, Jansen, Wegener Upper and Low er Bounds for Random ized Search Heuristics in Black-box Optim ization ToCS 2006  Unbiased Black-Box Models [LW10] Lehre, Witt Black-Box Search by Unbiased Variation GECCO 2010 [RV11] Rowe, Vose Unbiased Black Box Search Algorithm s GECCO 2011 [DKLW11] Doerr, Kötzing, Lengler, Winzen Black-Box Com plexities of Com binatorial Problem s GECCO 2011  OneMax-Complexities & (Derandomized) Random Sampling [ER63] Erd ős, Rényi On Tw o problem s of Inform ation Theory Magyar Tud. Akad. Mat. Kutató Int. Közl 1963 [AW09] Anil, Wiegand Black-Box Search by Elim ination of Fitness Functions FOGA 09 [DJKLWW11] Doerr, Johannsen, Kötzing, Lehre, Wagner, Winzen Faster Black-Box Algorithm s Through Higher Arity Operators FOGA 11 [DW12a] Doerr, Winzen Reducing the Arity in Unbiased Black-Box Com plexity GECCO 2012 [DW12] Doerr, Winzen Playing Masterm ind With Constant-Size Mem ory STACS 2012 [...] Many more!  LeadingOnes-Complexities [DW11] Doerr, Winzen Breaking the O(n log n) Barrier of LeadingOnes EA 2011  Jump k -Complexities & Black-Box Complexity of PARTITION [DKW11] Doerr, Kötzing, Winzen Too Fast Unbiased Black-Box Algorithm s FOGA 11 Winzen: Reducing the Arity in Unbiased Black-Box Complexity.